PLoS Computational Biology

Yelmen, Burak;Decelle, Aurélien;Boulos, Leila Lea;Szatkownik, Antoine;Furtlehner, Cyril;Charpiat, Guillaume;Jay, Flora

doi: 10.1371/journal.pcbi.1011584pmid: 37903158

Introduction Machine learning is an important staple in modern genomic studies. There have been numerous applications in demographic inference [1], detecting natural selection [2], genome-wide association studies [3] and functional genomics [4], many of which became state of the art [5, 6]. In the recent few years, generative machine learning approaches for the genomics field have also begun to gain research interest thanks to algorithmic advances and widespread availability of computational resources [7–11]. Broadly speaking, generative machine learning involves the utilisation of generative models which are trained to model the distribution of a given dataset so that new data instances with similar characteristics to the real data can be sampled from this learned distribution. Especially since the introduction of generative adversarial networks (GANs) in the preceding decade [12], generative modelling has become a widely-researched subject with a diverse scope of applications such as image and text generation [13, 14], dimensionality reduction [15] and imputation [16]. The amount of genetic data, both in sequence and SNP array formats, is now increasing at an unmatched rate, yet its accessibility remains relatively low. The main reason for this is the crucial protocols prepared to protect the privacy of donors. Genetic data belongs to a special category similar to private medical records in the General Data Protection Regulation (GDPR) [17] and is defined as protected health information in the Health Insurance Portability and Accountability Act (HIPAA) [18]. Although these protective measures are vital, they create an accessibility issue for researchers who must go through these protocols and, in many circumstances, might have to commit to collaborations to conduct research or simply test ideas. In our previous study, we introduced the concept of high quality artificial genomes (AGs) created by generative models as a possible future solution for this problem and as an alternative to other methods such as differential privacy [19] and federated learning [20]. We demonstrated that AGs can mimic the characteristics of real data such as allele frequency distribution, haplotypic integrity and population structure, and can be used in applications such as genomic imputation and natural selection scans [21]. However, our previous models lacked the capacity to generate large-scale genomic regions due to the computational requirements caused by the high number of parameters defining the fully connected architectures. In this study, we present two novel implementations better adapted to large sequential genomic data: (i) generative adversarial networks with convolutional architecture and Wasserstein loss (WGAN) [22], and (ii) restricted Boltzmann machines with conditional training (CRBM) [23] used together with an out-of-equilibrium procedure [24]. In more detail, we implemented a WGAN with gradient penalty (WGAN-GP) [25] which involved a deep generator and a deep critic architecture, multiple noise inputs at different resolutions for the generator [26], trainable location-specific vectors for preserving the positional information [27], residual blocks to prevent vanishing gradients [28] and packing for the critic to eliminate mode collapse [29]. For the CRBM, we used the more efficient out-of-equilibrium training scheme differently from our previous study and developed a novel procedure for conditionally training multiple RBMs based on shared genomic regions [30]. We assessed the AGs generated by these models with multiple statistics measuring data quality and possible privacy leakage. First, we compared the new models with the ones from [21] using the same 10,000 SNPs extracted from the 1000 Genomes human dataset [31] and then trained these models with a larger 65,535-SNP dataset to generate long sequence AGs. We performed multiple tests to evaluate (i) the quality of the generated haplotypes via allele frequency spectrum, linkage disequilibrium, population structure and complex haplotypic integrity analyses such as 3-point correlations and distribution of k-mer motifs; and (ii) privacy preservation via distance-based metrics, membership inference attacks and overfitting/underfitting scores. Materials and methods Data The two sets of 1000 Genomes [31] data we used include: (i) 10,000 SNPs from chromosome 15 (between 27,379,578—29,625,035 base pairs, ∼2 megabase pairs) identical to the ones picked by [21], and (ii) 65,535 SNPs from chromosome 1 between 534,247—81,813,279 base pairs, ∼80 megabase pairs) within the Omni 2.5 genotyping array framework. Further downsampling of the array framework was performed to create a dataset with a reasonable SNP number for faster training trials and the specific number of 65,535 SNPs was decided to be in the form of (2n—1) for easy implementation of convolutional scaling. For the 10,000-SNP dataset, we used padding with zeros to match the (2n—1) form. Both datasets have the same 2504 individuals and 5008 phased haplotypes used for training the models. The data format is the same as [21] where the rows are phased haplotypes and columns are positions which hold alleles represented by 0 (reference) and 1 (alternative). WGAN implementation We implemented a Wasserstein GAN with gradient penalty (WGAN-GP) consisting of a critic which estimates the earth mover’s distance between real and generated data distributions, and a generator which generates new genomic data from Gaussian noise (Fig 1). Unlike the discriminator in naive GAN which performs a classification task, the critic provides a “realness” score (an approximation of Earth-mover’s distance) for generated and real samples. WGAN objective function to be minimized by the generator and maximized by the critic is as follows: where C is the critic, G is the generator, x is real data point and z is Gaussian noise. The critic function C must be Lipschitz continuous; thus, the original study relied on weight clipping to enforce an upper bound for the gradient [22]. However, we designed a WGAN with gradient penalty (WGAN-GP) rather than weight clipping, as GP was shown to be a better alternative [25]. In our implementation, the critic uses convolution layers whereas the generator uses convolution and transposed convolution layers (Fig 1A). Both the generator and the critic have trainable location-specific vectors as additional channels at every block except for the residual blocks. These vectors, similar to the ones integrated by [27] in their autoencoder, consist of trainable variables that allow the models to preserve positional information which would otherwise diminish due to the invariance of convolution operations. In addition to this, the generator has two noise channels at every block. This gives the generator flexibility to decide at which depth the mapping to latent space can occur. We also implemented “packing” to overcome mode collapse by adjusting the discriminator to take multiple samples as input [29]. In our implementation, the input sample number for the discriminator was set to 3 intuitively, to match the 3 main population modes (Africa, West Eurasia, Asia) observable in the 1000 Genomes data. The initial length of the input for the generator is 4, which is gradually transformed into features of larger sizes until it reaches the sequence size of 65,535 or 16,383 in the output (Fig 1B). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Wasserstein GAN (WGAN) model for the 65,535-SNP dataset. a) Representation of the generator, critic and residual blocks. Channel dimensions are not proportional and do not reflect the real implementation. The generator block has one trainable location-specific variable (blue) and two latent space vectors (red) as additional channels concatenated to the input. The critic block has one trainable location-specific variable (blue) as an additional channel concatenated to the input. b) Architecture of the WGAN model. White rectangles correspond to generic generator and critic blocks whereas grey rectangles correspond to generic residual blocks. Numbers in parentheses above blocks show channels and length, respectively (C, L). Dotted connections are residual connections where the input value is added to the output value of the block before passing to the next block. Yellow input and output blocks differ from the generic ones for proper dimension adjustments. https://doi.org/10.1371/journal.pcbi.1011584.g001 The generator layers are followed by batch normalisation and leaky ReLU activation (alpha = 0.01) except for the final layer which has a sigmoid activation. The critic layers are followed by instance normalisation and leaky ReLU activation (alpha = 0.01) except for the final layer which is a fully connected layer with no activation. We used Adam optimizer to train both the generator and the critic with a learning rate of 0.0005 and β1, β2 = (0.5, 0,9). β1 was set to 0.5 as suggested by [13]. For each batch training of the generator, the critic was trained 10 times as suggested by the authors of the original WGAN [22]. We assessed the outputs of the generator at each epoch during training via PCA. We stopped training when generated and real genome clusters visually overlapped in PC space (components 1 to 4). In the case of 65,535-SNP data, this initial training was not sufficient to reach a good overlap of higher degree principal components, thus, we performed a second brief training (up to 200 epochs) with 10-fold lower generator learning rate (0.00005). Our WGAN architecture for the 10,000-SNP data was conceptually the same as for the 65,535-SNP data, but shallower with fewer blocks. All WGAN models were coded with python-3.9 and pythorch-1.11 [32]. The detailed python scripts can be accessed at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. RBM implementation An RBM is a generative stochastic neural network [33] defined as a probability distribution over a set of visible units, , representing SNPs in our case, and hidden units, , where both type of variables interact via a pairwise weight matrix and the local biases and help to adjust the mean value of each unit: In this study, we used binary units {0, 1} for both the visible and hidden nodes, and the number of hidden units was chosen to be about the same order as the number of visible ones. The likelihood of such model is given by: where the index m is indexing the samples of the dataset (M being the total number of samples) and Z is the normalization constant, or partition function, of the probability distribution. Learning an RBM consists in maximising this likelihood using gradient descent in order to optimize the weights and biases w, θ and η. In our implementation, the training was based on the out-of-equilibrium method [24, 34]. The main difference with the more conventional learning is that the sampling which is done to compute the correlation of the model is performed in a very precise way: a random initial condition is chosen amongst a certain probability distribution p0(x) (kept fixed during the learning), and a fix (all along learning) number of Monte Carlo steps is done during the training. When using this particular training procedure, in order to sample new data, it is enough to generate the Monte Carlo chains following the same dynamical process: same initial conditions p0(x) and same number of MC steps. In the provided experiments, the initial conditions were chosen uniformly at random (each unit having equal probability to be 0 or 1) and the number of MC steps was either 100 (for the 65,535-SNP dataset) or 200 (for the 10,000-SNP dataset) but the qualitative results were mostly not affected. The learning rate of the model was chosen such that the eigenvalues of the learning weight matrix are smoothly increasing during the first epochs from almost zero to values of ∼O(1). For the generation of very long sequences, we designed a novel procedure based on conditional-RBMs (CRBMs) [23, 35] (Fig 2). The CRBM consists in the learning of correlation patterns conditionally to some input variables. Hence, instead of considering all the dataset X, we separate it into two parts X = XP ∪ XI, and the CRBM will learn to generate XI (the inferred variables) based on XP (the pinned variables). Therefore, we design a gradient that will learn how to generate the variables XI given that we provide the variables Xp. In practice, denoting as the variables to be inferred and as the pinned ones, we will maximize the following quantity: Therefore, for each sample of the dataset, this construction is similar to a classical RBM with additional biases for the hidden nodes which depend on the pinned variables. During the learning, we infer the parameters of the model such that when showing a configuration of pinned variables XP, the model will generate a sample that is correlated to it accordingly. This conditional model can be used to learn very long chains of variables. Let us consider that we have a dataset with 100,000 input variables. We can learn initially a regular RBM on the first 10,000 input variables. In parallel, we can also learn a conditional RBM on the input variables si with i in [5,000:15,000] using the first 5,000 variables as pinned variables. Therefore, this second RBM will, given 5,000 pinned variables, learn how to generate the 5,000 following ones. The same procedure is repeated with various input sets, always using the first 5,000 input nodes as pinned variables. Once learning is completed for all the RBMs, we can proceed using the following sequential method to generate new data: Use the first RBM (non-conditional) to generate the first 10,000 input. Use the first CRBM to generate the next [10,000:15,000] inputs. To do that, we use the generated nodes [5,000:10,000] as pinned variables and generate the rest. We follow the same procedure until we finally generate the whole 100,000 variables. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Illustration of the learning and sampling of a large sequence using a “classical” and a conditional RBM (CRBM). Initially, we train RBM1 (left) and RBM2 (right) in parallel. Both RBMs are essentially trained in a similar manner: random inputs are drawn and k MC steps are performed before computing the gradient and updating the weights using gradient descent. The difference for the CRBM (RBM2) is that half of the variables in the visible layer are pinned (crossed squares) to the real data during training while the rest is generated conditionally on these pinned variables. After training both machines, we can sample a complete new sequence. To do so, we start from random input and perform k MC steps to generate the first part of the sequence (light yellow-red) using RBM1. Then, we use half of this generated sequence (light red) as the pinned visible variables of the RBM2 (crossed squares) and initialise the rest as random input. We perform k MC steps on RBM2 while keeping the pinned variables fixed to generate the rest of the sequence (light blue). The letters next to arrows show the order of this sampling procedure. https://doi.org/10.1371/journal.pcbi.1011584.g002 This method was used to generate the large-scale 65,535-SNP dataset. We first trained a “normal” RBM on the first 5,000 input variables. Then, we made a set of 10,000-input conditional RBMs, where the first 5,000 variables were used as pinned variables. All the models were trained with 1000 hidden nodes, and a learning rate of β = 0.005. The learning dynamics uses the Rdm-k method: each chain was generated starting from the uniform Bernoulli distribution, with k = 100 for the training of the RBM and the CRBMs. A recurrent issue with RBM (or training in general) is to decide when to stop the training. While in supervised setting it is easy to monitor the loss function (or the number of correctly classified samples), it is not the case for RBM since the partition function is intractable. In this work, the learning was affected by the small number of samples given the dimension of the inputs. Therefore (and to avoid overfitting), the meta-parameters such as the number of hidden nodes and the number of epochs were fixed a posteriori, by investigating various trained machines with different values of these parameters and choosing the one giving a good AATS score. The detailed python scripts can be accessed at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. The RBM implementations are based on the pytorch library and handle GPU to perform both training and sampling. Another example of the out-of-equilibrium code applied to images can be found at https://github.com/AurelienDecelle/TorchRBM. VAE implementation The VAE [36] architecture is very similar to the GAN architecture but somehow reversed: the encoder is an analogue to the critic and the decoder is an analogue to the generator (S1 Fig). The last layer of the encoder outputs two vectors containing means (μ) and standard deviations (σ) so that the latent space can be sampled based on these values and fed to the decoder. The loss function consists of the reconstruction term, which measures the likelihood of the generated genomes (via log loss in our implementation), and the regularisation term which is the Kullback-Leibler divergence between the standard normal distribution and the prior distribution of the latent space. The regularisation term directs the latent space towards a standard normal distribution. After training completion, this allows sampling of new latent points from the standard distribution which are further transformed into new data points (new genomic sequences) through the decoder. Each layer in our implementation is followed by batch normalisation and leaky ReLU with alpha set to 0.01, except for the final layers. The decoder final layer is followed by a sigmoid function, whereas the encoder final μ and σ layers have linear activation. We used Adam optimizer with default settings and the learning rate set to 0.001. Similarly to the WGAN, we evaluated the training based on coherence of the PCA performed on real and generated genomes (components 1 to 4). We could not successfully train a VAE model which generates plausible AGs for the 65,535-SNP dataset. Further architecture and hyperparameter optimization is needed to better assess VAE models in this context. VAE models were coded with python-3.9 and pythorch-1.11 [32]. The detailed python scripts can be accessed at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. Nearest neighbour adversarial accuracy (AATS) Similarly to [21], we assessed the overfitting/underfitting characteristics of AGs using the AATS score [37]. AATS is calculated as follows: where n is the number of samples in each dataset (real and generated), dTS(i) is the distance between the real (truth—T) sample indexed by i and its nearest neighbour in the generated (synthetic—S) dataset, dTT(i) is the distance between a real sample i and its nearest neighbour in the real dataset, dST(i) is the distance between a generated sample i and its nearest neighbour in the real dataset, dSS(i) is the distance between a generated sample i and its nearest neighbour in the generated dataset, and 1 is the indicator function which returns 1 if the argument is true and 0 otherwise. Based on this equation, an AATS value below 0.5 indicates overfitting and an AATS value above 0.5 indicates underfitting. We additionally obtained a privacy score in another analysis where we separated the datasets into two equal-sized train and test sets (2504 phased haplotypes each). The score is defined as follows: where Test AATS (resp. Train AATS) is the AATS computed with the test (resp. training) samples as truth. The expected value for the privacy score is 0 when there is no privacy leakage, with higher scores indicating higher leaks. Summary statistics For the 65,535-SNP dataset, LD was computed only on a subset of pairs in order to fasten computation. To sample these pairs in an efficient way (i.e. approximately uniformly along the SNP distance log scale without computing the full matrix of SNP distances), we used the script from [38]. The remaining summary statistics (allele frequencies, haplotypic pairwise distances, PC scores, 3-point correlations and LD for the 10,000-SNP dataset) were computed as in [21] using the publicly available scripts at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. For the radar plots, we transformed the scores so that they span values between 0 and 1, where 0 represents poor performance and 1 high or perfect performance. Precisely, we used the allele frequency correlation for alleles with low frequency (<= 0.2) for the Allele frequency score and correlation of SNPs separated by random distances for the 3–point correlations score. For the Pairwise distance score, we used the Wasserstein distance between the distributions of haplotypic pairwise distances of real and generated data. We performed min-max scaling, using the value for a simple binomial generator model (from [21]) for the lowest bound 0. For the other scores, we used the following equations: where LDgenerated and LDreal are average LD values for bins in the LD decay analyses. Overfitting and Underfitting equations were formulated to focus on below and above 0.5 AATS sweet spot, respectively, to provide a better resolution in terms of overfitting/underfitting assessment. To analyze the correlation of k-mer haplotypes between real and generated genomes, we divided 10,000-SNP and 65,535-SNP genomes into non-overlapping windows of size 4 and 8. We counted the number of occurrences of each unique motif in these windows and assessed the correlation of these counts between real and generated genomes for each window. Nearest neighbour chain analysis Since AATS scores for the CRBM AGs were anomalous, we perfomed a nearest neighbour chain analysis for further investigation. For this analysis, the frequencies of all observable patterns for the nearest neighbour chains of size 2 to 5 were computed. A pattern indicates the succession of data type (synthetic/S or truth/T) when starting from a point (the first letter) and moving successively to the nearest neighbour, then the nearest neighbour of the nearest neighbour, and so on until reaching the chain size. Hamming distance was used for identifying the nearest neighbours. Membership inference attacks We performed membership inference attacks on WGAN and RBM generated AGs using the approach proposed in [39]. We trained the WGAN and RBM models using half of the haplotypes (2504) of the 10,000-SNP data and kept the rest as test set. We considered the two following scenarios: a white-box attack where the adversary has access to the original critic optimized weights and architecture, and a black-box attack where the adversary has only access to the WGAN architecture (without its weights) and generated samples. For both scenarios, we also assume the adversary knows the size of the original training set and has a collection of samples some of which are suspected to belong to the training set. For the white-box attack, we used the already trained critic to score all the samples (a total of 5008 samples consisting of 2504 from training and 2504 from test sets) and sorted them based on these scores. The top n ranking samples are then predicted as belonging to the training set. For the black-box attack, we trained new models on generated AGs, using the exact same architecture as the previously trained WGAN, and the same stopping criterion, but we overtrained further up to 5000 epochs to induce overfitting. Since this attack is model agnostic, we trained one model on WGAN AGs and one on RBM AGs. Similar to the white-box attack, all samples are scored by the critic after the training and the top n ranking samples are assigned to the training set. In our experiments, n varies such that we assign 1%, 10%, 25%, or 50% of the total 5008 samples to the training set. The computed accuracy is simply the percentage of correct assignments for each value of n. Data The two sets of 1000 Genomes [31] data we used include: (i) 10,000 SNPs from chromosome 15 (between 27,379,578—29,625,035 base pairs, ∼2 megabase pairs) identical to the ones picked by [21], and (ii) 65,535 SNPs from chromosome 1 between 534,247—81,813,279 base pairs, ∼80 megabase pairs) within the Omni 2.5 genotyping array framework. Further downsampling of the array framework was performed to create a dataset with a reasonable SNP number for faster training trials and the specific number of 65,535 SNPs was decided to be in the form of (2n—1) for easy implementation of convolutional scaling. For the 10,000-SNP dataset, we used padding with zeros to match the (2n—1) form. Both datasets have the same 2504 individuals and 5008 phased haplotypes used for training the models. The data format is the same as [21] where the rows are phased haplotypes and columns are positions which hold alleles represented by 0 (reference) and 1 (alternative). WGAN implementation We implemented a Wasserstein GAN with gradient penalty (WGAN-GP) consisting of a critic which estimates the earth mover’s distance between real and generated data distributions, and a generator which generates new genomic data from Gaussian noise (Fig 1). Unlike the discriminator in naive GAN which performs a classification task, the critic provides a “realness” score (an approximation of Earth-mover’s distance) for generated and real samples. WGAN objective function to be minimized by the generator and maximized by the critic is as follows: where C is the critic, G is the generator, x is real data point and z is Gaussian noise. The critic function C must be Lipschitz continuous; thus, the original study relied on weight clipping to enforce an upper bound for the gradient [22]. However, we designed a WGAN with gradient penalty (WGAN-GP) rather than weight clipping, as GP was shown to be a better alternative [25]. In our implementation, the critic uses convolution layers whereas the generator uses convolution and transposed convolution layers (Fig 1A). Both the generator and the critic have trainable location-specific vectors as additional channels at every block except for the residual blocks. These vectors, similar to the ones integrated by [27] in their autoencoder, consist of trainable variables that allow the models to preserve positional information which would otherwise diminish due to the invariance of convolution operations. In addition to this, the generator has two noise channels at every block. This gives the generator flexibility to decide at which depth the mapping to latent space can occur. We also implemented “packing” to overcome mode collapse by adjusting the discriminator to take multiple samples as input [29]. In our implementation, the input sample number for the discriminator was set to 3 intuitively, to match the 3 main population modes (Africa, West Eurasia, Asia) observable in the 1000 Genomes data. The initial length of the input for the generator is 4, which is gradually transformed into features of larger sizes until it reaches the sequence size of 65,535 or 16,383 in the output (Fig 1B). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Wasserstein GAN (WGAN) model for the 65,535-SNP dataset. a) Representation of the generator, critic and residual blocks. Channel dimensions are not proportional and do not reflect the real implementation. The generator block has one trainable location-specific variable (blue) and two latent space vectors (red) as additional channels concatenated to the input. The critic block has one trainable location-specific variable (blue) as an additional channel concatenated to the input. b) Architecture of the WGAN model. White rectangles correspond to generic generator and critic blocks whereas grey rectangles correspond to generic residual blocks. Numbers in parentheses above blocks show channels and length, respectively (C, L). Dotted connections are residual connections where the input value is added to the output value of the block before passing to the next block. Yellow input and output blocks differ from the generic ones for proper dimension adjustments. https://doi.org/10.1371/journal.pcbi.1011584.g001 The generator layers are followed by batch normalisation and leaky ReLU activation (alpha = 0.01) except for the final layer which has a sigmoid activation. The critic layers are followed by instance normalisation and leaky ReLU activation (alpha = 0.01) except for the final layer which is a fully connected layer with no activation. We used Adam optimizer to train both the generator and the critic with a learning rate of 0.0005 and β1, β2 = (0.5, 0,9). β1 was set to 0.5 as suggested by [13]. For each batch training of the generator, the critic was trained 10 times as suggested by the authors of the original WGAN [22]. We assessed the outputs of the generator at each epoch during training via PCA. We stopped training when generated and real genome clusters visually overlapped in PC space (components 1 to 4). In the case of 65,535-SNP data, this initial training was not sufficient to reach a good overlap of higher degree principal components, thus, we performed a second brief training (up to 200 epochs) with 10-fold lower generator learning rate (0.00005). Our WGAN architecture for the 10,000-SNP data was conceptually the same as for the 65,535-SNP data, but shallower with fewer blocks. All WGAN models were coded with python-3.9 and pythorch-1.11 [32]. The detailed python scripts can be accessed at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. RBM implementation An RBM is a generative stochastic neural network [33] defined as a probability distribution over a set of visible units, , representing SNPs in our case, and hidden units, , where both type of variables interact via a pairwise weight matrix and the local biases and help to adjust the mean value of each unit: In this study, we used binary units {0, 1} for both the visible and hidden nodes, and the number of hidden units was chosen to be about the same order as the number of visible ones. The likelihood of such model is given by: where the index m is indexing the samples of the dataset (M being the total number of samples) and Z is the normalization constant, or partition function, of the probability distribution. Learning an RBM consists in maximising this likelihood using gradient descent in order to optimize the weights and biases w, θ and η. In our implementation, the training was based on the out-of-equilibrium method [24, 34]. The main difference with the more conventional learning is that the sampling which is done to compute the correlation of the model is performed in a very precise way: a random initial condition is chosen amongst a certain probability distribution p0(x) (kept fixed during the learning), and a fix (all along learning) number of Monte Carlo steps is done during the training. When using this particular training procedure, in order to sample new data, it is enough to generate the Monte Carlo chains following the same dynamical process: same initial conditions p0(x) and same number of MC steps. In the provided experiments, the initial conditions were chosen uniformly at random (each unit having equal probability to be 0 or 1) and the number of MC steps was either 100 (for the 65,535-SNP dataset) or 200 (for the 10,000-SNP dataset) but the qualitative results were mostly not affected. The learning rate of the model was chosen such that the eigenvalues of the learning weight matrix are smoothly increasing during the first epochs from almost zero to values of ∼O(1). For the generation of very long sequences, we designed a novel procedure based on conditional-RBMs (CRBMs) [23, 35] (Fig 2). The CRBM consists in the learning of correlation patterns conditionally to some input variables. Hence, instead of considering all the dataset X, we separate it into two parts X = XP ∪ XI, and the CRBM will learn to generate XI (the inferred variables) based on XP (the pinned variables). Therefore, we design a gradient that will learn how to generate the variables XI given that we provide the variables Xp. In practice, denoting as the variables to be inferred and as the pinned ones, we will maximize the following quantity: Therefore, for each sample of the dataset, this construction is similar to a classical RBM with additional biases for the hidden nodes which depend on the pinned variables. During the learning, we infer the parameters of the model such that when showing a configuration of pinned variables XP, the model will generate a sample that is correlated to it accordingly. This conditional model can be used to learn very long chains of variables. Let us consider that we have a dataset with 100,000 input variables. We can learn initially a regular RBM on the first 10,000 input variables. In parallel, we can also learn a conditional RBM on the input variables si with i in [5,000:15,000] using the first 5,000 variables as pinned variables. Therefore, this second RBM will, given 5,000 pinned variables, learn how to generate the 5,000 following ones. The same procedure is repeated with various input sets, always using the first 5,000 input nodes as pinned variables. Once learning is completed for all the RBMs, we can proceed using the following sequential method to generate new data: Use the first RBM (non-conditional) to generate the first 10,000 input. Use the first CRBM to generate the next [10,000:15,000] inputs. To do that, we use the generated nodes [5,000:10,000] as pinned variables and generate the rest. We follow the same procedure until we finally generate the whole 100,000 variables. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Illustration of the learning and sampling of a large sequence using a “classical” and a conditional RBM (CRBM). Initially, we train RBM1 (left) and RBM2 (right) in parallel. Both RBMs are essentially trained in a similar manner: random inputs are drawn and k MC steps are performed before computing the gradient and updating the weights using gradient descent. The difference for the CRBM (RBM2) is that half of the variables in the visible layer are pinned (crossed squares) to the real data during training while the rest is generated conditionally on these pinned variables. After training both machines, we can sample a complete new sequence. To do so, we start from random input and perform k MC steps to generate the first part of the sequence (light yellow-red) using RBM1. Then, we use half of this generated sequence (light red) as the pinned visible variables of the RBM2 (crossed squares) and initialise the rest as random input. We perform k MC steps on RBM2 while keeping the pinned variables fixed to generate the rest of the sequence (light blue). The letters next to arrows show the order of this sampling procedure. https://doi.org/10.1371/journal.pcbi.1011584.g002 This method was used to generate the large-scale 65,535-SNP dataset. We first trained a “normal” RBM on the first 5,000 input variables. Then, we made a set of 10,000-input conditional RBMs, where the first 5,000 variables were used as pinned variables. All the models were trained with 1000 hidden nodes, and a learning rate of β = 0.005. The learning dynamics uses the Rdm-k method: each chain was generated starting from the uniform Bernoulli distribution, with k = 100 for the training of the RBM and the CRBMs. A recurrent issue with RBM (or training in general) is to decide when to stop the training. While in supervised setting it is easy to monitor the loss function (or the number of correctly classified samples), it is not the case for RBM since the partition function is intractable. In this work, the learning was affected by the small number of samples given the dimension of the inputs. Therefore (and to avoid overfitting), the meta-parameters such as the number of hidden nodes and the number of epochs were fixed a posteriori, by investigating various trained machines with different values of these parameters and choosing the one giving a good AATS score. The detailed python scripts can be accessed at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. The RBM implementations are based on the pytorch library and handle GPU to perform both training and sampling. Another example of the out-of-equilibrium code applied to images can be found at https://github.com/AurelienDecelle/TorchRBM. VAE implementation The VAE [36] architecture is very similar to the GAN architecture but somehow reversed: the encoder is an analogue to the critic and the decoder is an analogue to the generator (S1 Fig). The last layer of the encoder outputs two vectors containing means (μ) and standard deviations (σ) so that the latent space can be sampled based on these values and fed to the decoder. The loss function consists of the reconstruction term, which measures the likelihood of the generated genomes (via log loss in our implementation), and the regularisation term which is the Kullback-Leibler divergence between the standard normal distribution and the prior distribution of the latent space. The regularisation term directs the latent space towards a standard normal distribution. After training completion, this allows sampling of new latent points from the standard distribution which are further transformed into new data points (new genomic sequences) through the decoder. Each layer in our implementation is followed by batch normalisation and leaky ReLU with alpha set to 0.01, except for the final layers. The decoder final layer is followed by a sigmoid function, whereas the encoder final μ and σ layers have linear activation. We used Adam optimizer with default settings and the learning rate set to 0.001. Similarly to the WGAN, we evaluated the training based on coherence of the PCA performed on real and generated genomes (components 1 to 4). We could not successfully train a VAE model which generates plausible AGs for the 65,535-SNP dataset. Further architecture and hyperparameter optimization is needed to better assess VAE models in this context. VAE models were coded with python-3.9 and pythorch-1.11 [32]. The detailed python scripts can be accessed at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. Nearest neighbour adversarial accuracy (AATS) Similarly to [21], we assessed the overfitting/underfitting characteristics of AGs using the AATS score [37]. AATS is calculated as follows: where n is the number of samples in each dataset (real and generated), dTS(i) is the distance between the real (truth—T) sample indexed by i and its nearest neighbour in the generated (synthetic—S) dataset, dTT(i) is the distance between a real sample i and its nearest neighbour in the real dataset, dST(i) is the distance between a generated sample i and its nearest neighbour in the real dataset, dSS(i) is the distance between a generated sample i and its nearest neighbour in the generated dataset, and 1 is the indicator function which returns 1 if the argument is true and 0 otherwise. Based on this equation, an AATS value below 0.5 indicates overfitting and an AATS value above 0.5 indicates underfitting. We additionally obtained a privacy score in another analysis where we separated the datasets into two equal-sized train and test sets (2504 phased haplotypes each). The score is defined as follows: where Test AATS (resp. Train AATS) is the AATS computed with the test (resp. training) samples as truth. The expected value for the privacy score is 0 when there is no privacy leakage, with higher scores indicating higher leaks. Summary statistics For the 65,535-SNP dataset, LD was computed only on a subset of pairs in order to fasten computation. To sample these pairs in an efficient way (i.e. approximately uniformly along the SNP distance log scale without computing the full matrix of SNP distances), we used the script from [38]. The remaining summary statistics (allele frequencies, haplotypic pairwise distances, PC scores, 3-point correlations and LD for the 10,000-SNP dataset) were computed as in [21] using the publicly available scripts at https://gitlab.inria.fr/ml_genetics/public/artificial_genomes. For the radar plots, we transformed the scores so that they span values between 0 and 1, where 0 represents poor performance and 1 high or perfect performance. Precisely, we used the allele frequency correlation for alleles with low frequency (<= 0.2) for the Allele frequency score and correlation of SNPs separated by random distances for the 3–point correlations score. For the Pairwise distance score, we used the Wasserstein distance between the distributions of haplotypic pairwise distances of real and generated data. We performed min-max scaling, using the value for a simple binomial generator model (from [21]) for the lowest bound 0. For the other scores, we used the following equations: where LDgenerated and LDreal are average LD values for bins in the LD decay analyses. Overfitting and Underfitting equations were formulated to focus on below and above 0.5 AATS sweet spot, respectively, to provide a better resolution in terms of overfitting/underfitting assessment. To analyze the correlation of k-mer haplotypes between real and generated genomes, we divided 10,000-SNP and 65,535-SNP genomes into non-overlapping windows of size 4 and 8. We counted the number of occurrences of each unique motif in these windows and assessed the correlation of these counts between real and generated genomes for each window. Nearest neighbour chain analysis Since AATS scores for the CRBM AGs were anomalous, we perfomed a nearest neighbour chain analysis for further investigation. For this analysis, the frequencies of all observable patterns for the nearest neighbour chains of size 2 to 5 were computed. A pattern indicates the succession of data type (synthetic/S or truth/T) when starting from a point (the first letter) and moving successively to the nearest neighbour, then the nearest neighbour of the nearest neighbour, and so on until reaching the chain size. Hamming distance was used for identifying the nearest neighbours. Membership inference attacks We performed membership inference attacks on WGAN and RBM generated AGs using the approach proposed in [39]. We trained the WGAN and RBM models using half of the haplotypes (2504) of the 10,000-SNP data and kept the rest as test set. We considered the two following scenarios: a white-box attack where the adversary has access to the original critic optimized weights and architecture, and a black-box attack where the adversary has only access to the WGAN architecture (without its weights) and generated samples. For both scenarios, we also assume the adversary knows the size of the original training set and has a collection of samples some of which are suspected to belong to the training set. For the white-box attack, we used the already trained critic to score all the samples (a total of 5008 samples consisting of 2504 from training and 2504 from test sets) and sorted them based on these scores. The top n ranking samples are then predicted as belonging to the training set. For the black-box attack, we trained new models on generated AGs, using the exact same architecture as the previously trained WGAN, and the same stopping criterion, but we overtrained further up to 5000 epochs to induce overfitting. Since this attack is model agnostic, we trained one model on WGAN AGs and one on RBM AGs. Similar to the white-box attack, all samples are scored by the critic after the training and the top n ranking samples are assigned to the training set. In our experiments, n varies such that we assign 1%, 10%, 25%, or 50% of the total 5008 samples to the training set. The computed accuracy is simply the percentage of correct assignments for each value of n. Results Comparisons with the previous models As our GAN concept has changed substantially compared to the previous models both in terms of architecture and loss functions, we initially performed training and analysis on 1000 Genomes data with the same 10,000 SNPs as [21] to be able to conduct one-to-one comparisons. For these tests, we additionally implemented a new RBM scheme along with a new variational autoencoder (VAE) which has a very similar architecture to our WGAN model. We used the VAE as a supplementary benchmark since the encoder-decoder form of the VAE can be seen as an analogue to the critic-generator form of the WGAN model. The objective function of VAE, on the other hand, is substantially different, which allows us to assess the robustness of the architecture we used (see Materials and methods). Based on the PCAs, all models generated AGs which could capture the population structure of the data, albeit new WGAN and RBM models were better at representing the real PC densities compared to the other models (Fig 3A). In our previous study, we reported that the GAN model had difficulty learning low frequency alleles observed in real genomes. The new WGAN and RBM showed improvement in capturing the real allele frequency distribution, especially for the low frequency (<= 0.2) alleles with correlation coefficients for previous GAN (GAN_prev), previous RBM (RBM_prev), VAE, WGAN and RBM being 0.94, 0.83, 0.94, 0.96 and 0.99, respectively (Fig 3B). In terms of LD structure, RBM generated AGs seemed to have the closest decay scheme to the real data while WGAN, GAN and VAE generated AGs all had similar results without substantial difference (S2 Fig). In addition, the new WGAN and RBM models preserved 3-point correlations better than the other models (S3 Fig). To assess whether AGs could preserve short motifs in real genomes, we also analyzed the correlation of non-overlapping 4-mer and 8-mer motifs between real and generated datasets (S4 Fig). AGs generated by all models showed high correlation and good fit overall, although RBM_prev had the highest variance despite the high correlation coefficient. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Principal component and allele frequency analyses of artificial genomes with 10,000-SNP size. a) Density plot of the PCA of combined real and artificial genome datasets. Density increases from red to blue. (b) Allele frequency correlation between real (x-axis) and artificial (y-axis) genome datasets. Bottom figures are zoomed at low frequency alleles (from 0 to 0.2 overall frequency in the real dataset). Values presented inside the figures are Pearson’s r, ordinary least squares regression slope and intercept. https://doi.org/10.1371/journal.pcbi.1011584.g003 None of the models have produced identical sequences and no full sequences were copied from the training dataset except for the benchmark VAE. In addition, the distribution of haplotypic pairwise differences between real and generated datasets overlapped well with the distribution of haplotypic pairwise differences within the real dataset (S5 Fig). The AATS scores for GAN_prev, RBM_prev, VAE, WGAN and RBM were 0.73 (AAtruth = 0.57, AAsyn = 0.89), 0.49 (AAtruth = 0.46, AAsyn = 0.52), 0.48 (AAtruth = 0.47, AAsyn = 0.50), 0.82 (AAtruth = 0.77, AAsyn = 0.87) and 0.47 (AAtruth = 0.47, AAsyn = 0.47), respectively. These values indicate underfitting (a hypothetical extreme case demonstrated in Fig 4A) for GAN generated AGs, and slight overfitting (a hypothetical extreme case demonstrated in Fig 4B) for the RBM and VAE generated AGs (S6 Fig). Although the new WGAN demonstrated slightly more underfitting than the previous GAN, the gap between the two components of AATS (AAtruth and AAsyn) was decreased (see Materials and methods for the details of the terminology). We previously hypothesised that the low value of AAtruth and high value of AAsyn for the previous GAN might be due to the generator creating AGs based on averages from a local set of samples in small pockets (extreme case demonstrated in Fig 4C). This can be seen as a generative aberration which does not generalise to the whole dataset but is observed only regionally in small subsections of the data. We do not observe this behaviour in the new WGAN model. A general comparison of all the models based on multiple aggregated statistics is provided in S7 Fig. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Schematic representation of different problematic training outcomes for generative models. Distances to the nearest neighbours are denoted by dxx where x can be T (truth/real) and S (synthetic/generated). a) An extreme case of underfitting (or optimization issue) in which the nearest neighbours of real data points are real and the nearest neighbours of synthetic data points are synthetic, revealing two distinct clusters (AATS >> 0.5). b) An extreme case of overfitting in which the nearest neighbours of real data points are systematically synthetic and vice versa (AATS << 0.5). c) An extreme case of a specific type of generative aberration in which the nearest neighbours of both real and synthetic data points are synthetic (AAsyn >> 0.5 and AAtruth << 0.5). Hypothetically, this might occur when the generator generates new instances based on average information from a small collection of samples, causing low local variation. d) An extreme case of a specific type of generative aberration in which the nearest neighbours of both real and synthetic data points are real (AAsyn << 0.5 and AAtruth >> 0.5). This might possibly be observed when the generative model learns the main modes in real data but fails to mimic the densities and generates instances in the axes of the main modes with high variance. https://doi.org/10.1371/journal.pcbi.1011584.g004 Generating large-scale genomic data Following the main motivation of the study, we trained the new WGAN and CRBM models on 1000 Genomes data with 65,535 SNPs. The WGAN model for this data was deeper compared to the model used for the 10,000-SNP data (see Materials and methods). We again implemented a VAE similar to the WGAN architecture yet we could not train this model with satisfactory results (see Discussion). Both WGAN and CRBM generated AGs were able to capture the real population structure and PCA modes quite well (Fig 5A) along with allele frequencies (correlation coefficient for low frequency alleles being 0.97 for both models; Fig 5B), yet WGAN AGs had substantially more fixed alleles which had low frequency in the real dataset (S8 Fig). Since computation of the full correlation matrix is very intensive due to large sequence size, we calculated an approximation of the LD decay based on a subset of SNP pairs (see Materials and methods). AGs generated by both models had on average lower LD than real genomes, similarly to our previous findings (Fig 5C). In 3-point correlation analysis, CRBM performed better than WGAN for SNP triplets seperated by 1, 4, 16, 64, 256, 512 and 1024 SNPs. However, the score was similar for SNP triplets seperated by random distances (WGAN = 0.43, CRBM = 0.41; S9 Fig). Furthermore, 4-mer and 8-mer motif distributions both for WGAN and CRBM generated AGs seemed to be similar to real genomes (S10 Fig). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Principal component, allele frequency and linkage disequilibrium (LD) analyses of artificial genomes with 65,535-SNP size. a) Density plot of the PCA of combined real genomes and artificial genomes generated by WGAN and CRBM. Density increases from red to blue. b) Allele frequency correlation between real and artificial genome datasets. Bottom figures are zoomed at low frequency alleles (from 0 to 0.2 overall frequency in the real dataset). Values presented inside the figures are Pearson’s r, ordinary least squares regression slope and intercept. The dashed black line is the identity line. c) LD decay approximation for real (grey), WGAN generated (blue) and CRBM generated (red) genomes (see Materials and methods for details). d) Nearest neighbour adversarial accuracy (AATS) of artificial genomes generated by different models for the 65,535-SNP dataset. Values below 0.5 (black line) indicate overfitting and values above indicate underfitting. https://doi.org/10.1371/journal.pcbi.1011584.g005 Similarly to the 10,000-SNP dataset, neither of the models produced identical sequences and no full sequences were copied from the training dataset. WGAN captured the haplotypic pairwise distribution better than CRBM (pairwise distance radar score for WGAN = 1.00, CRBM = 0.94; S11 Fig) but the two main peaks in real data were correctly represented by both models (S12 Fig)). The AATS value for WGAN AGs showed underfitting (AATS = 0.91) whereas the value for CRBM was slightly above the 0.5 sweet spot (AATS = 0.56; Fig 5D. However, there was a huge contrast between AAtruth and AAsyn values for CRBM AGs (AAtruth = 0.86, AAsyn = 0.26), which might be an indication of the anomaly depicted in Fig 4D. This is the opposite of what we observed for the previous GAN (GAN_prev) model and might be due to AGs which exist outside the real data space as can be seen from the PCA analysis (S13 Fig). A general comparison of the two models based on multiple aggregated statistics is provided in S11 Fig. To further investigate the anomaly for the CRBM AGs, we performed a nearest neighbour chain analysis (Table 1). Starting with a synthetic point, we observed substantially higher frequencies for chains of true data points (ST, STT, STTT, STTTT) compared to chains of synthetic data points (SS, SSS, SSSS, SSSSS). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Nearest neighbour chain analysis. Frequencies of series of generated (synthetic—S) and real (truth—T) samples in chains of nearest neighbours (from size 2 to 5). To avoid loops, a sample was removed once reached in the chain. Expected frequency for chains of size 2 is 0.25, size 3 is 0.125, size 4 is 0.0625 and size 5 is 0.03125. https://doi.org/10.1371/journal.pcbi.1011584.t001 Membership inference attacks and privacy leakage Assessing privacy leakage and performing membership inference attacks require an additional test set but we could not obtain good quality AGs using a subset of 65,535-SNP dataset. Therefore, we trained the new WGAN and RBM models using half of the samples from the 10,000-SNP dataset (2504 haplotypes). AGs generated via the WGAN model had similar summary statistics to the AGs generated via the model trained with the whole dataset. For the RBM model, results were slightly worse but better than the previous RBM model (RBM_prev) trained with the whole dataset (S14A–S14C Fig). AATS analysis showed underfitting for WGAN AGs (AATS = 0.80, AAtruth = 0.76, AAsyn = 0.83) and almost perfect scores for RBM AGs (AATS = 0.50, AAtruth = 0.49, AAsyn = 0.51). Based on the test set, WGAN AGs had good privacy score (0.03) and RBM AGs showed possible privacy leakage (0.23) (S14D Fig). Furthermore, both white-box attack on WGAN AGs and black-box attack on RBM AGs reached high accuracies (WGAN: 1%: 0.82, 10%: 0.826, 25%: 0.754, 50%: 0.677; RBM: 1%: 0.84, 10%: 0.70, 25%: 0.62, 50%: 0.57) for detecting a portion of the training samples whereas black-box attack on WGAN AGs produced substantially lower accuracies (1%: 0.56, 10%: 0.55, 25%: 0.54, 50%: 0.51), suggesting relatively better privacy preservation. The distribution of the critic scores for train and test samples also showed that the black-box attack on WGAN AGs was mainly unsuccessful for differentiating training and test samples (S15 Fig). Comparisons with the previous models As our GAN concept has changed substantially compared to the previous models both in terms of architecture and loss functions, we initially performed training and analysis on 1000 Genomes data with the same 10,000 SNPs as [21] to be able to conduct one-to-one comparisons. For these tests, we additionally implemented a new RBM scheme along with a new variational autoencoder (VAE) which has a very similar architecture to our WGAN model. We used the VAE as a supplementary benchmark since the encoder-decoder form of the VAE can be seen as an analogue to the critic-generator form of the WGAN model. The objective function of VAE, on the other hand, is substantially different, which allows us to assess the robustness of the architecture we used (see Materials and methods). Based on the PCAs, all models generated AGs which could capture the population structure of the data, albeit new WGAN and RBM models were better at representing the real PC densities compared to the other models (Fig 3A). In our previous study, we reported that the GAN model had difficulty learning low frequency alleles observed in real genomes. The new WGAN and RBM showed improvement in capturing the real allele frequency distribution, especially for the low frequency (<= 0.2) alleles with correlation coefficients for previous GAN (GAN_prev), previous RBM (RBM_prev), VAE, WGAN and RBM being 0.94, 0.83, 0.94, 0.96 and 0.99, respectively (Fig 3B). In terms of LD structure, RBM generated AGs seemed to have the closest decay scheme to the real data while WGAN, GAN and VAE generated AGs all had similar results without substantial difference (S2 Fig). In addition, the new WGAN and RBM models preserved 3-point correlations better than the other models (S3 Fig). To assess whether AGs could preserve short motifs in real genomes, we also analyzed the correlation of non-overlapping 4-mer and 8-mer motifs between real and generated datasets (S4 Fig). AGs generated by all models showed high correlation and good fit overall, although RBM_prev had the highest variance despite the high correlation coefficient. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Principal component and allele frequency analyses of artificial genomes with 10,000-SNP size. a) Density plot of the PCA of combined real and artificial genome datasets. Density increases from red to blue. (b) Allele frequency correlation between real (x-axis) and artificial (y-axis) genome datasets. Bottom figures are zoomed at low frequency alleles (from 0 to 0.2 overall frequency in the real dataset). Values presented inside the figures are Pearson’s r, ordinary least squares regression slope and intercept. https://doi.org/10.1371/journal.pcbi.1011584.g003 None of the models have produced identical sequences and no full sequences were copied from the training dataset except for the benchmark VAE. In addition, the distribution of haplotypic pairwise differences between real and generated datasets overlapped well with the distribution of haplotypic pairwise differences within the real dataset (S5 Fig). The AATS scores for GAN_prev, RBM_prev, VAE, WGAN and RBM were 0.73 (AAtruth = 0.57, AAsyn = 0.89), 0.49 (AAtruth = 0.46, AAsyn = 0.52), 0.48 (AAtruth = 0.47, AAsyn = 0.50), 0.82 (AAtruth = 0.77, AAsyn = 0.87) and 0.47 (AAtruth = 0.47, AAsyn = 0.47), respectively. These values indicate underfitting (a hypothetical extreme case demonstrated in Fig 4A) for GAN generated AGs, and slight overfitting (a hypothetical extreme case demonstrated in Fig 4B) for the RBM and VAE generated AGs (S6 Fig). Although the new WGAN demonstrated slightly more underfitting than the previous GAN, the gap between the two components of AATS (AAtruth and AAsyn) was decreased (see Materials and methods for the details of the terminology). We previously hypothesised that the low value of AAtruth and high value of AAsyn for the previous GAN might be due to the generator creating AGs based on averages from a local set of samples in small pockets (extreme case demonstrated in Fig 4C). This can be seen as a generative aberration which does not generalise to the whole dataset but is observed only regionally in small subsections of the data. We do not observe this behaviour in the new WGAN model. A general comparison of all the models based on multiple aggregated statistics is provided in S7 Fig. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Schematic representation of different problematic training outcomes for generative models. Distances to the nearest neighbours are denoted by dxx where x can be T (truth/real) and S (synthetic/generated). a) An extreme case of underfitting (or optimization issue) in which the nearest neighbours of real data points are real and the nearest neighbours of synthetic data points are synthetic, revealing two distinct clusters (AATS >> 0.5). b) An extreme case of overfitting in which the nearest neighbours of real data points are systematically synthetic and vice versa (AATS << 0.5). c) An extreme case of a specific type of generative aberration in which the nearest neighbours of both real and synthetic data points are synthetic (AAsyn >> 0.5 and AAtruth << 0.5). Hypothetically, this might occur when the generator generates new instances based on average information from a small collection of samples, causing low local variation. d) An extreme case of a specific type of generative aberration in which the nearest neighbours of both real and synthetic data points are real (AAsyn << 0.5 and AAtruth >> 0.5). This might possibly be observed when the generative model learns the main modes in real data but fails to mimic the densities and generates instances in the axes of the main modes with high variance. https://doi.org/10.1371/journal.pcbi.1011584.g004 Generating large-scale genomic data Following the main motivation of the study, we trained the new WGAN and CRBM models on 1000 Genomes data with 65,535 SNPs. The WGAN model for this data was deeper compared to the model used for the 10,000-SNP data (see Materials and methods). We again implemented a VAE similar to the WGAN architecture yet we could not train this model with satisfactory results (see Discussion). Both WGAN and CRBM generated AGs were able to capture the real population structure and PCA modes quite well (Fig 5A) along with allele frequencies (correlation coefficient for low frequency alleles being 0.97 for both models; Fig 5B), yet WGAN AGs had substantially more fixed alleles which had low frequency in the real dataset (S8 Fig). Since computation of the full correlation matrix is very intensive due to large sequence size, we calculated an approximation of the LD decay based on a subset of SNP pairs (see Materials and methods). AGs generated by both models had on average lower LD than real genomes, similarly to our previous findings (Fig 5C). In 3-point correlation analysis, CRBM performed better than WGAN for SNP triplets seperated by 1, 4, 16, 64, 256, 512 and 1024 SNPs. However, the score was similar for SNP triplets seperated by random distances (WGAN = 0.43, CRBM = 0.41; S9 Fig). Furthermore, 4-mer and 8-mer motif distributions both for WGAN and CRBM generated AGs seemed to be similar to real genomes (S10 Fig). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Principal component, allele frequency and linkage disequilibrium (LD) analyses of artificial genomes with 65,535-SNP size. a) Density plot of the PCA of combined real genomes and artificial genomes generated by WGAN and CRBM. Density increases from red to blue. b) Allele frequency correlation between real and artificial genome datasets. Bottom figures are zoomed at low frequency alleles (from 0 to 0.2 overall frequency in the real dataset). Values presented inside the figures are Pearson’s r, ordinary least squares regression slope and intercept. The dashed black line is the identity line. c) LD decay approximation for real (grey), WGAN generated (blue) and CRBM generated (red) genomes (see Materials and methods for details). d) Nearest neighbour adversarial accuracy (AATS) of artificial genomes generated by different models for the 65,535-SNP dataset. Values below 0.5 (black line) indicate overfitting and values above indicate underfitting. https://doi.org/10.1371/journal.pcbi.1011584.g005 Similarly to the 10,000-SNP dataset, neither of the models produced identical sequences and no full sequences were copied from the training dataset. WGAN captured the haplotypic pairwise distribution better than CRBM (pairwise distance radar score for WGAN = 1.00, CRBM = 0.94; S11 Fig) but the two main peaks in real data were correctly represented by both models (S12 Fig)). The AATS value for WGAN AGs showed underfitting (AATS = 0.91) whereas the value for CRBM was slightly above the 0.5 sweet spot (AATS = 0.56; Fig 5D. However, there was a huge contrast between AAtruth and AAsyn values for CRBM AGs (AAtruth = 0.86, AAsyn = 0.26), which might be an indication of the anomaly depicted in Fig 4D. This is the opposite of what we observed for the previous GAN (GAN_prev) model and might be due to AGs which exist outside the real data space as can be seen from the PCA analysis (S13 Fig). A general comparison of the two models based on multiple aggregated statistics is provided in S11 Fig. To further investigate the anomaly for the CRBM AGs, we performed a nearest neighbour chain analysis (Table 1). Starting with a synthetic point, we observed substantially higher frequencies for chains of true data points (ST, STT, STTT, STTTT) compared to chains of synthetic data points (SS, SSS, SSSS, SSSSS). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Nearest neighbour chain analysis. Frequencies of series of generated (synthetic—S) and real (truth—T) samples in chains of nearest neighbours (from size 2 to 5). To avoid loops, a sample was removed once reached in the chain. Expected frequency for chains of size 2 is 0.25, size 3 is 0.125, size 4 is 0.0625 and size 5 is 0.03125. https://doi.org/10.1371/journal.pcbi.1011584.t001 Membership inference attacks and privacy leakage Assessing privacy leakage and performing membership inference attacks require an additional test set but we could not obtain good quality AGs using a subset of 65,535-SNP dataset. Therefore, we trained the new WGAN and RBM models using half of the samples from the 10,000-SNP dataset (2504 haplotypes). AGs generated via the WGAN model had similar summary statistics to the AGs generated via the model trained with the whole dataset. For the RBM model, results were slightly worse but better than the previous RBM model (RBM_prev) trained with the whole dataset (S14A–S14C Fig). AATS analysis showed underfitting for WGAN AGs (AATS = 0.80, AAtruth = 0.76, AAsyn = 0.83) and almost perfect scores for RBM AGs (AATS = 0.50, AAtruth = 0.49, AAsyn = 0.51). Based on the test set, WGAN AGs had good privacy score (0.03) and RBM AGs showed possible privacy leakage (0.23) (S14D Fig). Furthermore, both white-box attack on WGAN AGs and black-box attack on RBM AGs reached high accuracies (WGAN: 1%: 0.82, 10%: 0.826, 25%: 0.754, 50%: 0.677; RBM: 1%: 0.84, 10%: 0.70, 25%: 0.62, 50%: 0.57) for detecting a portion of the training samples whereas black-box attack on WGAN AGs produced substantially lower accuracies (1%: 0.56, 10%: 0.55, 25%: 0.54, 50%: 0.51), suggesting relatively better privacy preservation. The distribution of the critic scores for train and test samples also showed that the black-box attack on WGAN AGs was mainly unsuccessful for differentiating training and test samples (S15 Fig). Discussion In this study, we implemented generative neural networks for large-scale genomic data generation and assessed various characteristics of the artificial genomes (AGs) generated by these networks. Initially, we generated AGs from the 10,000-SNP dataset to be able to compare to the previous results from [21]. For this dataset, we introduced a new RBM training scheme, a convolutional WGAN and a convolutional VAE. Both WGAN and new RBM models substantially improved the quality of AGs in terms of all summary statistics. For the 65,535-SNP dataset, we used a similar WGAN architecture (which is essentially deeper in comparison to the architecture for the 10,000-SNP dataset) and a conditional RBM (CRBM) protocol. AGs generated by these new models preserve population structure, allele frequency distribution and haplotypic integrity of real genomes reasonably well with little or no privacy leakage from the training data. Generative models trained with similar-sized genomic data have been reported in the literature but the main goal of these studies was characterization of population structure via dimensionality reduction and the generated genomes did not possess good haplotypic integrity [27, 40]. There have been other studies focusing on demographic parameter estimation [41] and data generation [8–11] for population genetics but these only included training with smaller genomic segments. The upscaling to a larger sequence size in comparison to our previous study is an essential step for AGs to be utilised in real-life applications as publicly accessible alternatives for sensitive genomic samples, yet obstacles remain. Although we could generate substantially larger sequences by incorporating convolutions for the WGAN and a conditional approach for the RBM, training these large models requires computational time and fine-tuning in most cases. The WGAN is usually preferred over naive GAN in literature as it mitigates hyperparameter optimization and provide more stable training [22]. However, we had to go through multiple combinations of architectures and hyperparameter values to reach satisfying results. Even then, the training in our application involved further adjustment of the generator learning rate after a certain epoch to capture the details of the population structure observable in higher PC dimensions. This second round of training was helpful but some of the runs did not reach an acceptable equilibrium (i.e., generated genomes had low quality after training), especially when experimenting with reduced training sample sizes. For future applications, automated architecture and hyperparameter tuning methods can help significantly for finding the optimal combinations without extensive trial and error [42, 43]. VAEs might be seen as better alternatives in terms of training stability compared to GANs since they do not suffer from the difficulty of balancing two networks in an adversarial manner, yet we could not obtain high-quality AGs for the 65,535-SNP dataset. Presumably, considerably better outcomes can be achieved with further architectural, hyperparameter and loss function exploration, which is outside the scope of this study. Another future research direction for the generative models could be the exploration of alternative stopping criteria for the training. Unlike image generation, where the generated outcomes can be assessed easily by visual inspection, assessment of generated genomic data is not trivial. For VAE and WGAN models, we decided to use PCA plots for initial inspection and as the stopping criterion since the highest variation in most genomic datasets is due to population structure which PCA captures well. This PCA match, inspected visually or measured through Wassertein distance, had been used previously as the stopping criterion [8, 21]. Although an obvious candidate for the stopping criterion of WGAN is the convergence of the critic’s loss to a value close to zero (since this loss provides an estimation for the Earth mover’s distance between real and generated data), we observed that it does not always coincide with good PCA outcomes for the AGs. For the RBM models, we had to inspect AATS score instead since overfitting was a more prominent issue. A possible alternative to these would be using some aggregate statistics crafted based on multiple summary statistics related to LD, site frequency spectrum, ancestry and overfitting scores. The other approach we presented to incorporate large-scale data was the conditional training of RBMs. Instead of a single training of a large and deep model as in the case of WGAN, CRBM training includes multiple training runs of a small model. A main advantage of this method is that any sequence size can theoretically be generated as long as the positional conditionality is not broken over the target genome segment. Therefore, the only bottleneck in terms of computation is the time needed to train all the RBMs. Currently, the learned weights are specific to each machine and no parameters are shared between RBMs. This advantageously allows parallel training when needed, but produces a large number of parameters to be optimized since the number of weights and biases has to be multiplied by the number of machines. Further studies could investigate whether parameter sharing between machines (i.e., between regions) and/or within each machine (through convolutions) is advantageous or instead complicates training. In addition, many datasets also present a difficulty during training, because the equilibrating time for Markov chains diverges as the training goes on, causing issues for sampling (and re-sampling) [44]. To overcome this, we relied on out-of-equilibrium training as described in Materials and Methods. While this approach improved training stability and also the quality of the generated samples with respect to other approaches, it has the disadvantage that the learned features cannot be easily interpreted. For the assessment of overfitting and underfitting, we utilised the AATS score and haplotypic pairwise distances as in [21]. An interesting finding was the large difference between two terms of the AATS score (AAtruth and AAsyn) for AGs generated by the CRBM even though the averaged AATS score was good. Interestingly, this is the opposite of what we had observed for our previously published GAN model and possibly points to the type of generative aberration demonstrated in Fig 4D. In fact, it is known that the likelihood function used to infer the parameters of RBMs tends to create potentially spurious modes -that do not match any region of the dataset-, which might explain these patterns. For further investigation, we performed a nearest neighbour chain analysis since we would expect the nearest neighbours of the nearest neighbours for CRBM AGs to be mostly real genomes with this anomaly (Table 1). High frequency of ST, STT, STTT, STTTT and low frequency of SS, SSS, SSSS, SSSSS measurements provide additional evidence that what we observe for the CRBM AGs is possibly similar to the scenario seen in Fig 4D. To the best of our knowledge, this type of anomaly for generative models has not been reported in literature and could have been missed by classical evaluation metrics. Such complex overfitting/underfitting phenomena require vigilance and further investigation in future generative studies. We furthermore performed membership inference attacks on AGs generated by WGAN and RBM models to assess possible privacy leakage. While the white-box attack on WGAN AGs and black-box attack on RBM AGs produced high accuracy for detecting a portion of the samples used in training, black-box attack on WGAN AGs was mainly not successful (S15 Fig). This indicates that even if the model architecture is available publicly and the adversary is in possession of some samples from the training dataset, it is not trivial to pinpoint the training samples via this attack without access to the model weights. We highlight that providing the critic’s weights of a GAN is not useful to the general user interested only in its generative properties, the white-box is thus a conservative attack (i.e., unrealistically beneficial for the adversary). Similarly, both the white and black-box attacks are conservative for evaluating privacy leaks as they assume that the adversary has the huge advantage of already possessing some genetic sequences from the original training set. However, it is crucial to underline here that this is only a single type of attack and more research on privacy preservation is essential before AGs are used in real-life scenarios [45]. Despite these issues which remain to be further studied, additional improvements in model training and increased privacy guarantees for generated genomes can pave the way for the first artificial genome banks in the near future, accelerating global access to the vast amount of restricted genomic data. Unlike conventional approaches for genomic simulations, generative neural networks do not require a priori information about the target dataset (such as underlying evolutionary history), allow the generated data to be utilized alongside real data and provide substantially better privacy outcomes in comparison to haplotype copying methods [21]. Supporting information S1 Fig. Architecture of the variational autoencoder (VAE) model for the 10,000 SNP dataset. Generic blocks of the encoder and decoder (white rectangles) are conceptually the same with the generic critic and generator blocks respectively (Fig 1A), except that there are no latent space channels concatenated to the input and no additional noise vectors at each block. The major difference from WGAN in terms of architecture is the last block of the encoder, which encodes mu and sigma as the mean and the standard deviation of the generated distribution, which are used to sample the latent space. Dotted connections are residual connections where the input value is added to the output value of the block before passing to the next block. Numbers in parentheses above blocks show channels and length, respectively (C, L). https://doi.org/10.1371/journal.pcbi.1011584.s001 (TIF) S2 Fig. Comparative linkage disequilibrium (LD) analysis of artificial genomes generated by different models for the 10,000-SNP data. a) LD heatmap based on r2 matrices. Sections below diagonals correspond to LD in real genomes and sections above diagonals correspond to LD in artificial genomes. b) LD decay as a function of SNP distance. SNPs were binned based on distance and average LD was calculated. c) LD decay correlation between real and artificial datasets. x axis corresponds to real LD bins and y axis corresponds to generated LD bins. Sites fixed in any of the datasets were removed for all the LD calculations. https://doi.org/10.1371/journal.pcbi.1011584.s002 (TIF) S3 Fig. 3-point correlation analysis of SNP triplets for the 10,000-SNP data with inter-SNP distances of 1, 4, 16, 64, 256, 512 and 1024 (from left to right, top to bottom). The last panel (bottom right) shows correlation for triplets of SNPs drawn randomly. In each plot, drawing order (z-order) of each AG group is shuffled. https://doi.org/10.1371/journal.pcbi.1011584.s003 (TIF) S4 Fig. Analysis of small haplotype motifs between real and generated 10,000-SNP datasets for a) 4-mer and b) 8-mer non-overlapping windows. For each unique k-mer in each window, number of occurrences in the real dataset was compared to the same number in the AG dataset. Each point corresponds to the occurrence number in real (x-axis) and AG (y-axis) datasets. Values presented inside the figures are Pearson’s r, ordinary least squares regression slope and intercept. https://doi.org/10.1371/journal.pcbi.1011584.s004 (TIF) S5 Fig. Distribution of haplotypic pairwise difference within (left figure) and between (right figure) 10,000-SNP datasets. https://doi.org/10.1371/journal.pcbi.1011584.s005 (TIF) S6 Fig. Nearest neighbour adversarial accuracy (AATS) of artificial genomes generated by different models for the 10,000-SNP dataset. Values below 0.5 (black line) indicate overfitting and values above indicate underfitting. See Materials and methods for the details of the metrics. https://doi.org/10.1371/journal.pcbi.1011584.s006 (TIF) S7 Fig. Radar plot comparing artificial genomes generated by different models for the 10,000-SNP dataset. Values closer to 0 indicate poor performance whereas values closer to 1 indicate good performance. See Materials and methods for the details of the representative statistics. https://doi.org/10.1371/journal.pcbi.1011584.s007 (TIF) S8 Fig. Analysis of fixed alleles in artificial genomes with 65,535 SNPs. Left figures show the number of fixed alleles in artificial genomes (x axis) versus the frequency of these alleles in the real dataset (y axis). Right figures show the distribution of the frequency of alleles fixed in the artificial dataset but not fixed in the real dataset. https://doi.org/10.1371/journal.pcbi.1011584.s008 (TIF) S9 Fig. 3-point correlation analysis of SNP triplets for the 65,535-SNP data with inter-SNP distances of 1, 4, 16, 64, 256, 512 and 1024 (from left to right, top to bottom) for a) WGAN and b) CRBM AGs. The last panels (bottom right) shows correlation for triplets of SNPs drawn randomly. In each plot, drawing order (z-order) of each AG group is shuffled. https://doi.org/10.1371/journal.pcbi.1011584.s009 (TIF) S10 Fig. Analysis of small haplotype motifs between real and generated 65,535-SNP datasets for a) 4-mer and b) 8-mer non-overlapping windows. For each unique k-mer in each window, number of occurrences in the real dataset was compared to the same number in the AG dataset. Each point corresponds to the occurrence number in real (x-axis) and AG (y-axis) datasets. Values presented inside the figures are Pearson’s r, ordinary least squares regression slope and intercept. https://doi.org/10.1371/journal.pcbi.1011584.s010 (TIF) S11 Fig. Radar plot comparing artificial genomes generated by WGAN and CRBM models for the 65,535-SNP dataset. Values closer to 0 indicate poor performance whereas values closer to 1 indicate good performance. See Materials and methods for the details of the representative statistics. https://doi.org/10.1371/journal.pcbi.1011584.s011 (TIF) S12 Fig. Distribution of haplotypic pairwise difference within (left figure) and between (right figure) 65,535-SNP datasets. https://doi.org/10.1371/journal.pcbi.1011584.s012 (TIF) S13 Fig. Principal component analysis (PCA) of combined real and artificial genomes with 65,535 SNPs. https://doi.org/10.1371/journal.pcbi.1011584.s013 (TIF) S14 Fig. Analysis of WGAN and RBM generated AGs using 2504 samples from 10,000-SNP dataset. a) Principal component analysis (PCA) of combined real and artificial genomes. b) Allele frequency correlation between real (x-axis) and artificial (y-axis) genome datasets. Bottom figures are zoomed at low frequency alleles (from 0 to 0.2 overall frequency in the real dataset). Values presented inside the figures are Pearson’s r, ordinary least squares regression slope and intercept. c) Nearest neighbour adversarial accuracy (AATS) of artificial genomes generated by different models and the test set. Values below 0.5 (black line) indicate overfitting and values above indicate underfitting. d) Privacy score for WGAN and RBM generated AGs. Values close to 0 indicate no privacy leakage. https://doi.org/10.1371/journal.pcbi.1011584.s014 (TIF) S15 Fig. Membership inference attack on generated 10,000-SNP genomes. a) White-box attack (adversary has access to the model architecture and weights) on WGAN AGs and black-box attacks with auxiliary information (adversary has only access to the model architecture) on b) WGAN and c) RBM AGs. For all attacks, the adversary is assumed to know the size of the training set (2504 in this analysis) and possesses a set of samples (5008 in this analysis) suspected of belonging to the training data. For each attack, the critic scores the samples and the adversary sets a threshold for assigning the top n scoring samples to the training dataset. Figures in the upper row show the accuracy of attacks depending on these thresholds (assigned samples ranging from the top 1% to the top 50%). The red dashed lines indicate the accuracy if the n samples were chosen randomly and not based on their scores. Figures in the lower row show the distribution of the critic score for train and test datasets. See Materials and methods for more details. https://doi.org/10.1371/journal.pcbi.1011584.s015 (TIF) Acknowledgments Thanks to the Inria TAU team of the University of Paris-Saclay and the High Performance Computing Center of the University of Tartu for providing computational resources.

Vecchiato, Giacomo;Hattermann, Tom;Palladino, Michele;Tedone, Fabio;Heuret, Patrick;Rowe, Nick P.;Marcati, Pierangelo

doi: 10.1371/journal.pcbi.1011538pmid: 37844126

Introduction The seminal work of Charles Darwin (1865) on the support-searching movements of climbing plants opened up new paths of thinking on plant life histories. Known to be the longest plants on land, climbing plants exhibit a wide diversity of mechanical architectures and ecological strategies [1] across many different phylogenetic groups [2]. Unlike trees that remain self-supporting throughout their entire life cycle, climbing plants are well-known for relying on physical support to grow vertically and towards light. To exploit supports, climbing plants rely on a developmental phase, known as the “searcher shoot”, which is specialized in crossing gaps, and searching and attaching to supports [3, 4]. To achieve these tasks, searcher shoots combine primary and secondary growth processes and actively modulate their development according to internal and external stimuli [5]. For example, it is well known how the contact with a support induces specific growth responses (i.e. thigmomorphogenesis), but it remains poorly understood how the searcher shoot respond and interact with gravity during different kinds of exploratory tasks, such as growing in a specific direction or exploring volumes by circumnutational movements. Plant shape and movement as a stimuli-response phenomenon The responses that led to a turning movement and alignment of growth of a plant organ with respect to external vectorial cues have been known for many years as tropisms [6, 7]. A plant organ may tend to orient or grow towards a stimulus (positive tropism), or away from it (negative tropism) [8]. For example, many aerial shoots of plants grow upwards against gravity whereas most roots tend to grow downwards [7] to locate the ground. Both organs perceive gravity via modified plastids (statholits) within specialised cells (statocytes), but the growth responses are induced by different biochemical signalling and auxin flux carriers [9, 10]. Supported by mathematical analysis and models, the gravitational responses have been linked with the statholits sedimentation, as well as the diffusion of auxin, and they are considered in terms of differential growth within the plant body [11–14]. Given the complexity of interaction between the growing activity of plants and external and internal cues, growth movements can be viewed as an integrative response to diverse specific stimuli-response processes. Brief review of mathematical models One of the earliest studies investigating the stimuli-response behaviour in plants was carried out by Julius Sachs in 1882 [15]. As a result of his observations, the formation of curvatures along the plant body against gravity had first only been linked to gravity perception during primary growth. The upward bending movement of the shoot being stronger in the horizontal position then the vertical one has been described according to the “sine law” because it can be reformulated with the relation ∂tκ ∝ sin θ. Here ∂tκ is the curvature change rate and θ is the inclination of the stem. This fundamental relation between curvature and inclination has remained in the biological cultural framework for over a century. Only in 2013 the mathematical studies of Bastien et al. [16] demonstrated that if the phenomenon of the response to gravity is described only on the base of the sine law, the plant can never reach a vertical steady state. This is due to infinite lateral oscillations that would arise during the upward growth of the shoot. To stabilise the self-supporting system, the sine law was modified with a positive proprioceptive term, becoming ∂tκ ≈ sin θ − κ. This additional term tends to regulate high curvature κ towards 0 in time. In parallel with the sensing activity of the plants, a wide literature has been developed on the physical and mechanical properties of shoot growth [17–23]. Knowledge of developmental changes and physical parameters, such as diameter, length and stiffness have been considered as the main descriptors of the stem shape. Founded on the Euler-Bernoulli beam theory, these mathematical models assume that a plant shoot behaves as a growing elastic rod, and they are mainly based on two configurations: (i) the current configuration, which corresponds to the actual shape of the elastic rod when subject to gravity or any external forces, and (ii) the relaxed or intrinsic configuration, that is the shape of the rod in the weightless limit case. In particular, in case (ii), the shape of the rod is described by a purely geometric evolution equation, which neglects mechanical effects. Further details about these configurations are given in section. Recent studies have also combined analysis on gravitropism and proprioception along with analyses on the mechanics of growth, including a planar model of a growing plant [24]. Here the elastic rod is subject to the effects of gravity and develops according to the sine law corrected via a proprioceptive term as proposed in [16]. Some excellent guidelines in the modelisation of the sensing activity and its interaction with the plant growth mechanics can be found in [25–27]. In these models the formation and the growth of tissue layers, resulting from the secondary growth, together with the development of the intrinsic curvature in function of the shoot inclination and the proprioception have never been considered [27]. For example, some models have included a linear density parameter ρ, which is constant along the stem and doesn’t change over time (for instance, [11, 24]). This potentially limits two important features characterising actual shoot growth: first, it assumes that the shape, the size of the stem cross-section and any internal growth expansion of mechanical tissue (e.g., of the wood cylinder) remain constant over time. Second, it does not take into account the build-up of mass along the shoot due to secondary growth of the wood cylinder and other tissues. Early development of additive growth, maturation and stiffening of mechanical tissue via early secondary growth are key development features in plant shoots in general and especially in young searcher shoots of climbing plants [28–30]. Stem stiffness and rigidity can be significantly modified by even small changes of expansion of the wood cylinder within the primary body of the plant stem even before noticeable changes in external stem diameter [3]. However, this is only one of the possible ways in which a searcher shoot can adjust its mechanical properties. From its base to the apex, a shoot can adjust its rigidity by decreasing the radius, modifying the structure and the chemistry of tissues [31] or modifying the gradient of tissue stiffness along the stem. The complexity of the interaction between all these mechanisms likely leads to a very complicated gradient of organisation, which is arguably difficult to capture or integrate using a single unifying model. Purpose of the study The aim of this study is to display the relevance of the mechanics in the behaviour of a climbing plant searcher shoot, considering in particular the radial expansion of the main stem. To this end, we first develop a mathematical model able to capture a variety of shapes and orientations observed in climbing plant searchers; second, we develop an approach to reconstruct extensional growth against gravity from a static description of the shoot final state. This approach aims to use a minimal number of parameters, which can be relatively easily obtained from selected field observations of different species with variable behaviours. In particular, we aim to show how the interplay between variable linear density, proprioception and external stimuli can generate variable shapes and orientations of searcher shoots observed in two different climbing plants species in the family of Apocynaceae: Trachelospermum jasminoides (Lindl.) Lem. and Condylocarpon guianense Desf. These relatively close-related species have been chosen because they share some fundamental properties during their searching and twining growth behaviour. Both species (i) attach to support by twining; (ii) are capable of reaching similar maximal reach capacities of around 110 cm in length (iii) and have similar values of structural Young’s modulus at the base of searcher shoots of around 3000 N m−2 [29, 30]. In order to model a generically directed stimulus we consider the equations used by Guillon et al. [21] corrected with the proprioceptive term introduced by Bastien et al. [16]. The resulting growth dynamics is addressed through numerical simulations. Some recent studies have used numerical tools based on arbitrary parameters to illustrate generic mechanical behaviours through simulations [11, 12, 24]. Rather than arbitrarily calibrating every parameter, we set their order of magnitude using measured data from the two climbing plant species T. jasminoides and C. guianense growing in natural conditions. This enabled us to calibrate the morphological parameters as well as the measurements of internode length at different times, which were crucial for estimating the growth parameters. Glossary In this study, we use some technical terms that might not be familiar or that need clarification for a diverse scientific community. To avoid confusion, we present a short glossary here. Searcher shoot: Part of a climbing plant responsible for spanning gaps, foraging for support and attaching. A searcher shoot always includes a main axial structure (most often a stem) which is mechanically self-supporting from the base. According to the species, it may bear different structures such as leaves, and branches, as well as structures modified into attachment systems such as tendrils and stem segments capable of twining. Searcher stems can often undergo growth-induced movements specialized in exploring its vicinity and support attachment. Reach: The effective length observed between the apical tip of a searcher shoot and the basal point from which it is attached or fixed (see Fig 1). Maximal reach can be viewed as a functional descriptor of searcher-shoot gap-spanning capacity in a functional and ecological context. Orientation: The slope of the line joining the base with the tip of a searcher stem in a vertical plane. Here, the orientation was measured only at the final configuration when the searcher shoot was estimated to have attained its maximal reach in a self-supporting state (see Fig 1). Primary Growth: The increase in growth results from cell division in the apical meristem of axes and subsequent cell elongation and maturation. These lead to the growth in length of the searcher stem. In this paper we use the term extension as the morphogenetic additive process leading to the “lengthening” of a shoot during primary growth. Hence, the term describes a macroscopic phenomenon that implicitly includes smaller-scale growth processes such as: cell initiation, multiplication, differentiation and maturation. Secondary Growth: The increase in growth that results from cell division in the vascular cambium; a ring-like meristematic tissue producing wood (secondary xylem and ray tissue) and secondary phloem. Subsequent expansion and maturation of additionally formed cells lead to a radial thickening of the stem. We use the term rigidification to refer to the process of stem thickening and stiffening that is achieved by the stem during secondary growth. Proprioception: The capability of the shoot to perceive changes in shape and orientation in terms of curvature and to respond to these changes in order to restore local straightness. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. (1A): Reach and orientation in searcher stems. Measurement of reach and orientation in a typical searcher shoot of a climbing plant. The reach is measured as a straight line form the base of the searcher stem at its point of attachment with the parent bearing stem to the curved hook-like apex. It represents the effective distance a self-supporting searcher stem often capable of movement towards the apex. The orientation is the slope of that joining line with respect to the horizontal line. (1B): Schematic illustration of the curve elaborated by the mathematical model. At each time t of searcher shoot growth, the mathematical model represents its position in the space with a curve r. In the figure, e1 and e2 are the two orthogonal vectors that span the plane in which the curve is confined. So, these vectors correspond respectively to the (x, y) coordinates of the curve. For a given time t, any point of the curve is identified with the vector r(S, t), where S is a parameter which varies in [0, ℓ0]. The figure shows that the mathematical model leads the point of the curve at time zero with position r(S, 0) to the point at time t with position r(S, t). Each point r(S, t) of the curve has a certain inclination to the vertical line, denoted by θ(S, t). ℓg is the parameter used by the model as length of the extension zone. The sensing equation and the growth involve only the points of that zone. https://doi.org/10.1371/journal.pcbi.1011538.g001 Plant shape and movement as a stimuli-response phenomenon The responses that led to a turning movement and alignment of growth of a plant organ with respect to external vectorial cues have been known for many years as tropisms [6, 7]. A plant organ may tend to orient or grow towards a stimulus (positive tropism), or away from it (negative tropism) [8]. For example, many aerial shoots of plants grow upwards against gravity whereas most roots tend to grow downwards [7] to locate the ground. Both organs perceive gravity via modified plastids (statholits) within specialised cells (statocytes), but the growth responses are induced by different biochemical signalling and auxin flux carriers [9, 10]. Supported by mathematical analysis and models, the gravitational responses have been linked with the statholits sedimentation, as well as the diffusion of auxin, and they are considered in terms of differential growth within the plant body [11–14]. Given the complexity of interaction between the growing activity of plants and external and internal cues, growth movements can be viewed as an integrative response to diverse specific stimuli-response processes. Brief review of mathematical models One of the earliest studies investigating the stimuli-response behaviour in plants was carried out by Julius Sachs in 1882 [15]. As a result of his observations, the formation of curvatures along the plant body against gravity had first only been linked to gravity perception during primary growth. The upward bending movement of the shoot being stronger in the horizontal position then the vertical one has been described according to the “sine law” because it can be reformulated with the relation ∂tκ ∝ sin θ. Here ∂tκ is the curvature change rate and θ is the inclination of the stem. This fundamental relation between curvature and inclination has remained in the biological cultural framework for over a century. Only in 2013 the mathematical studies of Bastien et al. [16] demonstrated that if the phenomenon of the response to gravity is described only on the base of the sine law, the plant can never reach a vertical steady state. This is due to infinite lateral oscillations that would arise during the upward growth of the shoot. To stabilise the self-supporting system, the sine law was modified with a positive proprioceptive term, becoming ∂tκ ≈ sin θ − κ. This additional term tends to regulate high curvature κ towards 0 in time. In parallel with the sensing activity of the plants, a wide literature has been developed on the physical and mechanical properties of shoot growth [17–23]. Knowledge of developmental changes and physical parameters, such as diameter, length and stiffness have been considered as the main descriptors of the stem shape. Founded on the Euler-Bernoulli beam theory, these mathematical models assume that a plant shoot behaves as a growing elastic rod, and they are mainly based on two configurations: (i) the current configuration, which corresponds to the actual shape of the elastic rod when subject to gravity or any external forces, and (ii) the relaxed or intrinsic configuration, that is the shape of the rod in the weightless limit case. In particular, in case (ii), the shape of the rod is described by a purely geometric evolution equation, which neglects mechanical effects. Further details about these configurations are given in section. Recent studies have also combined analysis on gravitropism and proprioception along with analyses on the mechanics of growth, including a planar model of a growing plant [24]. Here the elastic rod is subject to the effects of gravity and develops according to the sine law corrected via a proprioceptive term as proposed in [16]. Some excellent guidelines in the modelisation of the sensing activity and its interaction with the plant growth mechanics can be found in [25–27]. In these models the formation and the growth of tissue layers, resulting from the secondary growth, together with the development of the intrinsic curvature in function of the shoot inclination and the proprioception have never been considered [27]. For example, some models have included a linear density parameter ρ, which is constant along the stem and doesn’t change over time (for instance, [11, 24]). This potentially limits two important features characterising actual shoot growth: first, it assumes that the shape, the size of the stem cross-section and any internal growth expansion of mechanical tissue (e.g., of the wood cylinder) remain constant over time. Second, it does not take into account the build-up of mass along the shoot due to secondary growth of the wood cylinder and other tissues. Early development of additive growth, maturation and stiffening of mechanical tissue via early secondary growth are key development features in plant shoots in general and especially in young searcher shoots of climbing plants [28–30]. Stem stiffness and rigidity can be significantly modified by even small changes of expansion of the wood cylinder within the primary body of the plant stem even before noticeable changes in external stem diameter [3]. However, this is only one of the possible ways in which a searcher shoot can adjust its mechanical properties. From its base to the apex, a shoot can adjust its rigidity by decreasing the radius, modifying the structure and the chemistry of tissues [31] or modifying the gradient of tissue stiffness along the stem. The complexity of the interaction between all these mechanisms likely leads to a very complicated gradient of organisation, which is arguably difficult to capture or integrate using a single unifying model. Purpose of the study The aim of this study is to display the relevance of the mechanics in the behaviour of a climbing plant searcher shoot, considering in particular the radial expansion of the main stem. To this end, we first develop a mathematical model able to capture a variety of shapes and orientations observed in climbing plant searchers; second, we develop an approach to reconstruct extensional growth against gravity from a static description of the shoot final state. This approach aims to use a minimal number of parameters, which can be relatively easily obtained from selected field observations of different species with variable behaviours. In particular, we aim to show how the interplay between variable linear density, proprioception and external stimuli can generate variable shapes and orientations of searcher shoots observed in two different climbing plants species in the family of Apocynaceae: Trachelospermum jasminoides (Lindl.) Lem. and Condylocarpon guianense Desf. These relatively close-related species have been chosen because they share some fundamental properties during their searching and twining growth behaviour. Both species (i) attach to support by twining; (ii) are capable of reaching similar maximal reach capacities of around 110 cm in length (iii) and have similar values of structural Young’s modulus at the base of searcher shoots of around 3000 N m−2 [29, 30]. In order to model a generically directed stimulus we consider the equations used by Guillon et al. [21] corrected with the proprioceptive term introduced by Bastien et al. [16]. The resulting growth dynamics is addressed through numerical simulations. Some recent studies have used numerical tools based on arbitrary parameters to illustrate generic mechanical behaviours through simulations [11, 12, 24]. Rather than arbitrarily calibrating every parameter, we set their order of magnitude using measured data from the two climbing plant species T. jasminoides and C. guianense growing in natural conditions. This enabled us to calibrate the morphological parameters as well as the measurements of internode length at different times, which were crucial for estimating the growth parameters. Glossary In this study, we use some technical terms that might not be familiar or that need clarification for a diverse scientific community. To avoid confusion, we present a short glossary here. Searcher shoot: Part of a climbing plant responsible for spanning gaps, foraging for support and attaching. A searcher shoot always includes a main axial structure (most often a stem) which is mechanically self-supporting from the base. According to the species, it may bear different structures such as leaves, and branches, as well as structures modified into attachment systems such as tendrils and stem segments capable of twining. Searcher stems can often undergo growth-induced movements specialized in exploring its vicinity and support attachment. Reach: The effective length observed between the apical tip of a searcher shoot and the basal point from which it is attached or fixed (see Fig 1). Maximal reach can be viewed as a functional descriptor of searcher-shoot gap-spanning capacity in a functional and ecological context. Orientation: The slope of the line joining the base with the tip of a searcher stem in a vertical plane. Here, the orientation was measured only at the final configuration when the searcher shoot was estimated to have attained its maximal reach in a self-supporting state (see Fig 1). Primary Growth: The increase in growth results from cell division in the apical meristem of axes and subsequent cell elongation and maturation. These lead to the growth in length of the searcher stem. In this paper we use the term extension as the morphogenetic additive process leading to the “lengthening” of a shoot during primary growth. Hence, the term describes a macroscopic phenomenon that implicitly includes smaller-scale growth processes such as: cell initiation, multiplication, differentiation and maturation. Secondary Growth: The increase in growth that results from cell division in the vascular cambium; a ring-like meristematic tissue producing wood (secondary xylem and ray tissue) and secondary phloem. Subsequent expansion and maturation of additionally formed cells lead to a radial thickening of the stem. We use the term rigidification to refer to the process of stem thickening and stiffening that is achieved by the stem during secondary growth. Proprioception: The capability of the shoot to perceive changes in shape and orientation in terms of curvature and to respond to these changes in order to restore local straightness. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. (1A): Reach and orientation in searcher stems. Measurement of reach and orientation in a typical searcher shoot of a climbing plant. The reach is measured as a straight line form the base of the searcher stem at its point of attachment with the parent bearing stem to the curved hook-like apex. It represents the effective distance a self-supporting searcher stem often capable of movement towards the apex. The orientation is the slope of that joining line with respect to the horizontal line. (1B): Schematic illustration of the curve elaborated by the mathematical model. At each time t of searcher shoot growth, the mathematical model represents its position in the space with a curve r. In the figure, e1 and e2 are the two orthogonal vectors that span the plane in which the curve is confined. So, these vectors correspond respectively to the (x, y) coordinates of the curve. For a given time t, any point of the curve is identified with the vector r(S, t), where S is a parameter which varies in [0, ℓ0]. The figure shows that the mathematical model leads the point of the curve at time zero with position r(S, 0) to the point at time t with position r(S, t). Each point r(S, t) of the curve has a certain inclination to the vertical line, denoted by θ(S, t). ℓg is the parameter used by the model as length of the extension zone. The sensing equation and the growth involve only the points of that zone. https://doi.org/10.1371/journal.pcbi.1011538.g001 Materials and methods Formulation of the model: Mechanics and growth Notations for modelling a planar morphoelastic rod. We model the searcher shoot using the theory of morphoelastic rods [32]. Hence, we are assuming that climbing plants’ searcher shoot behaves like an elastic rod that at each time t changes its length and shape. In this framework, we make the following assumptions: (i) at each time t ∈ [0, T], where t = 0 and t = T are respectively the initial and the final time of the shoot’s development, the elastic rod’s centreline is confined in a plane spanned by two orthonormal vectors {e1, e2}; (ii) the rod is inextensible and unshearable. With these premises, the rod’s model is characterised by three elements (which have been partially introduced in Section: The intrinsic configuration at the initial time t = 0 (also called reference configuration). We assume that this configuration is just a straight line along e1 with arc-length parameter S ∈ [0, ℓ0]; The current configuration at a generic time t. We denote the corresponding centreline curve with r(s, t) ∈ span{e1, e2}, where s ∈ [0, ℓ(t)] is the arc-length parameter. It is convenient to describe the position of this curve using the angle θ(s, t) between the tangent vector ∂sr(s, t) and the vector e2 (see Fig 1). The symbol ∂s denotes the partial derivative with respect to the parameter s (a similar notation is used for ∂t and ∂S). Hence, the quantity ∂sθ(s, t) gives the curvature of this configuration; The intrinsic configuration at a generic time t. We only need to fix two notations about this configuration: (i) Since the rod is inextensible, this configuration has the same arc-length parameter s ∈ [0, ℓ(t)] of the current configuration; (ii) we indicate with κi(s, t) the curvature. Equations of balance with the gravity force. We assume that the only force acting on the rod is the gravity. Using the Kichhoff equations for the elastic rod in the static case (see [32] or Section A in S1 Text for further details), we get (1) This relation expresses the balance of gravity with the internal forces and moments (of force) developed by the rod in response. In the notation of Eq (1), g is the gravity acceleration constant, B(s, t) the flexural rigidity of the rod at r(s, t), ml(s, t) the mass of all the leaves between the point r(s, t) and the tip r(ℓ(t), t), and ρ(σ, t) the linear density at the point r(σ, t). The term linear density indicates the mass per unit of length. This quantity is different from the mass per unit of volume, which we refer to as volume density and denote with ρ3. If we assume that the rod has a circular cross-section with radius R(s, t) at the point r(s, t), the relation between these two quantities is simply given by Modeling the shoot extension. We consider primary growth as the result of a “growth from the tip”, so we are assuming that the addition of mass occurs only at the tip. As a result of such a process, a point initially located at r(S, 0) for some S ∈ [0, ℓ0], reaches the point r(s, t). The arc-length parameter s ∈ [0, ℓ(t)] at time t is a function of the initial arc-length parametrization S ∈ [0, ℓ0]. Assuming that the local extension occurs with a factor G0, the function s(S, t) has to satisfy the following partial differential equation: (2) where ℓg represents the length of the zone at the tip at which the extension occurs. Modeling the shoot rigidification. For modelling growing trees, the fundamental assumption is that the new layer of material formed during the secondary growth process do not affect the balance of the total forces and moments which were previously applied to the stem [33]. In other words, in an infinitesimal interval of time, the secondary growth does not affect the current configuration, which is described by the curvature ∂sθ, but affects the intrinsic configuration and its curvature κ. By applying the equations developed in [21] to our case (see Section B in S1 Text for further details, but also [20, 22, 33] for a biological and mechanical basis on the secondary growth), we get the following relation for the time evolution of the intrinsic curvature κ: (3) Equation for the tropisms: Directed stimulus, proprioception and secondary growth. External and internal stimuli affect the time evolution of the intrinsic configuration. To model this behaviour taking the rigidification into account, we need to make an assumption on the shape of the cross-section. As anticipated in Section, in our model the cross-section is a circle of radius R(s, t) at the point r(s, t). We can then complete Eq (3) with the terms of perception of a generically directed stimulus and proprioception [16, 21]: (4) where α and β represents the response to a directional stimulus, while γ represents the sensitivity with respect to the proprioception. We will refer to them as sensing parameters. δ is the parameter regulating the intensity of the radial expansion effect on the shoot development. vR represents the radial expansion rate. The notation α cos θ − β sin θ is equivalent to . Indeed, we can write the couple in polar coordinates by defining Hence, , and . Summary of the equations. We group the equations introduced in Sections—in the following system: (5) Regarding the boundary conditions, we are assuming that the searcher shoot is clamped at the base at an angle θ0. As stated in Section, at the initial time the intrinsic configuration consists in a straight rod, which means κ(⋅, 0) ≡ 0. The condition κ(ℓ(t), t) = ∂sθ(ℓ(t), t) means that there is no extra weight at the tip, hence the curvature is the same in both the intrinsic and the current configuration. As discussed in Sections C and D in S1 Text, to solve numerically system (5), we first rewrite all the equations with respect to the Lagrangian coordinates (S, t). Then, the system is numerically solved by discretizing it in time via the backward Euler method and integrating it in space using the finite element method. Application to experimental data Geometry and biomechanical properties of the samples. In a first experiment, we collected morphological and biomechanical data from five searcher shoots for each species. For each individual, we selected, as far as possible, the longest searcher shoots in a self-supporting state. In their natural position, we measured the reach (cm) and orientation (degrees from the horizontal) defined as a straight line from the base to the apex of the searcher shoot. The geometry of the shoot was described from the internodes of the main stem. For each successive internode, we measured its length and its median diameter obtained from the mean of two orthogonal measurements. Bending properties of the base of the searcher stem were measured using four-point bending tests on a stem segment constituted of several internodes [34]. Flexural rigidity EI (which is denoted with the function B in System (5)) and structural Young’s modulus Estr were calculated from applied bending forces plotted against maximum deflections. Up to five weights were applied manually and each deflection was measured by observation with a dissecting microscope on the apparatus. Weight increments were chosen according to the bending resistance of each sample. Weights were constituted of stainless-steel or brass ranging from 8 g (C. guianense) to 50 g (T. jasminoides). Span distances were defined as proportional to the mean elliptical diameter of the stem segment. The span support was 40 times greater than the diameter and was ranging from 107 mm to 218 mm for the longest. The load span was comprised between one-half and two-thirds of the span support and ranged from 67 mm to 120 mm. For three positions along the measured segment (basal, medial, apical), the vertical and horizontal diameter, dv and dh, were measured and the means were then used to calculate the second moment of area I of the axis. The experimental measurements show that the difference between dv and dh is small, so it is reasonable to assume a circular cross-section. The structural Young’s modulus of the stem (Estr) (MN/m2) was estimated by dividing measured values of EI by calculated values of I [35]. Extensional growth of the samples and radial expansion rate. In a second experiment, we monitored the shoot extension of young, self-supporting, searcher shoots over one week for 11 shoots of T. jasminoides and one month for 19 shoots of C. guianense (see section Materials and Methods in [29] for further information on the environmental conditions). Three dates at three days intervals were recorded for T. jasminoides and five dates at 7 days of interval for C. guianense. For both species, we applied the same protocol starting by defining a reference mark on the most basal node of each main stem. From this mark, we measured at each date: (i) the total length of the shoot (cm); (ii) the length of each successive internode up to and including the apex (cm). The natural organisation of the shoot into separate internodes allowed us to identify and measure which internodes elongated between two dates (see S2 Table). The data on the radius from the first experiment (discussed in Section) and the measurements of the extension from the second experiment allow us to compute the radial expansion rate vR. To accomplish this measurement, we consider the geometrical structure of the samples from the first experiment, i.e. the internodes. For each sample, we enumerate the internodes starting from the base. At the base of the i-th internode, we assign its arc-length parameter, which we denote with si. Hence, according to our notation, s1 is the arc length at the base of the first internode, so s1 = 0, and if j > i, then sj > si. Moreover, since sj > si, the point r(si, t) is closer to the base than r(sj, t), hence more mature. Consider the situation in which j = i + 1. Let Δti be the time it takes the plant to elongate by an amount equal to the length of the i-th internode, that is Assuming that the maturation process depends merely on the distance from the tip (see Section below), this implies that the radius R(si+i, t) is going to expand until reaching the value R(si+1, t + Δti) = R(si, t) after its distance from the tip has increased by the length of the i-th internode. Hence, to retrieve the expansion rate at the base of the i-th internode, we take the difference between the radii at its extremes and divide it by Δti: We retrieve the extension rate from the data of the second experiment, as explained in Section below. Fitting of the model’s biomechanical functions. The data provided by the first experiment were to estimate volume density ρ3, radius R, radial expansion speed vR, flexural rigidity B and leaves mass ml. Such an estimate was specific for each shoot, that is, for each shoot, we retrieved the parameters of the above-mentioned functions. The data from the first experiment describe the biomechanical properties of the samples at a fixed time of their development. To get information about the time evolution of those properties, we assume that they depend only on the distance from the tip. Hence, we have ρ3(s, t) = ρ3(ℓ(t) − s), R(s, t) = R(ℓ(t) − s) and the same for vR, B and ml (more precisely, the mass of a single leaf as explained below). In particular, we assume that where x = ℓ(t) − s is the distance from the tip and is the mass of a single leaf. The coefficients (a3, b3, c3), …, (al, bl, cl) are retrieved by fitting the functions with the experimental data (see Figures A-B in S1 Text). We choose a sigmoid for the flexural rigidity B and the mass of a single leaf because we assume that the greater the distance from the tip, the more their value increases. To retrieve the mass of the leaves ml from the mass of a single leaf at a certain point of the shoot r(s, t), it is sufficient to sum the single leaf masses at the internode basis whose arc-length is greater than s. To transpose this concept into an equation, we denote with I(t) the number of internodes in the shoot at time t, starting the enumeration from the base. For instance, if there are four internodes, I(t) = {1, 2, 3, 4}, where 1 represents the internode at the base and 4 is the internode at the tip. We denote with si(t) the arc length of the basis i-th internode (as for the radial expansion rate case). Then, we write the equation for ml as follows: where is the number of leaves at the base of the i-th internode at time t. The numerical scheme used to simulate System (5) results to be stable with respect to this fitting procedure. Indeed, by changing the function for the flexural rigidity B from a sigmoid to a polynomial, the resulting simulation of the main stem does not display any significant change. See Section G in S1 Text for further details. Estimate of the extension zone parameter ℓg and the local extension factor G0. We model the primary growth process as a growth from the tip, with an extension zone of length ℓg located at the apex and a local extension factor G0. We consider only the case in which the length of the shoot is greater than ℓg and consequently, the length of the shoot ℓ(t) follows a linear law (i.e. a constant extension rate , see for instance [32], Section 4.3.1). For this reason, we consider only the samples in which the extension process is involving just the apical part and not the whole shoot. Such a property can be verified by looking at the internodes extension: if the length of the internodes at the base is not changing, we can use the sample; otherwise, we discard it. We estimate the extension rate in cm/day by looking at the difference in the total length of the sample at different time intervals, dividing by the time intervals themselves, and taking the average. The analytical solution of System (2) (see Section C in S1 Text) leads to a relation between this estimate and the extension parameters, which consists of the following equation: Regarding ℓg, for each shoot, we sum the lengths in cm of the extending internodes and average them. Since many random effects may have influenced the shoot extension over these periods (e.g. local light, temperature, herbivores, support finding, etc.), we decided to compute a single value of and ℓg for all the shoots of the same species. To do so, we took the average also over all the estimated and ℓg. To retrieve the parameter G0, we divide the estimate for the extension rate by the estimate for the extension zone length ℓg. Estimate of the sensing parameters from reach and orientation. To estimate the sensing parameters (α, β, γ) for a fixed intensity δ of the radial expansion term (see Eq (4)), we employ the data about reach and orientation and the numerical solution of the model. More precisely, let rf(α, β, γ) the apical point r(ℓ(T), T) of the current configuration resulting from the numerical solution of system 5 at the final time T with sensing parameters (α, β, γ). Hence, the reach and orientation of the simulated shoot are where rf,1 and rf,2 are the components of rf respectively along e1 and e2. Let Reache and Oriente be the experimental reach and orientation for the sample considered by the simulation; we consider the parameter-dependent difference between simulated and experimental values with the function Hence, to estimate the sensing parameters, we fix an initial guess (α0, β0, γ0) and we minimize the value of F in a neighbourhood of that guess (see Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Graph of the function (α, β) → F(α, β, γ) for a sample of T. jasminoides. In this fgigure, γ is fixed at 0.008. The dot represents the minimum of the surface (approximately at α = 0.0005, β = −0.005). To estimate parameters (α, β, γ) we look for the minima of such surfaces for different values of gamma. Hence, we choose the minimum among those minima. https://doi.org/10.1371/journal.pcbi.1011538.g002 Formulation of the model: Mechanics and growth Notations for modelling a planar morphoelastic rod. We model the searcher shoot using the theory of morphoelastic rods [32]. Hence, we are assuming that climbing plants’ searcher shoot behaves like an elastic rod that at each time t changes its length and shape. In this framework, we make the following assumptions: (i) at each time t ∈ [0, T], where t = 0 and t = T are respectively the initial and the final time of the shoot’s development, the elastic rod’s centreline is confined in a plane spanned by two orthonormal vectors {e1, e2}; (ii) the rod is inextensible and unshearable. With these premises, the rod’s model is characterised by three elements (which have been partially introduced in Section: The intrinsic configuration at the initial time t = 0 (also called reference configuration). We assume that this configuration is just a straight line along e1 with arc-length parameter S ∈ [0, ℓ0]; The current configuration at a generic time t. We denote the corresponding centreline curve with r(s, t) ∈ span{e1, e2}, where s ∈ [0, ℓ(t)] is the arc-length parameter. It is convenient to describe the position of this curve using the angle θ(s, t) between the tangent vector ∂sr(s, t) and the vector e2 (see Fig 1). The symbol ∂s denotes the partial derivative with respect to the parameter s (a similar notation is used for ∂t and ∂S). Hence, the quantity ∂sθ(s, t) gives the curvature of this configuration; The intrinsic configuration at a generic time t. We only need to fix two notations about this configuration: (i) Since the rod is inextensible, this configuration has the same arc-length parameter s ∈ [0, ℓ(t)] of the current configuration; (ii) we indicate with κi(s, t) the curvature. Equations of balance with the gravity force. We assume that the only force acting on the rod is the gravity. Using the Kichhoff equations for the elastic rod in the static case (see [32] or Section A in S1 Text for further details), we get (1) This relation expresses the balance of gravity with the internal forces and moments (of force) developed by the rod in response. In the notation of Eq (1), g is the gravity acceleration constant, B(s, t) the flexural rigidity of the rod at r(s, t), ml(s, t) the mass of all the leaves between the point r(s, t) and the tip r(ℓ(t), t), and ρ(σ, t) the linear density at the point r(σ, t). The term linear density indicates the mass per unit of length. This quantity is different from the mass per unit of volume, which we refer to as volume density and denote with ρ3. If we assume that the rod has a circular cross-section with radius R(s, t) at the point r(s, t), the relation between these two quantities is simply given by Modeling the shoot extension. We consider primary growth as the result of a “growth from the tip”, so we are assuming that the addition of mass occurs only at the tip. As a result of such a process, a point initially located at r(S, 0) for some S ∈ [0, ℓ0], reaches the point r(s, t). The arc-length parameter s ∈ [0, ℓ(t)] at time t is a function of the initial arc-length parametrization S ∈ [0, ℓ0]. Assuming that the local extension occurs with a factor G0, the function s(S, t) has to satisfy the following partial differential equation: (2) where ℓg represents the length of the zone at the tip at which the extension occurs. Modeling the shoot rigidification. For modelling growing trees, the fundamental assumption is that the new layer of material formed during the secondary growth process do not affect the balance of the total forces and moments which were previously applied to the stem [33]. In other words, in an infinitesimal interval of time, the secondary growth does not affect the current configuration, which is described by the curvature ∂sθ, but affects the intrinsic configuration and its curvature κ. By applying the equations developed in [21] to our case (see Section B in S1 Text for further details, but also [20, 22, 33] for a biological and mechanical basis on the secondary growth), we get the following relation for the time evolution of the intrinsic curvature κ: (3) Equation for the tropisms: Directed stimulus, proprioception and secondary growth. External and internal stimuli affect the time evolution of the intrinsic configuration. To model this behaviour taking the rigidification into account, we need to make an assumption on the shape of the cross-section. As anticipated in Section, in our model the cross-section is a circle of radius R(s, t) at the point r(s, t). We can then complete Eq (3) with the terms of perception of a generically directed stimulus and proprioception [16, 21]: (4) where α and β represents the response to a directional stimulus, while γ represents the sensitivity with respect to the proprioception. We will refer to them as sensing parameters. δ is the parameter regulating the intensity of the radial expansion effect on the shoot development. vR represents the radial expansion rate. The notation α cos θ − β sin θ is equivalent to . Indeed, we can write the couple in polar coordinates by defining Hence, , and . Summary of the equations. We group the equations introduced in Sections—in the following system: (5) Regarding the boundary conditions, we are assuming that the searcher shoot is clamped at the base at an angle θ0. As stated in Section, at the initial time the intrinsic configuration consists in a straight rod, which means κ(⋅, 0) ≡ 0. The condition κ(ℓ(t), t) = ∂sθ(ℓ(t), t) means that there is no extra weight at the tip, hence the curvature is the same in both the intrinsic and the current configuration. As discussed in Sections C and D in S1 Text, to solve numerically system (5), we first rewrite all the equations with respect to the Lagrangian coordinates (S, t). Then, the system is numerically solved by discretizing it in time via the backward Euler method and integrating it in space using the finite element method. Notations for modelling a planar morphoelastic rod. We model the searcher shoot using the theory of morphoelastic rods [32]. Hence, we are assuming that climbing plants’ searcher shoot behaves like an elastic rod that at each time t changes its length and shape. In this framework, we make the following assumptions: (i) at each time t ∈ [0, T], where t = 0 and t = T are respectively the initial and the final time of the shoot’s development, the elastic rod’s centreline is confined in a plane spanned by two orthonormal vectors {e1, e2}; (ii) the rod is inextensible and unshearable. With these premises, the rod’s model is characterised by three elements (which have been partially introduced in Section: The intrinsic configuration at the initial time t = 0 (also called reference configuration). We assume that this configuration is just a straight line along e1 with arc-length parameter S ∈ [0, ℓ0]; The current configuration at a generic time t. We denote the corresponding centreline curve with r(s, t) ∈ span{e1, e2}, where s ∈ [0, ℓ(t)] is the arc-length parameter. It is convenient to describe the position of this curve using the angle θ(s, t) between the tangent vector ∂sr(s, t) and the vector e2 (see Fig 1). The symbol ∂s denotes the partial derivative with respect to the parameter s (a similar notation is used for ∂t and ∂S). Hence, the quantity ∂sθ(s, t) gives the curvature of this configuration; The intrinsic configuration at a generic time t. We only need to fix two notations about this configuration: (i) Since the rod is inextensible, this configuration has the same arc-length parameter s ∈ [0, ℓ(t)] of the current configuration; (ii) we indicate with κi(s, t) the curvature. Equations of balance with the gravity force. We assume that the only force acting on the rod is the gravity. Using the Kichhoff equations for the elastic rod in the static case (see [32] or Section A in S1 Text for further details), we get (1) This relation expresses the balance of gravity with the internal forces and moments (of force) developed by the rod in response. In the notation of Eq (1), g is the gravity acceleration constant, B(s, t) the flexural rigidity of the rod at r(s, t), ml(s, t) the mass of all the leaves between the point r(s, t) and the tip r(ℓ(t), t), and ρ(σ, t) the linear density at the point r(σ, t). The term linear density indicates the mass per unit of length. This quantity is different from the mass per unit of volume, which we refer to as volume density and denote with ρ3. If we assume that the rod has a circular cross-section with radius R(s, t) at the point r(s, t), the relation between these two quantities is simply given by Modeling the shoot extension. We consider primary growth as the result of a “growth from the tip”, so we are assuming that the addition of mass occurs only at the tip. As a result of such a process, a point initially located at r(S, 0) for some S ∈ [0, ℓ0], reaches the point r(s, t). The arc-length parameter s ∈ [0, ℓ(t)] at time t is a function of the initial arc-length parametrization S ∈ [0, ℓ0]. Assuming that the local extension occurs with a factor G0, the function s(S, t) has to satisfy the following partial differential equation: (2) where ℓg represents the length of the zone at the tip at which the extension occurs. Modeling the shoot rigidification. For modelling growing trees, the fundamental assumption is that the new layer of material formed during the secondary growth process do not affect the balance of the total forces and moments which were previously applied to the stem [33]. In other words, in an infinitesimal interval of time, the secondary growth does not affect the current configuration, which is described by the curvature ∂sθ, but affects the intrinsic configuration and its curvature κ. By applying the equations developed in [21] to our case (see Section B in S1 Text for further details, but also [20, 22, 33] for a biological and mechanical basis on the secondary growth), we get the following relation for the time evolution of the intrinsic curvature κ: (3) Equation for the tropisms: Directed stimulus, proprioception and secondary growth. External and internal stimuli affect the time evolution of the intrinsic configuration. To model this behaviour taking the rigidification into account, we need to make an assumption on the shape of the cross-section. As anticipated in Section, in our model the cross-section is a circle of radius R(s, t) at the point r(s, t). We can then complete Eq (3) with the terms of perception of a generically directed stimulus and proprioception [16, 21]: (4) where α and β represents the response to a directional stimulus, while γ represents the sensitivity with respect to the proprioception. We will refer to them as sensing parameters. δ is the parameter regulating the intensity of the radial expansion effect on the shoot development. vR represents the radial expansion rate. The notation α cos θ − β sin θ is equivalent to . Indeed, we can write the couple in polar coordinates by defining Hence, , and . Summary of the equations. We group the equations introduced in Sections—in the following system: (5) Regarding the boundary conditions, we are assuming that the searcher shoot is clamped at the base at an angle θ0. As stated in Section, at the initial time the intrinsic configuration consists in a straight rod, which means κ(⋅, 0) ≡ 0. The condition κ(ℓ(t), t) = ∂sθ(ℓ(t), t) means that there is no extra weight at the tip, hence the curvature is the same in both the intrinsic and the current configuration. As discussed in Sections C and D in S1 Text, to solve numerically system (5), we first rewrite all the equations with respect to the Lagrangian coordinates (S, t). Then, the system is numerically solved by discretizing it in time via the backward Euler method and integrating it in space using the finite element method. Application to experimental data Geometry and biomechanical properties of the samples. In a first experiment, we collected morphological and biomechanical data from five searcher shoots for each species. For each individual, we selected, as far as possible, the longest searcher shoots in a self-supporting state. In their natural position, we measured the reach (cm) and orientation (degrees from the horizontal) defined as a straight line from the base to the apex of the searcher shoot. The geometry of the shoot was described from the internodes of the main stem. For each successive internode, we measured its length and its median diameter obtained from the mean of two orthogonal measurements. Bending properties of the base of the searcher stem were measured using four-point bending tests on a stem segment constituted of several internodes [34]. Flexural rigidity EI (which is denoted with the function B in System (5)) and structural Young’s modulus Estr were calculated from applied bending forces plotted against maximum deflections. Up to five weights were applied manually and each deflection was measured by observation with a dissecting microscope on the apparatus. Weight increments were chosen according to the bending resistance of each sample. Weights were constituted of stainless-steel or brass ranging from 8 g (C. guianense) to 50 g (T. jasminoides). Span distances were defined as proportional to the mean elliptical diameter of the stem segment. The span support was 40 times greater than the diameter and was ranging from 107 mm to 218 mm for the longest. The load span was comprised between one-half and two-thirds of the span support and ranged from 67 mm to 120 mm. For three positions along the measured segment (basal, medial, apical), the vertical and horizontal diameter, dv and dh, were measured and the means were then used to calculate the second moment of area I of the axis. The experimental measurements show that the difference between dv and dh is small, so it is reasonable to assume a circular cross-section. The structural Young’s modulus of the stem (Estr) (MN/m2) was estimated by dividing measured values of EI by calculated values of I [35]. Extensional growth of the samples and radial expansion rate. In a second experiment, we monitored the shoot extension of young, self-supporting, searcher shoots over one week for 11 shoots of T. jasminoides and one month for 19 shoots of C. guianense (see section Materials and Methods in [29] for further information on the environmental conditions). Three dates at three days intervals were recorded for T. jasminoides and five dates at 7 days of interval for C. guianense. For both species, we applied the same protocol starting by defining a reference mark on the most basal node of each main stem. From this mark, we measured at each date: (i) the total length of the shoot (cm); (ii) the length of each successive internode up to and including the apex (cm). The natural organisation of the shoot into separate internodes allowed us to identify and measure which internodes elongated between two dates (see S2 Table). The data on the radius from the first experiment (discussed in Section) and the measurements of the extension from the second experiment allow us to compute the radial expansion rate vR. To accomplish this measurement, we consider the geometrical structure of the samples from the first experiment, i.e. the internodes. For each sample, we enumerate the internodes starting from the base. At the base of the i-th internode, we assign its arc-length parameter, which we denote with si. Hence, according to our notation, s1 is the arc length at the base of the first internode, so s1 = 0, and if j > i, then sj > si. Moreover, since sj > si, the point r(si, t) is closer to the base than r(sj, t), hence more mature. Consider the situation in which j = i + 1. Let Δti be the time it takes the plant to elongate by an amount equal to the length of the i-th internode, that is Assuming that the maturation process depends merely on the distance from the tip (see Section below), this implies that the radius R(si+i, t) is going to expand until reaching the value R(si+1, t + Δti) = R(si, t) after its distance from the tip has increased by the length of the i-th internode. Hence, to retrieve the expansion rate at the base of the i-th internode, we take the difference between the radii at its extremes and divide it by Δti: We retrieve the extension rate from the data of the second experiment, as explained in Section below. Fitting of the model’s biomechanical functions. The data provided by the first experiment were to estimate volume density ρ3, radius R, radial expansion speed vR, flexural rigidity B and leaves mass ml. Such an estimate was specific for each shoot, that is, for each shoot, we retrieved the parameters of the above-mentioned functions. The data from the first experiment describe the biomechanical properties of the samples at a fixed time of their development. To get information about the time evolution of those properties, we assume that they depend only on the distance from the tip. Hence, we have ρ3(s, t) = ρ3(ℓ(t) − s), R(s, t) = R(ℓ(t) − s) and the same for vR, B and ml (more precisely, the mass of a single leaf as explained below). In particular, we assume that where x = ℓ(t) − s is the distance from the tip and is the mass of a single leaf. The coefficients (a3, b3, c3), …, (al, bl, cl) are retrieved by fitting the functions with the experimental data (see Figures A-B in S1 Text). We choose a sigmoid for the flexural rigidity B and the mass of a single leaf because we assume that the greater the distance from the tip, the more their value increases. To retrieve the mass of the leaves ml from the mass of a single leaf at a certain point of the shoot r(s, t), it is sufficient to sum the single leaf masses at the internode basis whose arc-length is greater than s. To transpose this concept into an equation, we denote with I(t) the number of internodes in the shoot at time t, starting the enumeration from the base. For instance, if there are four internodes, I(t) = {1, 2, 3, 4}, where 1 represents the internode at the base and 4 is the internode at the tip. We denote with si(t) the arc length of the basis i-th internode (as for the radial expansion rate case). Then, we write the equation for ml as follows: where is the number of leaves at the base of the i-th internode at time t. The numerical scheme used to simulate System (5) results to be stable with respect to this fitting procedure. Indeed, by changing the function for the flexural rigidity B from a sigmoid to a polynomial, the resulting simulation of the main stem does not display any significant change. See Section G in S1 Text for further details. Estimate of the extension zone parameter ℓg and the local extension factor G0. We model the primary growth process as a growth from the tip, with an extension zone of length ℓg located at the apex and a local extension factor G0. We consider only the case in which the length of the shoot is greater than ℓg and consequently, the length of the shoot ℓ(t) follows a linear law (i.e. a constant extension rate , see for instance [32], Section 4.3.1). For this reason, we consider only the samples in which the extension process is involving just the apical part and not the whole shoot. Such a property can be verified by looking at the internodes extension: if the length of the internodes at the base is not changing, we can use the sample; otherwise, we discard it. We estimate the extension rate in cm/day by looking at the difference in the total length of the sample at different time intervals, dividing by the time intervals themselves, and taking the average. The analytical solution of System (2) (see Section C in S1 Text) leads to a relation between this estimate and the extension parameters, which consists of the following equation: Regarding ℓg, for each shoot, we sum the lengths in cm of the extending internodes and average them. Since many random effects may have influenced the shoot extension over these periods (e.g. local light, temperature, herbivores, support finding, etc.), we decided to compute a single value of and ℓg for all the shoots of the same species. To do so, we took the average also over all the estimated and ℓg. To retrieve the parameter G0, we divide the estimate for the extension rate by the estimate for the extension zone length ℓg. Estimate of the sensing parameters from reach and orientation. To estimate the sensing parameters (α, β, γ) for a fixed intensity δ of the radial expansion term (see Eq (4)), we employ the data about reach and orientation and the numerical solution of the model. More precisely, let rf(α, β, γ) the apical point r(ℓ(T), T) of the current configuration resulting from the numerical solution of system 5 at the final time T with sensing parameters (α, β, γ). Hence, the reach and orientation of the simulated shoot are where rf,1 and rf,2 are the components of rf respectively along e1 and e2. Let Reache and Oriente be the experimental reach and orientation for the sample considered by the simulation; we consider the parameter-dependent difference between simulated and experimental values with the function Hence, to estimate the sensing parameters, we fix an initial guess (α0, β0, γ0) and we minimize the value of F in a neighbourhood of that guess (see Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Graph of the function (α, β) → F(α, β, γ) for a sample of T. jasminoides. In this fgigure, γ is fixed at 0.008. The dot represents the minimum of the surface (approximately at α = 0.0005, β = −0.005). To estimate parameters (α, β, γ) we look for the minima of such surfaces for different values of gamma. Hence, we choose the minimum among those minima. https://doi.org/10.1371/journal.pcbi.1011538.g002 Geometry and biomechanical properties of the samples. In a first experiment, we collected morphological and biomechanical data from five searcher shoots for each species. For each individual, we selected, as far as possible, the longest searcher shoots in a self-supporting state. In their natural position, we measured the reach (cm) and orientation (degrees from the horizontal) defined as a straight line from the base to the apex of the searcher shoot. The geometry of the shoot was described from the internodes of the main stem. For each successive internode, we measured its length and its median diameter obtained from the mean of two orthogonal measurements. Bending properties of the base of the searcher stem were measured using four-point bending tests on a stem segment constituted of several internodes [34]. Flexural rigidity EI (which is denoted with the function B in System (5)) and structural Young’s modulus Estr were calculated from applied bending forces plotted against maximum deflections. Up to five weights were applied manually and each deflection was measured by observation with a dissecting microscope on the apparatus. Weight increments were chosen according to the bending resistance of each sample. Weights were constituted of stainless-steel or brass ranging from 8 g (C. guianense) to 50 g (T. jasminoides). Span distances were defined as proportional to the mean elliptical diameter of the stem segment. The span support was 40 times greater than the diameter and was ranging from 107 mm to 218 mm for the longest. The load span was comprised between one-half and two-thirds of the span support and ranged from 67 mm to 120 mm. For three positions along the measured segment (basal, medial, apical), the vertical and horizontal diameter, dv and dh, were measured and the means were then used to calculate the second moment of area I of the axis. The experimental measurements show that the difference between dv and dh is small, so it is reasonable to assume a circular cross-section. The structural Young’s modulus of the stem (Estr) (MN/m2) was estimated by dividing measured values of EI by calculated values of I [35]. Extensional growth of the samples and radial expansion rate. In a second experiment, we monitored the shoot extension of young, self-supporting, searcher shoots over one week for 11 shoots of T. jasminoides and one month for 19 shoots of C. guianense (see section Materials and Methods in [29] for further information on the environmental conditions). Three dates at three days intervals were recorded for T. jasminoides and five dates at 7 days of interval for C. guianense. For both species, we applied the same protocol starting by defining a reference mark on the most basal node of each main stem. From this mark, we measured at each date: (i) the total length of the shoot (cm); (ii) the length of each successive internode up to and including the apex (cm). The natural organisation of the shoot into separate internodes allowed us to identify and measure which internodes elongated between two dates (see S2 Table). The data on the radius from the first experiment (discussed in Section) and the measurements of the extension from the second experiment allow us to compute the radial expansion rate vR. To accomplish this measurement, we consider the geometrical structure of the samples from the first experiment, i.e. the internodes. For each sample, we enumerate the internodes starting from the base. At the base of the i-th internode, we assign its arc-length parameter, which we denote with si. Hence, according to our notation, s1 is the arc length at the base of the first internode, so s1 = 0, and if j > i, then sj > si. Moreover, since sj > si, the point r(si, t) is closer to the base than r(sj, t), hence more mature. Consider the situation in which j = i + 1. Let Δti be the time it takes the plant to elongate by an amount equal to the length of the i-th internode, that is Assuming that the maturation process depends merely on the distance from the tip (see Section below), this implies that the radius R(si+i, t) is going to expand until reaching the value R(si+1, t + Δti) = R(si, t) after its distance from the tip has increased by the length of the i-th internode. Hence, to retrieve the expansion rate at the base of the i-th internode, we take the difference between the radii at its extremes and divide it by Δti: We retrieve the extension rate from the data of the second experiment, as explained in Section below. Fitting of the model’s biomechanical functions. The data provided by the first experiment were to estimate volume density ρ3, radius R, radial expansion speed vR, flexural rigidity B and leaves mass ml. Such an estimate was specific for each shoot, that is, for each shoot, we retrieved the parameters of the above-mentioned functions. The data from the first experiment describe the biomechanical properties of the samples at a fixed time of their development. To get information about the time evolution of those properties, we assume that they depend only on the distance from the tip. Hence, we have ρ3(s, t) = ρ3(ℓ(t) − s), R(s, t) = R(ℓ(t) − s) and the same for vR, B and ml (more precisely, the mass of a single leaf as explained below). In particular, we assume that where x = ℓ(t) − s is the distance from the tip and is the mass of a single leaf. The coefficients (a3, b3, c3), …, (al, bl, cl) are retrieved by fitting the functions with the experimental data (see Figures A-B in S1 Text). We choose a sigmoid for the flexural rigidity B and the mass of a single leaf because we assume that the greater the distance from the tip, the more their value increases. To retrieve the mass of the leaves ml from the mass of a single leaf at a certain point of the shoot r(s, t), it is sufficient to sum the single leaf masses at the internode basis whose arc-length is greater than s. To transpose this concept into an equation, we denote with I(t) the number of internodes in the shoot at time t, starting the enumeration from the base. For instance, if there are four internodes, I(t) = {1, 2, 3, 4}, where 1 represents the internode at the base and 4 is the internode at the tip. We denote with si(t) the arc length of the basis i-th internode (as for the radial expansion rate case). Then, we write the equation for ml as follows: where is the number of leaves at the base of the i-th internode at time t. The numerical scheme used to simulate System (5) results to be stable with respect to this fitting procedure. Indeed, by changing the function for the flexural rigidity B from a sigmoid to a polynomial, the resulting simulation of the main stem does not display any significant change. See Section G in S1 Text for further details. Estimate of the extension zone parameter ℓg and the local extension factor G0. We model the primary growth process as a growth from the tip, with an extension zone of length ℓg located at the apex and a local extension factor G0. We consider only the case in which the length of the shoot is greater than ℓg and consequently, the length of the shoot ℓ(t) follows a linear law (i.e. a constant extension rate , see for instance [32], Section 4.3.1). For this reason, we consider only the samples in which the extension process is involving just the apical part and not the whole shoot. Such a property can be verified by looking at the internodes extension: if the length of the internodes at the base is not changing, we can use the sample; otherwise, we discard it. We estimate the extension rate in cm/day by looking at the difference in the total length of the sample at different time intervals, dividing by the time intervals themselves, and taking the average. The analytical solution of System (2) (see Section C in S1 Text) leads to a relation between this estimate and the extension parameters, which consists of the following equation: Regarding ℓg, for each shoot, we sum the lengths in cm of the extending internodes and average them. Since many random effects may have influenced the shoot extension over these periods (e.g. local light, temperature, herbivores, support finding, etc.), we decided to compute a single value of and ℓg for all the shoots of the same species. To do so, we took the average also over all the estimated and ℓg. To retrieve the parameter G0, we divide the estimate for the extension rate by the estimate for the extension zone length ℓg. Estimate of the sensing parameters from reach and orientation. To estimate the sensing parameters (α, β, γ) for a fixed intensity δ of the radial expansion term (see Eq (4)), we employ the data about reach and orientation and the numerical solution of the model. More precisely, let rf(α, β, γ) the apical point r(ℓ(T), T) of the current configuration resulting from the numerical solution of system 5 at the final time T with sensing parameters (α, β, γ). Hence, the reach and orientation of the simulated shoot are where rf,1 and rf,2 are the components of rf respectively along e1 and e2. Let Reache and Oriente be the experimental reach and orientation for the sample considered by the simulation; we consider the parameter-dependent difference between simulated and experimental values with the function Hence, to estimate the sensing parameters, we fix an initial guess (α0, β0, γ0) and we minimize the value of F in a neighbourhood of that guess (see Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Graph of the function (α, β) → F(α, β, γ) for a sample of T. jasminoides. In this fgigure, γ is fixed at 0.008. The dot represents the minimum of the surface (approximately at α = 0.0005, β = −0.005). To estimate parameters (α, β, γ) we look for the minima of such surfaces for different values of gamma. Hence, we choose the minimum among those minima. https://doi.org/10.1371/journal.pcbi.1011538.g002 Results We integrate system (5) numerically, calibrating the parameters for two specific climbing plant samples, one for T. jasminoides species, the other for C. guianense species (see Tables 1 and 2). For such simulations, the calibration of the sensing parameters (α, β, γ) is obtained by setting the radial extension parameter to δ = 0. The simulated stems are displayed in Fig 3. For T. jasminoides, the initial inclination of the stem with respect to the vertical line is π/4 radians. Initially, the stem grows upwards, but it is clear from the changes in the inclination of the tip that the growth direction is changing. Soon, the apical part of the shoot starts growing downwards. The case of C. guianense is different. The simulation displays a strong horizontal extension of the shoot. The initial inclination of the main stem is π/2 radians with respect to the vertical line, which means that, in the beginning, the shoot is directed horizontally. Looking at the time evolution displayed in Fig 3, we see that the tip is directed upward. Nevertheless, it is clear that in the simulation the whole C. guianense stem is drooping due to its weight. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. (3A),(3B): Simulations for T. jasminoides and C. guianense, respectively. The axes represent the (x, y)-coordinates of the simulation in the plane span{e1, e2}, where x is the coordinate along e1 and y the coordinate along e2. The different colours represent the different time steps of the simulation; each line of a single colour represents the main stem. Each stem has a fixed origin point at (0,0). The simulation of T. jasminoides has an initial inclination of π/4 with respect to the vertical line, while that of C. guianense has a horizontal initial inclination. In both cases, the plant develops horizontally. The behaviour of the T. jasminoides is due to its sensing activity, which displays a negative parameter β. In contrast, that of C. guianense droops because of its weight. (3C),(3D): Images of the two samples of T. jasminoides and C. guianense, respectively, in their natural habitat. These are the samples on which the numerical simulations are based. As one can observe, their behaviour resembles those displayed in Fig 3A and 3B: the T. jasminoides curls around towards itself, growing downwards; the C. guianense is strongly directed horizontally and points upwards with its tip. https://doi.org/10.1371/journal.pcbi.1011538.g003 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Estimated parameters for a sample of T. jasminoides. https://doi.org/10.1371/journal.pcbi.1011538.t001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Estimated parameters for a sample of C. guianense. https://doi.org/10.1371/journal.pcbi.1011538.t002 To further investigate the role of the weight distributiuon along the stem, we numerically integrate a “weightless model” based just on the growing Eq (2) and on the following evolution equation of the curvature (6) We can extrapolate G0, ℓg, R and vR from the experimental data, calibrate the sensing parameters (α, β, γ) from the reach and the orientation of the shoot in consideration and set all these values as parameters in Eqs (2) and (6). The resulting simulations are displayed in Fig 4. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Simulations for T. jasminoides and C. guianense respectively. These simulations are based just on the growth Eqs (2) and (6), that is, they neglect the approximation of stem mechanics. The sensing parameters (α, β, γ) are optimised to obtain the best reach and orientation according to the experimental data. Such values are (6.8e − 03, −3e − 03, 2e − 02), (2.5e − 03, 2e − 03, 2e − 02), (in the ususal units of measure) respectively for the T. jasminoides and the C. guianense. We can observe that the simulation 4A for the T. jasminoides is growing downwards. The displayed behaviour is due to the sensing activity, in accordance with the result obtained for the simulation of the T. jasminoides in Fig 3A. On the other hand, the C. guianense is just growing upwards, differently from Fig 3B. This means that the mass can play a fundamental role in the formation of the plant shape. https://doi.org/10.1371/journal.pcbi.1011538.g004 Considering again the model described by System (5), we fixed different values of δ and calibrated again the sensing parameters (α, β, γ) as described in Section. The parameters G0, ℓg, R and vR are based on the C. guianense dataset. The resulting simulations for different values of δ are displayed in Fig 5, in which we observe an increasing curling behaviour of the apex as δ increases. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Simulations of C. guianense for increasing values of δ. Each of these simulations are obtained by following the procedure described in Section, so the sensing parameters are optimized to meet the experimental reach and orientation and may change among the simulations. We can observe the effect of the “negative proprioceptive” term (∂tB/B)∂sθ which amplifies the curling behaviour of the tip. This curling effect resembles the gravitropic “sign reversal” displayed in [24]. https://doi.org/10.1371/journal.pcbi.1011538.g005 Discussion Interpretation and limitaitons of the result The model developed in this study combines the primary growth of a searcher shoot (i.e. tip growth inducing stem extension) with the postural responses incorporating lateral expansion of the stem (i.e. secondary growth of the wood cylinder, thickening and maturation of primary tissues). These processes have been complemented by the variation of stem flexural rigidity EI (expressed by the function B in Eq (1)) and through the time development of the intrinsic curvature κ. As displayed in Section, the flexural rigidity was calibrated from the values of the (structural) Young’s modulus E observed at the base of the shoot, and from the series of diameters measured along the shoot to estimate the second moment of area I. In this procedure, E is considered to be constant in time and space. This simplification does not take into account the variations in E that exist due to the non-homogeneous tapering of tissues in the stem and their maturation. Although the variations along the climbing plant shoot of Young’s Modulus and second moment of area can well be of the same order, the variation of R plays a major role in the displayed simulations since it appears at the denominator of the sensing parameters in Eq (4). In the simulations displayed in Fig 3, we observed that the stem is sagging. However, looking at the sensing parameters obtained in the fitting procedure (see Tables 1 and 2), we were able to distinguish whether the behaviour is due to the sensing activity of the shoot or to its weight. The negative β for the T. jasminoides expresses a downward stimulus which causes downward growth. So, we could conclude that the main driver of the T. jasminoides behaviour is the sensing activity. On the other hand, applying the same reasoning to the C. guianense stem, we reached the opposite conclusion. The positive β denotes a preference for upward growth, which explains the behaviour of the tip. However, as a consequence of its own weight, the main stem has a mostly horizontal development with a steep vertical change of growth in the part of the stem close to the tip. Another evidence of the distinct role of the weight distribution along the stem in T. jasminoides and C. guianense behaviours are provided by the outcome of the simulations of the weightless model (displayed in Fig 4). A comparison between the two sets of simulations confirms the results on the role of the weight distribution along the shoots detailed previously. Indeed, as reported in Fig 4, the sensing parameter β for the weightless T. jasminoides is negative and the sensing activity orients the tip shoot downwards in a way that is similar (but not equal) to the case in which the weight affects the plant shoot development. On the other hand, the weightless model of C. guianense immediately grows upwards, without displaying a horizontal structure in the shoot portion before the apex. The role of the radial expansion parameter δ In the study by Guillon et al. [21], it is assumed that the stem radial accretion does not change the local current curvature (see, e.g., the simulations in Fig 5 and related caption). Such an assumption led Guillon et al. [21] to add the term (7) in the sensing equation. The tendency to maintain the local current curvature rather than straighten the stem is in contrast with the stem proprioceptive activity. From a mathematical point of view, we can observe that the term appearing in Eq (4) has an opposite sign with respect to the proprioceptive term, acting like a “negative proprioception” term in the sensing equation. Indeed, the ratio is non-negative since the flexural rigidity B is both positive and increasing in time. Hence, if we consider the Eq (4) with δ > 0, it follows that the term has an opposite sign with respect to the proprioceptive term −Gγ∂θ. In particular, this means that the term can mitigate (or destroy) the straightening effect of the proprioceptive term, introduced in [16] exactly to stabilize the sensing equation. The occurrence of a “negative proprioception” is shown in the simulations in Fig 5, in which we can observe that the curling behaviour of the tip increases as the parameter δ increases. In particular, this behaviour affects the elongation direction of the tip, which grows downwards when it starts curling and then grows upwards again, resembling the gravitropic “sign reversal” displayed in [24]. The destabilising effect of the negative proprioception is confirmed by the spectral analysis of the operator involved in the simple equation developed by Bastien et al. [16]: (8) In this simple case, we observe that as the parameter varies, the stem dampens its oscillations around the vertical or not (see Fig 6). Linearizing the system around the equilibrium θ ≡ 0, we obtain the linear system (9) With the help of a numerical simulation, we can compute the eigenvalues of the linear operator . The computed eigenvalues are displayed in Fig 6. We can observe that for γ ≥ 0 the real eigenvalues are negative, while when γ < 0 the real eigenvalues are positive. This means that in the negative proprioception case, there is an instability in the vertical equilibrium. This consideration is coherent with the behaviour of the steady state for system (9). Indeed, the steady state is given by Assuming β > 0, for γ > 0 the curve θ(s) tends to zero, which means that the correspondent stem tends to the vertical line. On the other hand, for γ < 0 θ(s) increases its values and consequently the corresponding stem tends to curl around itself. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Simulations of the system (8) with β = 10 and (S, T) ∈ [0, 1] × [0, 2]. In Fig 6D, the eigenvalues for the linear operator in system (9). In green, the eigenvalues for γ > 0, in red for γ < 0 and in blue for γ = 0. https://doi.org/10.1371/journal.pcbi.1011538.g006 Despite the destabilizing effect, it seems that a parameter δ ≃ 0.25 is more suited to describe the shape of the real C. guianense sample (see Fig 3). Indeed, as displayed in Fig 5, the tip tends to curl slightly in a similar way to Fig 3. This shows that our model provides a way to take secondary growth into account quantitatively with the parameter δ. However, given the dataset at our disposal, we cannot give an estimate of δ. Moreover, also due to its contrast with the proprioception, the estimate of δ represents a technical challenge and an interesting direction for future research. Biological interpretations The application of the 2D model with δ ≈ 0 and calibrated with field data showed that a full description of primary and secondary growth processes as well as a description of the sensing activity was fundamental to reproducing the self-supporting growth of searcher shoots. Numerical simulations were able to reproduce similar maximal reach capacities as those observed in the field for both the samples in consideration. Simulations showed that both exhibited horizontal trajectories of self-supporting growth to attain their maximal reach capacities. 2D postures obtained at the end of the simulations were qualitatively equivalent to the ones observed in the field. Both species expressed a self-weight movement induced by mass distribution (in space) and accretion (in time). Nevertheless, they differed in terms of postural dynamics. These differences can be interpreted numerically by both sensing and mechanical parameters. From a sensing point of view, C. guianense exhibited a stronger anti-gravitropic behaviour than T. jasminoides. These explain the apical orientation of searcher shoots leading to an upward growth for C. guianense and a downward growth for T. jasminoides. From a mechanical point of view, C. guianense was more strongly effected than T. jasminoides in maintaining an upward growth in view of a faster mass accretion along the searcher shoot. However, these biological interpretations must be interpreted with caution with respect to the size of the dataset and observations that have been made under natural conditions without control over external factors. The samples considered in this work are not representative of all the variations we expect to see in nature. See for instance Section H in S1 Text for further photos of samples. Future improvements The model developed here provides a theoretical and analytical framework that can be a useful tool to better understand the strategies of lianas to cross gaps with their searcher shoots under the effects of gravitational constraints. In particular, it allows us to see how the mass distribution, the stiffness and the length of the shoot influence its development and space orientation. To this purpose, the model parameters are fitted with the experimental data. As displayed in Section, for the T. jasminoides case this fitting procedure returns a negative parameter β, which is related to the vertical direction of the stimuli response activity. Since the climbing plant samples were not raised in a controlled environment, we cannot immediately interpret the negative value of β and the related behaviour of the simulation (Fig 3) with positive gravitropic behaviour. Nevertheless, the relation between the values of the sensing parameters and the observed behaviour of the climbing plant under unpredictable environmental conditions can be a matter of further studies. Despite the uncontrolled environment and the basic measurements based on fully elongated shoots, the 2D model is revealed to be robust enough to reproduce the self-supporting state of the climbing plant species considered in this work. Moreover, the model can already take into account the tissue anatomical variations along the stems in terms of flexural rigidity, mass and radius, and how such variations affect the final climbing plant shape. Moving in the direction of a more in-depth study of the shoot structure, it may be interesting to investigate how the biomechanical state and biomass allocation change during the extension of a searcher shoot from a dynamic point of view. This would imply working on a large set of searcher shoots, regularly sacrificing some of them for destructive measurements, and reasoning in terms of chronosequence (i.e. by extrapolation), trying to work under conditions that are as homogeneous as possible in order to minimize the uncertainty factors (ontogenetic stage, environment, etc.). These measurements associated with the anatomical study of the tissues could allow a better understanding of the mechanism at the origin of the rigidification dynamics at all points of the searcher shoots. The anatomical organizations in the mechanical and postural life-histories of plants are potentially embracing ecology, biomechanics and robotics [36]. Searcher shoots are highly diverse in climbing plants and can have relatively complex structures depending on types of connected structures such as leaves, hooks, tendrils, branches, stem segments capable of circumnutatory movements, as well as intrinsic stem extension and postural dynamics [29]. The movements associated with these structures and generated by growth differentials within tissues are of significant importance for exploring and crossing spaces and attaching to supports. The model considered here in 2D is relatively simple and corresponds to an unbranched shoot with differential dynamics between internodes and leaves. The consideration of more complex shoot structures, involving variable mass load distributions in space and time and in a 3D volume is a higher step to properly simulate the self-supporting and searching mobility of searcher shoots in climbing plants. For instance, a 3D version of the model presented in this paper could be used to understand the role of circumnutation in climbing plant searching strategy. Another possible application of a 3D model could be to study how climbing plants change their mechanical internal properties when they encounter external obstacles to which they attach. All these problems cannot be addressed by using a 2D model and will be the subject of future studies. Conclusion We formulated a model that takes secondary growth and proprioception into account. We showed how it is possible to estimate the parameters of the model from experimental data. Using the information about the extension of the stem, we computed the extension parameter G0 and the length of the extension zone ℓg. Then, based on experimental data on the final reach and orientation of each stem, we calibrated the sensing parameters α, β and γ. Based on such measurements and on the related simulations in Fig 3, we were able to understand the role of the weight in the shapes observed in the photos in Fig 3. For the two samples under consideration, we were able to say that the mechanical effect of the weight plays an important role in the shape of the C. guianense, while the downward growth of the T. jasminoides is mainly the consequence of its sensing activity. These conclusions are supported by the simulations based on a simplified model which considers just the plant sensing activity (see Fig 4). Such results confirm that the mechanical aspects in climbing plant searcher shoots, and, in particular, the variable linear density and the secondary growth are not negligible in order to obtain realistic climbing plant shapes in experimental data-based simulations. Interpretation and limitaitons of the result The model developed in this study combines the primary growth of a searcher shoot (i.e. tip growth inducing stem extension) with the postural responses incorporating lateral expansion of the stem (i.e. secondary growth of the wood cylinder, thickening and maturation of primary tissues). These processes have been complemented by the variation of stem flexural rigidity EI (expressed by the function B in Eq (1)) and through the time development of the intrinsic curvature κ. As displayed in Section, the flexural rigidity was calibrated from the values of the (structural) Young’s modulus E observed at the base of the shoot, and from the series of diameters measured along the shoot to estimate the second moment of area I. In this procedure, E is considered to be constant in time and space. This simplification does not take into account the variations in E that exist due to the non-homogeneous tapering of tissues in the stem and their maturation. Although the variations along the climbing plant shoot of Young’s Modulus and second moment of area can well be of the same order, the variation of R plays a major role in the displayed simulations since it appears at the denominator of the sensing parameters in Eq (4). In the simulations displayed in Fig 3, we observed that the stem is sagging. However, looking at the sensing parameters obtained in the fitting procedure (see Tables 1 and 2), we were able to distinguish whether the behaviour is due to the sensing activity of the shoot or to its weight. The negative β for the T. jasminoides expresses a downward stimulus which causes downward growth. So, we could conclude that the main driver of the T. jasminoides behaviour is the sensing activity. On the other hand, applying the same reasoning to the C. guianense stem, we reached the opposite conclusion. The positive β denotes a preference for upward growth, which explains the behaviour of the tip. However, as a consequence of its own weight, the main stem has a mostly horizontal development with a steep vertical change of growth in the part of the stem close to the tip. Another evidence of the distinct role of the weight distribution along the stem in T. jasminoides and C. guianense behaviours are provided by the outcome of the simulations of the weightless model (displayed in Fig 4). A comparison between the two sets of simulations confirms the results on the role of the weight distribution along the shoots detailed previously. Indeed, as reported in Fig 4, the sensing parameter β for the weightless T. jasminoides is negative and the sensing activity orients the tip shoot downwards in a way that is similar (but not equal) to the case in which the weight affects the plant shoot development. On the other hand, the weightless model of C. guianense immediately grows upwards, without displaying a horizontal structure in the shoot portion before the apex. The role of the radial expansion parameter δ In the study by Guillon et al. [21], it is assumed that the stem radial accretion does not change the local current curvature (see, e.g., the simulations in Fig 5 and related caption). Such an assumption led Guillon et al. [21] to add the term (7) in the sensing equation. The tendency to maintain the local current curvature rather than straighten the stem is in contrast with the stem proprioceptive activity. From a mathematical point of view, we can observe that the term appearing in Eq (4) has an opposite sign with respect to the proprioceptive term, acting like a “negative proprioception” term in the sensing equation. Indeed, the ratio is non-negative since the flexural rigidity B is both positive and increasing in time. Hence, if we consider the Eq (4) with δ > 0, it follows that the term has an opposite sign with respect to the proprioceptive term −Gγ∂θ. In particular, this means that the term can mitigate (or destroy) the straightening effect of the proprioceptive term, introduced in [16] exactly to stabilize the sensing equation. The occurrence of a “negative proprioception” is shown in the simulations in Fig 5, in which we can observe that the curling behaviour of the tip increases as the parameter δ increases. In particular, this behaviour affects the elongation direction of the tip, which grows downwards when it starts curling and then grows upwards again, resembling the gravitropic “sign reversal” displayed in [24]. The destabilising effect of the negative proprioception is confirmed by the spectral analysis of the operator involved in the simple equation developed by Bastien et al. [16]: (8) In this simple case, we observe that as the parameter varies, the stem dampens its oscillations around the vertical or not (see Fig 6). Linearizing the system around the equilibrium θ ≡ 0, we obtain the linear system (9) With the help of a numerical simulation, we can compute the eigenvalues of the linear operator . The computed eigenvalues are displayed in Fig 6. We can observe that for γ ≥ 0 the real eigenvalues are negative, while when γ < 0 the real eigenvalues are positive. This means that in the negative proprioception case, there is an instability in the vertical equilibrium. This consideration is coherent with the behaviour of the steady state for system (9). Indeed, the steady state is given by Assuming β > 0, for γ > 0 the curve θ(s) tends to zero, which means that the correspondent stem tends to the vertical line. On the other hand, for γ < 0 θ(s) increases its values and consequently the corresponding stem tends to curl around itself. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Simulations of the system (8) with β = 10 and (S, T) ∈ [0, 1] × [0, 2]. In Fig 6D, the eigenvalues for the linear operator in system (9). In green, the eigenvalues for γ > 0, in red for γ < 0 and in blue for γ = 0. https://doi.org/10.1371/journal.pcbi.1011538.g006 Despite the destabilizing effect, it seems that a parameter δ ≃ 0.25 is more suited to describe the shape of the real C. guianense sample (see Fig 3). Indeed, as displayed in Fig 5, the tip tends to curl slightly in a similar way to Fig 3. This shows that our model provides a way to take secondary growth into account quantitatively with the parameter δ. However, given the dataset at our disposal, we cannot give an estimate of δ. Moreover, also due to its contrast with the proprioception, the estimate of δ represents a technical challenge and an interesting direction for future research. Biological interpretations The application of the 2D model with δ ≈ 0 and calibrated with field data showed that a full description of primary and secondary growth processes as well as a description of the sensing activity was fundamental to reproducing the self-supporting growth of searcher shoots. Numerical simulations were able to reproduce similar maximal reach capacities as those observed in the field for both the samples in consideration. Simulations showed that both exhibited horizontal trajectories of self-supporting growth to attain their maximal reach capacities. 2D postures obtained at the end of the simulations were qualitatively equivalent to the ones observed in the field. Both species expressed a self-weight movement induced by mass distribution (in space) and accretion (in time). Nevertheless, they differed in terms of postural dynamics. These differences can be interpreted numerically by both sensing and mechanical parameters. From a sensing point of view, C. guianense exhibited a stronger anti-gravitropic behaviour than T. jasminoides. These explain the apical orientation of searcher shoots leading to an upward growth for C. guianense and a downward growth for T. jasminoides. From a mechanical point of view, C. guianense was more strongly effected than T. jasminoides in maintaining an upward growth in view of a faster mass accretion along the searcher shoot. However, these biological interpretations must be interpreted with caution with respect to the size of the dataset and observations that have been made under natural conditions without control over external factors. The samples considered in this work are not representative of all the variations we expect to see in nature. See for instance Section H in S1 Text for further photos of samples. Future improvements The model developed here provides a theoretical and analytical framework that can be a useful tool to better understand the strategies of lianas to cross gaps with their searcher shoots under the effects of gravitational constraints. In particular, it allows us to see how the mass distribution, the stiffness and the length of the shoot influence its development and space orientation. To this purpose, the model parameters are fitted with the experimental data. As displayed in Section, for the T. jasminoides case this fitting procedure returns a negative parameter β, which is related to the vertical direction of the stimuli response activity. Since the climbing plant samples were not raised in a controlled environment, we cannot immediately interpret the negative value of β and the related behaviour of the simulation (Fig 3) with positive gravitropic behaviour. Nevertheless, the relation between the values of the sensing parameters and the observed behaviour of the climbing plant under unpredictable environmental conditions can be a matter of further studies. Despite the uncontrolled environment and the basic measurements based on fully elongated shoots, the 2D model is revealed to be robust enough to reproduce the self-supporting state of the climbing plant species considered in this work. Moreover, the model can already take into account the tissue anatomical variations along the stems in terms of flexural rigidity, mass and radius, and how such variations affect the final climbing plant shape. Moving in the direction of a more in-depth study of the shoot structure, it may be interesting to investigate how the biomechanical state and biomass allocation change during the extension of a searcher shoot from a dynamic point of view. This would imply working on a large set of searcher shoots, regularly sacrificing some of them for destructive measurements, and reasoning in terms of chronosequence (i.e. by extrapolation), trying to work under conditions that are as homogeneous as possible in order to minimize the uncertainty factors (ontogenetic stage, environment, etc.). These measurements associated with the anatomical study of the tissues could allow a better understanding of the mechanism at the origin of the rigidification dynamics at all points of the searcher shoots. The anatomical organizations in the mechanical and postural life-histories of plants are potentially embracing ecology, biomechanics and robotics [36]. Searcher shoots are highly diverse in climbing plants and can have relatively complex structures depending on types of connected structures such as leaves, hooks, tendrils, branches, stem segments capable of circumnutatory movements, as well as intrinsic stem extension and postural dynamics [29]. The movements associated with these structures and generated by growth differentials within tissues are of significant importance for exploring and crossing spaces and attaching to supports. The model considered here in 2D is relatively simple and corresponds to an unbranched shoot with differential dynamics between internodes and leaves. The consideration of more complex shoot structures, involving variable mass load distributions in space and time and in a 3D volume is a higher step to properly simulate the self-supporting and searching mobility of searcher shoots in climbing plants. For instance, a 3D version of the model presented in this paper could be used to understand the role of circumnutation in climbing plant searching strategy. Another possible application of a 3D model could be to study how climbing plants change their mechanical internal properties when they encounter external obstacles to which they attach. All these problems cannot be addressed by using a 2D model and will be the subject of future studies. Conclusion We formulated a model that takes secondary growth and proprioception into account. We showed how it is possible to estimate the parameters of the model from experimental data. Using the information about the extension of the stem, we computed the extension parameter G0 and the length of the extension zone ℓg. Then, based on experimental data on the final reach and orientation of each stem, we calibrated the sensing parameters α, β and γ. Based on such measurements and on the related simulations in Fig 3, we were able to understand the role of the weight in the shapes observed in the photos in Fig 3. For the two samples under consideration, we were able to say that the mechanical effect of the weight plays an important role in the shape of the C. guianense, while the downward growth of the T. jasminoides is mainly the consequence of its sensing activity. These conclusions are supported by the simulations based on a simplified model which considers just the plant sensing activity (see Fig 4). Such results confirm that the mechanical aspects in climbing plant searcher shoots, and, in particular, the variable linear density and the secondary growth are not negligible in order to obtain realistic climbing plant shapes in experimental data-based simulations. Supporting information S1 Text. Section A: Derivation of the mechanical model; Section B: Derivation of the secondary growth equation; Section C: Explicit solution to the extension equation; Section D: Numerical Integration of the model; Section E: Formulas used to retrieve the biomechanical properties of the samples; Section F: Fitting Figures; (G): Stability of the simulations. Figure A: Plot of the functions resulting from the fitting procedure for T. jasminoides. In blue we plot fitted curves for volume density, radius, radial expansion rate, flexural rigidity and leaves mass. Together with such plots, we display also the values of the experimental data and the curve of the absolute error. Figure B: Plot of the functions resulting from the fitting procedure for C. guianense. In blue we plot fitted curves for volume density, radius, radial expansion rate, flexural rigidity and leaves mass. Together with such plots, we display also the values of the experimental data and the curve of the absolute error. Figure C: (a)—(b): Simulations for C. guianense and T. jasminoides respectively, with polynomial flexural rigidity B. (c)—(d): Comparison between simulations displayed in Fig 3 of the main text with Figures Ca-Cb. The simulations obtained from the polynomial fitting are in magenta, while those with the sigmoid fitting are in green. Figure D: (a)—(b): two samples of T. jasminoides. In (a), the tip of the sample displays a negative gravitropic behaviour, while in (b) the sample has developed a main stem horizontally directed. (c)—(d): two samples of C. guianense. In (a), the main stem is only horizontally directed, while in (b), the tip is curling downwards. https://doi.org/10.1371/journal.pcbi.1011538.s001 (PDF) S1 Table. Experimental morphological data. https://doi.org/10.1371/journal.pcbi.1011538.s002 (CSV) S2 Table. Experimental growth data. https://doi.org/10.1371/journal.pcbi.1011538.s003 (ZIP) S1 Code. Code used for the simulations. Simulation of model (5) with null delta https://zenodo.org/record/8359924, simulation of model (5) with positive delta https://zenodo.org/record/8360202, simulation of the weightless model (6) https://zenodo.org/record/8360221. https://doi.org/10.1371/journal.pcbi.1011538.s004 (ZIP) Acknowledgments We thank Laureline Petit-Bagnard for her assistance on the field.

Massonis, Gemma;Villaverde, Alejandro F.;Banga, Julio R.

doi: 10.1371/journal.pcbi.1011014pmid: 37851682

Introduction Mathematical models are increasingly used to describe, monitor, analyze and predict the behavior of complex biological systems. One of the major benefits of using mathematical models to study biology is that they can provide an objective and quantitative understanding that would be difficult to achieve through any other means. In systems biology, dynamical models (typically sets of ordinary differential equations, ODEs) are widely used to provide mechanistic insights into the functioning of biological systems [1, 2]. The use of dynamical systems theory originated in Newtonian mechanics is now pervasive in all the natural and engineering sciences [3]. Dynamic models are highly versatile, enabling researchers to study complex biosystems from a range of different perspectives, such as (i) analyzing the effect of changes in conditions and scenarios different from those studied experimentally, (ii) guiding research by identifying key aspects that need to be further investigated, (iii) helping to generate new testable hypotheses, or (iv) guiding the design of interventions. However, the systematic development of mechanistic dynamic models is a non-trivial exercise. In the case of biological systems, the situation is particularly difficult due to the fact that we cannot rely on first principles in the same way as in e.g. physics. As a consequence, model development is one of the key open problems in mathematical biology [4]. Can we automate the development of mechanistic models? This question of model discovery (in the sense of symbolic reconstruction of equations) from data was already addressed by pioneering attempts in the field of artificial intelligence several decades ago [5–7]. However, the data-driven automatic identification of nonlinear dynamic models has only been addressed more recently. In this area, several different statistical and machine learning frameworks have been considered, including symbolic regression [8, 9], grammar-based methods [10, 11], sparse regression [12], neural networks [13–15], Gaussian process regression [16, 17] and Bayesian approaches [18–20]. More detailed reviews can be found in [21–25]. The sparse identification of nonlinear dynamics (SINDy) algorithm [12] has been particularly successful, and a number of extensions have been developed (see review in [26]). In the case of biological systems, a large amount of research has been devoted to different classes of subproblems with different simplifying assumptions (such as e.g. static networks, non-mechanistic dynamic networks, linear dynamics, etc.), as reviewed by [27–29]. In this work, we consider the more general problem of fully reconstructing interpretable (mechanistic and parameterized) nonlinear dynamic models from time-series data. Recently, several approaches using methods based on sparse regression, Bayesian identification or symbolic regression have appeared [18, 30–36]. In this context, SINDy-PI [37] is an especially interesting parallel implicit version of SINDy because it allows the incorporation of implicit dynamics and rational nonlinear terms, thus enabling the discovery of kinetic functions (such as Michaelis-Menten) common in biochemical networks. Many of these SINDy-based methods pay special attention to the recovery of parsimonious models, usually penalizing model complexity [38] or evaluating performance on a validation data-set [37]. The objective is to find the simplest model which can explain the data, in agreement with the well known principle of Occam’s razor. These strategies help to discard more complex models which would be indistinguishable (i.e. would explain the data equally well but adding spurious terms). Besides enforcing simplicity, a related key aspect in model discovery is ensuring structural identifiability and observability (SIO). The property of structural identifiability refers to the theoretical possibility of inferring the unknown parameters of a given model (assuming that its equations are known, except for the numerical values of the parameters) from observations of the model output, which typically consist of time-resolved measurements of its state variables, or of a subset of them [39]. Likewise, observability is the possibility of inferring all the state variables of a model at a given time from future observations of a subset of them. Since lack of SIO makes it impossible to uniquely infer parameters and state variables, it can compromise the usefulness of the model [40–46]. The analysis of these properties can be performed with symbolic computation tools [47], and numerical approaches have also been proposed for their study [48, 49]. However, to the best of our knowledge, ensuring SIO has not been considered in dynamic model discovery yet. Here we present a methodology that ensures SIO in automatic model discovery in two possible scenarios: with and without prior knowledge. In both cases the end product is a dynamic model of a biological system consisting of (typically nonlinear) ODEs. The equations may contain rational terms, such as Michaelis-Menten kinetics, thus being suitable for the description of many biochemical processes. If there is no prior knowledge about the model structure, the methodology performs equation discovery with the SINDy-PI approach, and incorporates a SIO analysis as a post-processing stage. If there is prior knowledge (i.e. we have a candidate model), another SIO analysis is added as a pre-processing step. If the analyses reveal structural unidentifiabilities, a reparameterization step is carried out to ensure that the resulting model is fully identifiable and observable. Furthermore, equivalent model reformulations are generated to facilitate its interpretation in a mechanistic sense. Using representative case studies, we illustrate how ignoring these structural properties can lead to wrong conclusions or poorly identified models. Although we demonstrate the use of the methodology with SINDy-PI, it is straightforward to apply it in combination with other automatic discovery methods. In particular, it could easily be adapted to future methods capable of considering partially-observed systems. Overall, our study presents a novel and non-trivial integrated methodology to ensure that the discovered models are structurally identifiable and interpretable. To the best of our knowledge, this is the first study to address these questions in model discovery. Further, our method involves an original and non-obvious combination of algorithmic steps regarding structural identifiability analysis (SIO), reparameterization, reformulation and interpretability analysis. While the concepts of SIO and reparameterization draw on recent ideas developed in our group, the remaining steps and their integration represent fresh and innovative contributions to the field. Methods In this section we describe the methodology, which can be used in two different scenarios. Both of them entail performing model discovery (using SINDy-PI or a similar approach) and performing SIO analysis. If a model is structurally identifiable and observable, we say that it is FISPO (full input, state, and parameter observability). If the SIO analysis reveals that the model is not FISPO, our method suggests a reparameterization step. The two scenarios and their procedures are as follows: Scenario (I): full model discovery from time-series data with no prior knowledge. Since we assume zero prior knowledge, we use SINDy-PI to discover a candidate model (CM). We then analyse its SIO. If it is not FISPO, we reparameterize it in order to obtain an equivalent model which is FISPO. Finally, we check if the model is interpretable, in the sense that it contains monomials and simple rational terms which belong to a dictionary of mechanistic kinetic terms. If not, we apply a symbolic reformulation step in order to render it interpretable. Scenario (II): model discovery from time-series data with prior knowledge. This scenario corresponds to situations where we seek model (in)validation and/or refinement. We assume good prior knowledge and time-series data, that is we are reasonably confident that our prior model (PM), which already represents the data quite well, is close to the ‘true’ one. Here the motivation to use SINDy-PI is to compare this PM with an alternative candidate obtained via model discovery (CM). To this end we check the SIO of the PM, obtaining a reparameterized version if needed. In parallel, we apply SINDy-PI to the data, obtaining a CM, and we make sure that it is FISPO (using reparameterisation if not). If needed, we use model reformulation techniques to obtain interpretable versions of the CM and the PM. Finally, we perform a comparative analysis of these latter models. An schematic diagram of our method considering these two scenarios is depicted in Fig 1. In the remainder of this section we describe in detail each of the steps. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Workflow of the methodology. Scenario (I) (solid black lines only): data-driven full model discovery from (time-series) data with no prior knowledge. We apply SINDy-PI and test the SIO of the discovered candidate model (CM). If the CM is not FISPO, we reparameterize it. Next, we check if the model is interpretable; if not, we reformulate it via symbolic manipulation. The result is a FISPO interpretable model, M*. Scenario (II) (solid black lines + dashed, dark orange lines in the lower part): model discovery from (time-series) data with good prior knowledge. In this scenario we seek model (in)validation and/or refinement. We have a prior model (PM) which we want to compare with an alternative candidate discovered from data (CM). To this end, we check the SIO of the PM and reparameterize if needed. In parallel, we apply SINDy-PI to the data to obtain a CM, and we make sure it is FISPO (using reparameterisation if not). Then, we use model reformulation techniques to ensure interpretable versions (M* and PM*) if needed. Lastly, we perform a comparative analysis. https://doi.org/10.1371/journal.pcbi.1011014.g001 Automatic model discovery using sparse regression We assume that the dynamical system is governed by classical reaction-rate nonlinear ordinary differential equations with the following form: where is the state vector, is the parameter vector, the function f(x(t), p) represents the dynamics, y(t) is the measurable output, and x0 is the vector of initial conditions. SINDy [12] assumes a fully observed system, y(t) = x(t). In the reminder of this section, we will consider to simplify the notation. SINDy also assumes that f(x(t)) can be expressed as the product of a suitable library function, Θ(x(t)), and a sparse vector ξ (indicating the active library terms), where each entry in the library function is a candidate term: (1) By arranging the time-series data as a matrix, X = [x(t1), …, x(tm)], and its associated derivative matrix , can be expressed as: (2) where Ξ corresponds to the sparse matrix of active terms. When the system includes rational terms, f(x) can be rewritten as: (3) leading to the implicit problem formulation [31]: (4) Eq 4 has a different kind of term in each side of the equality: the Left Hand Side (LHS), in which there are combinations of term involving the derivative data and the candidate library, and the Right Hand Side (RHS), in which we only have library terms. When f(x) includes rational terms, model complexity can be viewed as the number of terms in the LHS, as they will involve the denominator degree too. The generalized function library Θ(X) allows the inclusion of X and . Under this consideration, the implicit problem formulation can be rewritten as: (5) For example, if a model has two states and the chosen degree is 2, the function library for the first state, i.e. x1, will be: (6) It should be noted that the function library for state x2 will differ from Eq 6 as it will include instead of . The design of the library of candidate functions is a critical aspect of SINDy-PI. However, in the context of dynamic modelling of biochemical and biological phenomena, by including monomials up to an order of 5 or 6, we can accommodate the vast majority of nonlinear terms, such as e.g. mass-action kinetics, or feedback regulatory loops. Furthermore, common nonlinear rational terms such as Michaelis-Menten in enzyme kinetics, or Monod in microbial growth, can be inferred from such library due to the implicit nature of SINDy-PI. The case studies considered here cover a wide range of nonlinear terms, illustrating the capabilities of SINDy-PI in biological modelling. The implicit form of Eq 5 admits the sparsest trivial solution Ξ = 0. The implicit-SINDy algorithm [31] surmounted this issue by using nonconvex optimization to find the sparsest vector ξ in the null space. However this particular formulation is very sensitive towards noise levels, thereby affecting the robustness of the method. In an effort to address this issue, Kaheman et al. [37] introduced SINDy-PI, a novel method that solves the problem using a sequence of convex relaxations of the non-convex optimization problem. By doing so, the algorithm can utilize the same noise levels as those employed in the original SINDy algorithm. The authors achieved this by assuming the knowledge of at least one term on the LHS, specifically of the form . Consequently, Eq 5 can be rewritten as: (7) where denotes the library without the θj element. SINDy-PI proceeds iteratively by examining each term within the library. In order to balance accuracy and complexity, the method performs Pareto optimal model selection. To find the sparsest vector ξ, SINDy-PI considers the problem: (8) SINDy-PI solves this non–convex optimization problem using a sequentially thresholded least-squares (STLSQ) approach. This method proceeds by iteratively solving the least squares term in the cost function, zeroing out elements of ξ that are below a certain threshold λ. This threshold must be fine-tuned to select the model that provides the best trade-off between accuracy and efficiency. To discover the model, SINDy-PI considers a finite set of λ values and proceeds by sweeping the library terms for each value of the threshold λ, obtaining a family of possible candidate models. Next, SINDy-PI performs model selection choosing the best trade-off, i.e. the Pareto-optimal model. The Pareto front is obtained by considering a model complexity metric (such as the Akaike information criterion, AIC), and the score for each candidate model. Details of this process, illustrated with an example, are given in the Supporting Information. Structural identifiability and observability analysis and reparameterization Once a candidate model structure (CM) has been discovered, the next step is to analyse its structural identifiability and observability (SIO) [50]. This test assesses the possibility of determining the values of the model parameters and state variables, respectively, from output measurements. These properties are structural (i.e. they depend only on the model equations) and hence they can be analysed a priori (i.e. before taking experimental measurements) using symbolic computation. They should not be confused with the so-called practical versions of these properties, which depend on the features of the experimental data and are analysed a posteriori, i.e. after performing measurements [39]. We can provide a mathematical definition of structural local identifiability (SLI) as follows. Let us denote by y(t, p) the output vector obtained with a parameter vector p at time t. (For fully observed systems y(t, p) = x(t, p), while for partially observed systems y typically consists of a subset of x.) We say that a parameter pi (which is the ith element of the parameter vector ) is structurally locally identifiable (SLI) if, for almost any parameter vector , there is a neighbourhood such that: (9) The definition of structural global identifiability is similar, but with the neighbourhood extending to the whole parameter space. In this paper we focus on SLI. There are several approaches for determining structural local identifiability and observability. We apply a differential geometry approach, which we explain briefly in these paragraphs. In this framework, parameters are treated as state variables that happen to be constant, i.e. the state vector is augmented with the parameters, , and has dimension The augmented dynamic equations are and the output function is omitting the dependence on time for ease of notation. Thus, SLI is considered as a particular case of a more general property, observability, which describes the possibility of inferring the internal state of a model by observing its output vector—hence the use of the term FISPO for “full input, state, and parameter observability” (note that this concept also allows for the treatment of unknown inputs as additional state variables, a possibility that we will not consider in this paper). We analyse SIO by building an observability-identifiability matrix and computing its rank. The matrix is built with Lie derivatives of the output function. The zero-order Lie derivative is and for i ≥ 1 the i–order Lie derivatives are obtained as: The observability-identifiability matrix is: (10) A model is FISPO around a point if the rank of its observability-identifiability matrix equals the number of its states and parameters, rank . If the rank is smaller, the model contains structurally unidentifiable parameters. By performing additional tests it is possible to determine which specific parameters are structurally identifiable, and which state variables are observable. If a model is not FISPO, its calibration will almost surely produce wrong parameter estimates. Furthermore, structural unidentifiability is often linked with non-observability, in which case the simulations of some state variables will also be wrong. Thus, structural non-identifiability and non-observability are undesirable features of a model’s structure, which compromise its reliability as a source of biological insight. These features are caused by symmetries in the differential equations of the model that make its output invariant with respect to certain changes in their parameters and/or state variables [51–53]. Said symmetries can be studied in the framework of Lie group theory. We say that a mapping of the form (11) is a one-parameter Lie group of transformations (with ε being the parameter) if it has the following properties: it is smooth in x and analytic in ε, it satisfies the four group axioms (closure, associativity, and existence of an identity and an inverse), and the identity element can be chosen as ε = 0. The transformation above is also called a symmetry transformation, or a Lie symmetry. Examples of the simplest and possibly most common symmetries in biological modelling include the following: Translation: (12) Scaling: (13) Moebius: (14) It is sometimes possible to remove or ‘break’ these symmetries by transforming the model equations via a suitable reparameterization. To this end, we first search for the symmetry transformations admitted by the model. If a model has such symmetries, it is overparameterized and therefore structurally unidentifiable. Then, we express the ε of those transformations in terms of other parameters, thus setting the value of one of the transformed parameters to one and removing it from the equations. The end result of the reparameterization is a FISPO model that has exactly the same dynamic behaviour as the original one. In previous work [54] we presented a methodology to perform such reparameterizations automatically, which has been integrated in the workflow described here. In summary, if the SIO analysis of the CM reveals structural unidentifiability and/or non-observability, our methodology applies a symmetry-breaking reparameterization that makes it FISPO. Model reformulation for interpretability The dynamic model obtained in the previous step supports the experimental data and is structurally identifiable and observable. However, the rational expressions in 3 may lack a clear biological interpretation. In the case of biological networks, we will need to reformulate expressions of the form N(x)/D(x) into terms that belong to a dictionary of interpretable terms. Our model reformulation procedure seeks to transform it into simple monomials and rational terms that have a mapping with the dictionary of kinetic and regulatory terms compatible with the specific type of biological reaction network under study. Typically, this dictionary will include mass-action kinetics and simple rational functions (e.g. Michaelis-Menten for enzyme kinetics, or Hill for cooperative binding). However, care should be taken in order to ensure that the reformulation does not destroy identifiability and observability. Further, as shown in the case studies below, sometimes these rational terms can have high degrees, complicating model discovery. Our reformulation procedure makes use of symbolic manipulation and involves the following steps: Obtain the list of p non-trivial divisors of the denominator: (15) Obtain a family A of expressions composed of monomials (interpretable as e.g. mass-action kinetics), minimizing the number of rational terms and their degree, by obtaining the quotients and the residuals: (16) If any residuals ri(x) lack interpretability, factorize and simplify N(x) by means of the nested Horner form: (17) obtaining a family of coupled and factorized equations with the same degree, but with monomials involving different state combinations: (18) (19) Thus, the Horner nested form gives different possible decompositions of the numerator. Next, obtain a family B of reformulations by simplifying the rational terms using the divisors of Eq 15 and the Horner form of the numerator. As an example, consider Eq 20 below, where we can decompose the fraction in the left into a monomial plus a simpler rational term as follows: (20) Match the monomials and simplified rational terms in families A and B with elements in the dictionary of canonical kinetic and regulatory expressions (or by inspection by a human domain expert), finding members that are fully interpretable. Ensure that the resulting interpretable model is FISPO. If not, reparameterize and repeat until an interpretable and identifiable model is obtained. Implementation We implemented our methodology, as depicted in Fig 1, using Matlab and the Symbolic Math Toolbox, integrating the following components: Sparse regression using SINDy-PI [37] with some modifications, as detailed in the Supporting Information. Structural identifiability and observability (SIO) analysis using the algorithm FISPO [55], plus reparameterization using the algorithm AutoRepar [54], as implemented in STRIKE-GOLDD 4.0 and later releases [56]. Reformulation for interpretability, implementing the algorithm described above using symbolic manipulation. The resulting code is available at https://doi.org/10.5281/zenodo.7713047. In order to facilitate reproducibility and illustrate the results at each step of the workflow, we have included interactive notebooks (Matlab live scripts) and reports (in HTML format) for each of the case studies described below. More details are given in S1 File. Automatic model discovery using sparse regression We assume that the dynamical system is governed by classical reaction-rate nonlinear ordinary differential equations with the following form: where is the state vector, is the parameter vector, the function f(x(t), p) represents the dynamics, y(t) is the measurable output, and x0 is the vector of initial conditions. SINDy [12] assumes a fully observed system, y(t) = x(t). In the reminder of this section, we will consider to simplify the notation. SINDy also assumes that f(x(t)) can be expressed as the product of a suitable library function, Θ(x(t)), and a sparse vector ξ (indicating the active library terms), where each entry in the library function is a candidate term: (1) By arranging the time-series data as a matrix, X = [x(t1), …, x(tm)], and its associated derivative matrix , can be expressed as: (2) where Ξ corresponds to the sparse matrix of active terms. When the system includes rational terms, f(x) can be rewritten as: (3) leading to the implicit problem formulation [31]: (4) Eq 4 has a different kind of term in each side of the equality: the Left Hand Side (LHS), in which there are combinations of term involving the derivative data and the candidate library, and the Right Hand Side (RHS), in which we only have library terms. When f(x) includes rational terms, model complexity can be viewed as the number of terms in the LHS, as they will involve the denominator degree too. The generalized function library Θ(X) allows the inclusion of X and . Under this consideration, the implicit problem formulation can be rewritten as: (5) For example, if a model has two states and the chosen degree is 2, the function library for the first state, i.e. x1, will be: (6) It should be noted that the function library for state x2 will differ from Eq 6 as it will include instead of . The design of the library of candidate functions is a critical aspect of SINDy-PI. However, in the context of dynamic modelling of biochemical and biological phenomena, by including monomials up to an order of 5 or 6, we can accommodate the vast majority of nonlinear terms, such as e.g. mass-action kinetics, or feedback regulatory loops. Furthermore, common nonlinear rational terms such as Michaelis-Menten in enzyme kinetics, or Monod in microbial growth, can be inferred from such library due to the implicit nature of SINDy-PI. The case studies considered here cover a wide range of nonlinear terms, illustrating the capabilities of SINDy-PI in biological modelling. The implicit form of Eq 5 admits the sparsest trivial solution Ξ = 0. The implicit-SINDy algorithm [31] surmounted this issue by using nonconvex optimization to find the sparsest vector ξ in the null space. However this particular formulation is very sensitive towards noise levels, thereby affecting the robustness of the method. In an effort to address this issue, Kaheman et al. [37] introduced SINDy-PI, a novel method that solves the problem using a sequence of convex relaxations of the non-convex optimization problem. By doing so, the algorithm can utilize the same noise levels as those employed in the original SINDy algorithm. The authors achieved this by assuming the knowledge of at least one term on the LHS, specifically of the form . Consequently, Eq 5 can be rewritten as: (7) where denotes the library without the θj element. SINDy-PI proceeds iteratively by examining each term within the library. In order to balance accuracy and complexity, the method performs Pareto optimal model selection. To find the sparsest vector ξ, SINDy-PI considers the problem: (8) SINDy-PI solves this non–convex optimization problem using a sequentially thresholded least-squares (STLSQ) approach. This method proceeds by iteratively solving the least squares term in the cost function, zeroing out elements of ξ that are below a certain threshold λ. This threshold must be fine-tuned to select the model that provides the best trade-off between accuracy and efficiency. To discover the model, SINDy-PI considers a finite set of λ values and proceeds by sweeping the library terms for each value of the threshold λ, obtaining a family of possible candidate models. Next, SINDy-PI performs model selection choosing the best trade-off, i.e. the Pareto-optimal model. The Pareto front is obtained by considering a model complexity metric (such as the Akaike information criterion, AIC), and the score for each candidate model. Details of this process, illustrated with an example, are given in the Supporting Information. Structural identifiability and observability analysis and reparameterization Once a candidate model structure (CM) has been discovered, the next step is to analyse its structural identifiability and observability (SIO) [50]. This test assesses the possibility of determining the values of the model parameters and state variables, respectively, from output measurements. These properties are structural (i.e. they depend only on the model equations) and hence they can be analysed a priori (i.e. before taking experimental measurements) using symbolic computation. They should not be confused with the so-called practical versions of these properties, which depend on the features of the experimental data and are analysed a posteriori, i.e. after performing measurements [39]. We can provide a mathematical definition of structural local identifiability (SLI) as follows. Let us denote by y(t, p) the output vector obtained with a parameter vector p at time t. (For fully observed systems y(t, p) = x(t, p), while for partially observed systems y typically consists of a subset of x.) We say that a parameter pi (which is the ith element of the parameter vector ) is structurally locally identifiable (SLI) if, for almost any parameter vector , there is a neighbourhood such that: (9) The definition of structural global identifiability is similar, but with the neighbourhood extending to the whole parameter space. In this paper we focus on SLI. There are several approaches for determining structural local identifiability and observability. We apply a differential geometry approach, which we explain briefly in these paragraphs. In this framework, parameters are treated as state variables that happen to be constant, i.e. the state vector is augmented with the parameters, , and has dimension The augmented dynamic equations are and the output function is omitting the dependence on time for ease of notation. Thus, SLI is considered as a particular case of a more general property, observability, which describes the possibility of inferring the internal state of a model by observing its output vector—hence the use of the term FISPO for “full input, state, and parameter observability” (note that this concept also allows for the treatment of unknown inputs as additional state variables, a possibility that we will not consider in this paper). We analyse SIO by building an observability-identifiability matrix and computing its rank. The matrix is built with Lie derivatives of the output function. The zero-order Lie derivative is and for i ≥ 1 the i–order Lie derivatives are obtained as: The observability-identifiability matrix is: (10) A model is FISPO around a point if the rank of its observability-identifiability matrix equals the number of its states and parameters, rank . If the rank is smaller, the model contains structurally unidentifiable parameters. By performing additional tests it is possible to determine which specific parameters are structurally identifiable, and which state variables are observable. If a model is not FISPO, its calibration will almost surely produce wrong parameter estimates. Furthermore, structural unidentifiability is often linked with non-observability, in which case the simulations of some state variables will also be wrong. Thus, structural non-identifiability and non-observability are undesirable features of a model’s structure, which compromise its reliability as a source of biological insight. These features are caused by symmetries in the differential equations of the model that make its output invariant with respect to certain changes in their parameters and/or state variables [51–53]. Said symmetries can be studied in the framework of Lie group theory. We say that a mapping of the form (11) is a one-parameter Lie group of transformations (with ε being the parameter) if it has the following properties: it is smooth in x and analytic in ε, it satisfies the four group axioms (closure, associativity, and existence of an identity and an inverse), and the identity element can be chosen as ε = 0. The transformation above is also called a symmetry transformation, or a Lie symmetry. Examples of the simplest and possibly most common symmetries in biological modelling include the following: Translation: (12) Scaling: (13) Moebius: (14) It is sometimes possible to remove or ‘break’ these symmetries by transforming the model equations via a suitable reparameterization. To this end, we first search for the symmetry transformations admitted by the model. If a model has such symmetries, it is overparameterized and therefore structurally unidentifiable. Then, we express the ε of those transformations in terms of other parameters, thus setting the value of one of the transformed parameters to one and removing it from the equations. The end result of the reparameterization is a FISPO model that has exactly the same dynamic behaviour as the original one. In previous work [54] we presented a methodology to perform such reparameterizations automatically, which has been integrated in the workflow described here. In summary, if the SIO analysis of the CM reveals structural unidentifiability and/or non-observability, our methodology applies a symmetry-breaking reparameterization that makes it FISPO. Model reformulation for interpretability The dynamic model obtained in the previous step supports the experimental data and is structurally identifiable and observable. However, the rational expressions in 3 may lack a clear biological interpretation. In the case of biological networks, we will need to reformulate expressions of the form N(x)/D(x) into terms that belong to a dictionary of interpretable terms. Our model reformulation procedure seeks to transform it into simple monomials and rational terms that have a mapping with the dictionary of kinetic and regulatory terms compatible with the specific type of biological reaction network under study. Typically, this dictionary will include mass-action kinetics and simple rational functions (e.g. Michaelis-Menten for enzyme kinetics, or Hill for cooperative binding). However, care should be taken in order to ensure that the reformulation does not destroy identifiability and observability. Further, as shown in the case studies below, sometimes these rational terms can have high degrees, complicating model discovery. Our reformulation procedure makes use of symbolic manipulation and involves the following steps: Obtain the list of p non-trivial divisors of the denominator: (15) Obtain a family A of expressions composed of monomials (interpretable as e.g. mass-action kinetics), minimizing the number of rational terms and their degree, by obtaining the quotients and the residuals: (16) If any residuals ri(x) lack interpretability, factorize and simplify N(x) by means of the nested Horner form: (17) obtaining a family of coupled and factorized equations with the same degree, but with monomials involving different state combinations: (18) (19) Thus, the Horner nested form gives different possible decompositions of the numerator. Next, obtain a family B of reformulations by simplifying the rational terms using the divisors of Eq 15 and the Horner form of the numerator. As an example, consider Eq 20 below, where we can decompose the fraction in the left into a monomial plus a simpler rational term as follows: (20) Match the monomials and simplified rational terms in families A and B with elements in the dictionary of canonical kinetic and regulatory expressions (or by inspection by a human domain expert), finding members that are fully interpretable. Ensure that the resulting interpretable model is FISPO. If not, reparameterize and repeat until an interpretable and identifiable model is obtained. Implementation We implemented our methodology, as depicted in Fig 1, using Matlab and the Symbolic Math Toolbox, integrating the following components: Sparse regression using SINDy-PI [37] with some modifications, as detailed in the Supporting Information. Structural identifiability and observability (SIO) analysis using the algorithm FISPO [55], plus reparameterization using the algorithm AutoRepar [54], as implemented in STRIKE-GOLDD 4.0 and later releases [56]. Reformulation for interpretability, implementing the algorithm described above using symbolic manipulation. The resulting code is available at https://doi.org/10.5281/zenodo.7713047. In order to facilitate reproducibility and illustrate the results at each step of the workflow, we have included interactive notebooks (Matlab live scripts) and reports (in HTML format) for each of the case studies described below. More details are given in S1 File. Results Below, we apply our methodology to a set of challenging case studies. (Table 1 summarises their main features). These examples are presented in order to illustrate the performance of our method for a variety of situations of increasing complexity, from models without rational terms and fully identifiable and observable (FISPO) structure, to larger (in terms of number of parameters and states), non-FISPO models with more difficult non-linearities, as indicated by the different maximum degrees in their rational terms. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Main features of the case studies: Relevant references and main characteristics of the models considered in the case studies. The fourth and fifth rows show the maximum degree of N(x) and D(x) in Eq 4. The last row indicates if the original (ground truth, GT) model is fully identifiable and observable (FISPO). https://doi.org/10.1371/journal.pcbi.1011014.t001 In order to illustrate all the steps and capabilities of our workflow, we consider Scenario (II) in all these examples. For each problem, a ground truth (GT) model is defined and subsequently considered as prior model (PM) for the sake of simplicity but without loss of generality. This GT model is used to generate training data sets in all the case studies. After confirming the identifiability and interpretability of the final discovered model M*, we also assess its structural, parametric and predictive accuracy. The predictive power is evaluated taking into account conditions different from those used for generating the training data. Details regarding the training data generation and the conditions to evaluate predictive accuracy are given in S1 File. Lorenz system (Lorenz) This case study involves the well-known Lorenz system [57], which is a classical example of dynamic model with chaotic behaviour. This model was previously used in [12] to demonstrate the original SINDy algorithm. The governing equations describe the dynamics of a fluid layer warmed from below and cooled from above: (21a) (21b) (21c) where x1 is proportional to the rate of convection, x2 to the horizontal temperature variation and x3 to the vertical one. For certain values of parameters a, b, and c, the system exhibits chaotic dynamics. We consider the ideal Scenario (II) case where the prior model (PM) is the same as the nominal (or ground truth, GT) model. We generate a synthetic training data set using the GT model and settings similar to [12] (details in Supporting Information). Following the workflow in Fig 1, we perform structural and identifiability analysis and confirm that the PM is fully identifiable and observable (FISPO). We then apply SINDy-PI to the training data, obtaining the following candidate model (CM): (22a) (22b) (22c) Our algorithm then confirms that this inferred CM model is FISPO and interpretable (thus, it corresponds to M*). Further, it is fully equivalent to the expanded ground truth model in terms of structural, parametric and predictive accuracy, as shown in Fig 2. In summary, in this case study we have the ideal situation where both the nominal and the inferred models are fully observable and identifiable. As we will see below, this situation might change as soon as we consider rational terms in the dynamics. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Lorenz case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1, x2 and x3) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g002 Competition between bacteria and the immune system (Immunity) This model describes the influence of quorum sensing signaling molecules (QSSM) on the competition between bacteria and the immune system, as studied in [58]. The following differential equations depict the dynamics for the concentrations of bacteria () and immune cells (): (23a) (23b) where it is assumed that bacteria grow logistically at rate a with an effective carrying capacity of the environment given by parameter k, and that they are cleared by the immune system following a mass action term ex1x2. The rational term at the end of Eq 23a represents the modulation of QSSM in the competition between bacteria and the immune system. We consider Eqs 23a and 23b as the GT model. We consider Scenario (II) again, assuming that the prior model (PM) is the same as the ground truth (GT) or nominal model. Considering that the unknown parameters are a, k, e, β, γ, α, S, d and δ, the structural identifiability analysis of the PM indicates that two of the three parameters involved in the rational term are unidentifiable. Specifically, there is a scaling symmetry between γ and α, which is probably the most common type of symmetry in biological models [63]. Our reparameterization step indicates that this issue can be solved by dividing the numerator and denominator by one of the unidentifiable parameters; for example, if we choose α, Eq 23a will be: (24) where γ/α = γ*. Thus our new reference model will be the following PM*: (25a) (25b) Next, our workflow proceeds by applying SINDy-PI to a data set generated with the GT model, obtaining the following candidate model (CM): (26a) (26b) Interestingly, our method then finds that this CM is not FISPO due to three structurally unidentifiable parameters: p4, p5, p6. The reformulation step is then able to find structurally identifiable reformulations of the form: (27) We chose j = 6, but the same result can be obtained with j = 4, 5. Denoting as , the resulting dynamic system becomes identifiable. Eq 27 is re-arranged as: (28) The resulting model is fully identifiable, but the rational term does not match the one in the GT describing the modulation of QSSM. However, the reformulation step in our workflow produces an interpretable form M* which is FISPO: (29a) (29b) Fig 3 illustrates the importance of ensuring structural identifiability and observability. The CM found by SINDy-PI is not FISPO, so there are other different parameter realizations producing exactly the same output, as shown by CM2. This means that if this CM structure is used for parameter identification, the estimated parameters will not be unique, i.e. there exist different parameterizations of CM in full agreement with the same output measurements. However, the reformulated model M* is FISPO, i.e. there is a unique set of parameter values compatible with the output. Finally, in Fig 4 we confirm the structural, parametric and predictive accuracy of M*. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Immunity model. Structural unidentifiability in CM (unidentifiable parameters in red, identifiable parameters in blue) leads to the same output dynamics when different parameterizations are considered, as can be seen in CM2. In contrast, the reformulation M* is FISPO and therefore there is a unique set of parameters compatible with the output measurements. https://doi.org/10.1371/journal.pcbi.1011014.g003 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Immunity case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1 and x2) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g004 Stress response in bacteria (Bacterial) This model describes the stress response in Bacillus subtilis [59]. It was used by Mangan et al [31] to illustrate how an implicit SINDy approach was able to infer biological nonlinear dynamics. Under nutrient limitation, the majority of B. subtilis cells switch to sporulation, but a small fraction switch to an alternative behaviour, the so called state of competence, in which they are capable of taking up extracellular DNA. This latter fraction might subsequently return to vegetative growth. Süel et al [59] described the regulatory system of this mechanism using a dynamic model with two states. In dimensionless form, the ground truth (GT) model for this example is: (30a) (30b) where x1 and x2 represent the concentration levels of the ComK and ComS proteins. The rational terms arise from time-scale separation assumptions about the regulation: an autoregulatory positive feedback loop of ComK plus and indirect negative feedback loop mediated by ComS. In Eq 30a, a1 corresponds to the minimal rate of ComK production. The second term describes the autoregulation (via a positive feedback loop) of ComK activating its own production, where a2 is the fully activated rate of ComK generation. The first term in Eq 30b describes the negative feedback loop regulating the repression of ComS, where b1 is the maximum rate of ComS expression. Both the auto-activation of ComK and the repression of ComS follow Hill kinetics where the exponent indicates the level of cooperativity (2 and 5, respectively). The last term in both equations represents the degradation of both ComK and ComS. We again consider PM = GT and check the identifiability of PM. Considering unknown parameters a1, a2, a3, b1 and b2, the model is fully identifiable, thus PM* = GT. Next, SINDy-PI is applied to the training data generated using GT, obtaining the following candidate model (CM): (31a) (31b) This CM has 16 parameters, pj, j = 1, …, 16, and the SIO analysis reveals that all of them are non-identifiable, with the exception of p1. The reformulation step indicated that we can obtain an identifiable model with four scaling transformations, one per rational term, i.e. the second term in Eq 31a is scaled by pj, j ∈ [2, 3, 4], and the third term by pk, k ∈ [5, …, 9]. In Eq 31b the same strategy is applied for pl, l ∈ [10, 11, 12], and pm, m ∈ [13, …, 16]. That is: (32a) (32b) Choosing j = 4, k = 7, l = 11, m = 14, the resulting structurally identifiable model is: (33a) (33b) where * denotes a reparameterized parameter. This reformulated model is now fully identifiable, but no longer directly interpretable: Eq 33a does not explicitly have the term involving the autoregulation of ComK. By means of the reformulation procedure, we are able to recover the autoregulation and degradation terms as in Eq 30a: (34a) (34b) This reformulated model M* is structurally identifiable and interpretable, and equivalent to the PM. Fig 5 shows the assessment of the structural, parametric and predictive accuracy of the inferred model M*. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Bacterial case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1 and x2) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g005 Microbial growth (Microbial) This case study considers microbial growth in a batch reactor, as presented by [64] and later used by [60] to study identifiable reparameterizations of unidentifiable systems. The following model describes the dynamics of microbial and substrate concentrations assuming Monod kinetics (similar in functional form to Michaelis-Menten enzyme kinetics): (35a) (35b) where x1 and x2 represent the concentrations of microorganisms and growth-limiting substrate, respectively. The rational term in Eq 35a is the Monod kinetic term, where μ is the maximum growth velocity and Ks the substrate concentration corresponding to . In Eq 35a, the same rational term appears scaled by γ (the yield coefficient) to represent the depletion of substrate. The last term in Eq 35a describes the death of microorganisms assuming first order kinetics where Kd is the decay rate. We consider a prior model (PM) that matches the ground truth (GT) model, Eqs 35a and 35b. When the initial conditions are known and different from zero, our algorithm confirms that this PM is structurally identifiable. Next, our workflow discovers the following dynamics using SINDy-PI: (36a) (36b) Next, the FISPO step finds that only p1 is identifiable, i.e. parameters pi, i = 2, …, 7 are unidentifiable. The reformulation step finds that it is possible to find an identifiable form by scaling each rational term by the same unidentifiable parameter. For simplicity, we have chosen that , , and . Then, the resulting structurally identifiable model is: (37a) (37b) However, Eqs 37a and 37b are not directly interpretable because they do not contain the expected Monod kinetics terms explicitly. Next, the reformulation step finds an equivalent structure which is both interpretable and identifiable (M*): (38a) (38b) This inferred model (M*) is compared to the ground truth in Fig 6, confirming its structural, parametric and predictive accuracy. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Microbial case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. On the right, predictive accuracy: time evolution of the different states (x1 and x2) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g006 Cell cycle in the colonic crypt (Crypt) This example considers a cell population model describing the cell renewal cycle in the colonic crypt [61]. This cycle is heavily regulated and the model was used to explain the rupture of homeostasis and the initiation of tumorigenesis. The equations describing the dynamics are: (39a) (39b) (39c) where the state variables represent the populations of stem cells (x1), semi-differentiated cells (x2), and fully-differentiated cells (x3). Stem cells have first order kinetics for renewal (rate given parameter a3), differentiation (parameter a2), and death (parameter a1). Semi-differentiated cells have similar renewal, differentiation and death kinetics (with parameters bi), plus a source term due to the differentiation of stem cells. Fully differentiated cells are generated from semi-differentiated cells with first order rate b2 and removed with a rate modulated by parameter g. The rational terms correspond to saturating feedback mechanism in the differentiation rates. We take the above model as GT, and PM = GT. Our algorithm finds that this PM is not structurally identifiable: it is not possible to uniquely infer a1, a3, b1 and b3 due to the presence of a translation symmetry. Next, the reformulation step finds a reparameterized prior model (PM*): (40a) (40b) (40c) where and . Next, SINDy-PI is applied to the training data, obtaining the following candidate model (CM): (41a) (41b) (41c) Considering pi, i = 1, ‥, 21 as unknown parameters, the FISPO algorithm indicates that only p1, p2, p16, p17 and p21 are structurally identifiable. The reformulation step finds the following structurally identifiable alternative: (42a) (42b) (42c) The above model is not directly interpretable, but the reformulation process is able to find the following interpretable and identifiable reformulation M*: (43a) (43b) (43c) This discovered model M* is fully equivalent to the identifiable version of the ground truth model in terms of structural, parametric and predictive accuracy, as shown in Fig 7. This example reinforces the importance of checking the identifiability of both the ground truth and the inferred model. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Crypt case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1, x2 and x3) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g007 Oscillations in yeast glycolysis (Glycolysis) Glycolysis is the transformation (in a series of reactions catalyzed by enzymes) of glucose into smaller molecules to produce energy for the cell. In many cell types, glycolysis exhibits oscillations in the concentrations of many intermediate metabolites. This phenomena has been particularly well studied in yeast cells. Wolf and Heinrich [62] studied the oscillatory dynamics of a simplified reaction scheme for yeast glycolysis under anaerobic conditions, where alcoholic fermentation takes place, proposing the following mathematical description: (44a) (44b) (44c) (44d) (44e) (44f) (44g) where the state variables represent the concentrations in the cell of glucose (x1), the pool of triose phosphates (x2), 1,3-bisphosphoglycerate (x3), pool of pyruvate and acetaldehyde (x4), NADH (x5), ATP (x6), and x7 represents the pool of pyruvate and acetaldehyde in the external solution. We consider here the same formulation and parameter values as in [31, 37]. We take the above as GT, and assume PM = GT. Considering all parameters as unknown (ci, i = 1, 2, 3; di, i = 1, ‥, 4; ei, i = 1, …, 4; fi, i = 1, …, 5; gi, i = 1, 2; hi, i = 1, …, 5 and ji, i = 1, 2, 3), our algorithm confirms that the model is structurally identifiable and observable, i.e. PM* = PM. This problem is quite challenging for SINDy-PI due to its large number of states and parameters, and the large degree in several terms, leading to a very large library of candidate functions (over 3000 terms). However, it is able to correctly recover the following candidate model (CM): (45a) (45b) (45c) (45d) (45e) (45f) (45g) This model has 29 inferred coefficients, pi, i = 1, …, 29. Our algorithm analyzes their identifiability and finds that the parameters with indices i = 2, 3, 4, 7, 8, 9, 24, 25, 26 are unidentifiable. Next, the reformulation step finds possible reparameterizations by scaling the rational terms. That is, considering the first rational term scaled by p4, then and ; scaling the second term with p9 produces and ; and using p25 yields and for the last rational term. The end result is an interpretable and identifiable model M*: (46a) (46b) (46c) (46d) (46e) (46f) (46g) Fig 8 illustrates the excellent structural, parametric and predictive accuracy of the inferred model. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Yeast-Glycolysis case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Due to the large number of terms in ξ, the candidate functions are not shown. Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1, x2, x3, x4, x5, x6 and x7) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g008 Lorenz system (Lorenz) This case study involves the well-known Lorenz system [57], which is a classical example of dynamic model with chaotic behaviour. This model was previously used in [12] to demonstrate the original SINDy algorithm. The governing equations describe the dynamics of a fluid layer warmed from below and cooled from above: (21a) (21b) (21c) where x1 is proportional to the rate of convection, x2 to the horizontal temperature variation and x3 to the vertical one. For certain values of parameters a, b, and c, the system exhibits chaotic dynamics. We consider the ideal Scenario (II) case where the prior model (PM) is the same as the nominal (or ground truth, GT) model. We generate a synthetic training data set using the GT model and settings similar to [12] (details in Supporting Information). Following the workflow in Fig 1, we perform structural and identifiability analysis and confirm that the PM is fully identifiable and observable (FISPO). We then apply SINDy-PI to the training data, obtaining the following candidate model (CM): (22a) (22b) (22c) Our algorithm then confirms that this inferred CM model is FISPO and interpretable (thus, it corresponds to M*). Further, it is fully equivalent to the expanded ground truth model in terms of structural, parametric and predictive accuracy, as shown in Fig 2. In summary, in this case study we have the ideal situation where both the nominal and the inferred models are fully observable and identifiable. As we will see below, this situation might change as soon as we consider rational terms in the dynamics. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Lorenz case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1, x2 and x3) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g002 Competition between bacteria and the immune system (Immunity) This model describes the influence of quorum sensing signaling molecules (QSSM) on the competition between bacteria and the immune system, as studied in [58]. The following differential equations depict the dynamics for the concentrations of bacteria () and immune cells (): (23a) (23b) where it is assumed that bacteria grow logistically at rate a with an effective carrying capacity of the environment given by parameter k, and that they are cleared by the immune system following a mass action term ex1x2. The rational term at the end of Eq 23a represents the modulation of QSSM in the competition between bacteria and the immune system. We consider Eqs 23a and 23b as the GT model. We consider Scenario (II) again, assuming that the prior model (PM) is the same as the ground truth (GT) or nominal model. Considering that the unknown parameters are a, k, e, β, γ, α, S, d and δ, the structural identifiability analysis of the PM indicates that two of the three parameters involved in the rational term are unidentifiable. Specifically, there is a scaling symmetry between γ and α, which is probably the most common type of symmetry in biological models [63]. Our reparameterization step indicates that this issue can be solved by dividing the numerator and denominator by one of the unidentifiable parameters; for example, if we choose α, Eq 23a will be: (24) where γ/α = γ*. Thus our new reference model will be the following PM*: (25a) (25b) Next, our workflow proceeds by applying SINDy-PI to a data set generated with the GT model, obtaining the following candidate model (CM): (26a) (26b) Interestingly, our method then finds that this CM is not FISPO due to three structurally unidentifiable parameters: p4, p5, p6. The reformulation step is then able to find structurally identifiable reformulations of the form: (27) We chose j = 6, but the same result can be obtained with j = 4, 5. Denoting as , the resulting dynamic system becomes identifiable. Eq 27 is re-arranged as: (28) The resulting model is fully identifiable, but the rational term does not match the one in the GT describing the modulation of QSSM. However, the reformulation step in our workflow produces an interpretable form M* which is FISPO: (29a) (29b) Fig 3 illustrates the importance of ensuring structural identifiability and observability. The CM found by SINDy-PI is not FISPO, so there are other different parameter realizations producing exactly the same output, as shown by CM2. This means that if this CM structure is used for parameter identification, the estimated parameters will not be unique, i.e. there exist different parameterizations of CM in full agreement with the same output measurements. However, the reformulated model M* is FISPO, i.e. there is a unique set of parameter values compatible with the output. Finally, in Fig 4 we confirm the structural, parametric and predictive accuracy of M*. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Immunity model. Structural unidentifiability in CM (unidentifiable parameters in red, identifiable parameters in blue) leads to the same output dynamics when different parameterizations are considered, as can be seen in CM2. In contrast, the reformulation M* is FISPO and therefore there is a unique set of parameters compatible with the output measurements. https://doi.org/10.1371/journal.pcbi.1011014.g003 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Immunity case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1 and x2) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g004 Stress response in bacteria (Bacterial) This model describes the stress response in Bacillus subtilis [59]. It was used by Mangan et al [31] to illustrate how an implicit SINDy approach was able to infer biological nonlinear dynamics. Under nutrient limitation, the majority of B. subtilis cells switch to sporulation, but a small fraction switch to an alternative behaviour, the so called state of competence, in which they are capable of taking up extracellular DNA. This latter fraction might subsequently return to vegetative growth. Süel et al [59] described the regulatory system of this mechanism using a dynamic model with two states. In dimensionless form, the ground truth (GT) model for this example is: (30a) (30b) where x1 and x2 represent the concentration levels of the ComK and ComS proteins. The rational terms arise from time-scale separation assumptions about the regulation: an autoregulatory positive feedback loop of ComK plus and indirect negative feedback loop mediated by ComS. In Eq 30a, a1 corresponds to the minimal rate of ComK production. The second term describes the autoregulation (via a positive feedback loop) of ComK activating its own production, where a2 is the fully activated rate of ComK generation. The first term in Eq 30b describes the negative feedback loop regulating the repression of ComS, where b1 is the maximum rate of ComS expression. Both the auto-activation of ComK and the repression of ComS follow Hill kinetics where the exponent indicates the level of cooperativity (2 and 5, respectively). The last term in both equations represents the degradation of both ComK and ComS. We again consider PM = GT and check the identifiability of PM. Considering unknown parameters a1, a2, a3, b1 and b2, the model is fully identifiable, thus PM* = GT. Next, SINDy-PI is applied to the training data generated using GT, obtaining the following candidate model (CM): (31a) (31b) This CM has 16 parameters, pj, j = 1, …, 16, and the SIO analysis reveals that all of them are non-identifiable, with the exception of p1. The reformulation step indicated that we can obtain an identifiable model with four scaling transformations, one per rational term, i.e. the second term in Eq 31a is scaled by pj, j ∈ [2, 3, 4], and the third term by pk, k ∈ [5, …, 9]. In Eq 31b the same strategy is applied for pl, l ∈ [10, 11, 12], and pm, m ∈ [13, …, 16]. That is: (32a) (32b) Choosing j = 4, k = 7, l = 11, m = 14, the resulting structurally identifiable model is: (33a) (33b) where * denotes a reparameterized parameter. This reformulated model is now fully identifiable, but no longer directly interpretable: Eq 33a does not explicitly have the term involving the autoregulation of ComK. By means of the reformulation procedure, we are able to recover the autoregulation and degradation terms as in Eq 30a: (34a) (34b) This reformulated model M* is structurally identifiable and interpretable, and equivalent to the PM. Fig 5 shows the assessment of the structural, parametric and predictive accuracy of the inferred model M*. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Bacterial case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1 and x2) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g005 Microbial growth (Microbial) This case study considers microbial growth in a batch reactor, as presented by [64] and later used by [60] to study identifiable reparameterizations of unidentifiable systems. The following model describes the dynamics of microbial and substrate concentrations assuming Monod kinetics (similar in functional form to Michaelis-Menten enzyme kinetics): (35a) (35b) where x1 and x2 represent the concentrations of microorganisms and growth-limiting substrate, respectively. The rational term in Eq 35a is the Monod kinetic term, where μ is the maximum growth velocity and Ks the substrate concentration corresponding to . In Eq 35a, the same rational term appears scaled by γ (the yield coefficient) to represent the depletion of substrate. The last term in Eq 35a describes the death of microorganisms assuming first order kinetics where Kd is the decay rate. We consider a prior model (PM) that matches the ground truth (GT) model, Eqs 35a and 35b. When the initial conditions are known and different from zero, our algorithm confirms that this PM is structurally identifiable. Next, our workflow discovers the following dynamics using SINDy-PI: (36a) (36b) Next, the FISPO step finds that only p1 is identifiable, i.e. parameters pi, i = 2, …, 7 are unidentifiable. The reformulation step finds that it is possible to find an identifiable form by scaling each rational term by the same unidentifiable parameter. For simplicity, we have chosen that , , and . Then, the resulting structurally identifiable model is: (37a) (37b) However, Eqs 37a and 37b are not directly interpretable because they do not contain the expected Monod kinetics terms explicitly. Next, the reformulation step finds an equivalent structure which is both interpretable and identifiable (M*): (38a) (38b) This inferred model (M*) is compared to the ground truth in Fig 6, confirming its structural, parametric and predictive accuracy. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Microbial case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. On the right, predictive accuracy: time evolution of the different states (x1 and x2) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g006 Cell cycle in the colonic crypt (Crypt) This example considers a cell population model describing the cell renewal cycle in the colonic crypt [61]. This cycle is heavily regulated and the model was used to explain the rupture of homeostasis and the initiation of tumorigenesis. The equations describing the dynamics are: (39a) (39b) (39c) where the state variables represent the populations of stem cells (x1), semi-differentiated cells (x2), and fully-differentiated cells (x3). Stem cells have first order kinetics for renewal (rate given parameter a3), differentiation (parameter a2), and death (parameter a1). Semi-differentiated cells have similar renewal, differentiation and death kinetics (with parameters bi), plus a source term due to the differentiation of stem cells. Fully differentiated cells are generated from semi-differentiated cells with first order rate b2 and removed with a rate modulated by parameter g. The rational terms correspond to saturating feedback mechanism in the differentiation rates. We take the above model as GT, and PM = GT. Our algorithm finds that this PM is not structurally identifiable: it is not possible to uniquely infer a1, a3, b1 and b3 due to the presence of a translation symmetry. Next, the reformulation step finds a reparameterized prior model (PM*): (40a) (40b) (40c) where and . Next, SINDy-PI is applied to the training data, obtaining the following candidate model (CM): (41a) (41b) (41c) Considering pi, i = 1, ‥, 21 as unknown parameters, the FISPO algorithm indicates that only p1, p2, p16, p17 and p21 are structurally identifiable. The reformulation step finds the following structurally identifiable alternative: (42a) (42b) (42c) The above model is not directly interpretable, but the reformulation process is able to find the following interpretable and identifiable reformulation M*: (43a) (43b) (43c) This discovered model M* is fully equivalent to the identifiable version of the ground truth model in terms of structural, parametric and predictive accuracy, as shown in Fig 7. This example reinforces the importance of checking the identifiability of both the ground truth and the inferred model. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Crypt case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1, x2 and x3) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g007 Oscillations in yeast glycolysis (Glycolysis) Glycolysis is the transformation (in a series of reactions catalyzed by enzymes) of glucose into smaller molecules to produce energy for the cell. In many cell types, glycolysis exhibits oscillations in the concentrations of many intermediate metabolites. This phenomena has been particularly well studied in yeast cells. Wolf and Heinrich [62] studied the oscillatory dynamics of a simplified reaction scheme for yeast glycolysis under anaerobic conditions, where alcoholic fermentation takes place, proposing the following mathematical description: (44a) (44b) (44c) (44d) (44e) (44f) (44g) where the state variables represent the concentrations in the cell of glucose (x1), the pool of triose phosphates (x2), 1,3-bisphosphoglycerate (x3), pool of pyruvate and acetaldehyde (x4), NADH (x5), ATP (x6), and x7 represents the pool of pyruvate and acetaldehyde in the external solution. We consider here the same formulation and parameter values as in [31, 37]. We take the above as GT, and assume PM = GT. Considering all parameters as unknown (ci, i = 1, 2, 3; di, i = 1, ‥, 4; ei, i = 1, …, 4; fi, i = 1, …, 5; gi, i = 1, 2; hi, i = 1, …, 5 and ji, i = 1, 2, 3), our algorithm confirms that the model is structurally identifiable and observable, i.e. PM* = PM. This problem is quite challenging for SINDy-PI due to its large number of states and parameters, and the large degree in several terms, leading to a very large library of candidate functions (over 3000 terms). However, it is able to correctly recover the following candidate model (CM): (45a) (45b) (45c) (45d) (45e) (45f) (45g) This model has 29 inferred coefficients, pi, i = 1, …, 29. Our algorithm analyzes their identifiability and finds that the parameters with indices i = 2, 3, 4, 7, 8, 9, 24, 25, 26 are unidentifiable. Next, the reformulation step finds possible reparameterizations by scaling the rational terms. That is, considering the first rational term scaled by p4, then and ; scaling the second term with p9 produces and ; and using p25 yields and for the last rational term. The end result is an interpretable and identifiable model M*: (46a) (46b) (46c) (46d) (46e) (46f) (46g) Fig 8 illustrates the excellent structural, parametric and predictive accuracy of the inferred model. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Yeast-Glycolysis case study. Structural accuracy: on the left, active terms in ξ (non-zero terms of the prior model PM in black, and of the inferred model M* in green). Due to the large number of terms in ξ, the candidate functions are not shown. Parameter accuracy: center, matching parametric ODEs for PM and M*. Predictive accuracy: on the right, time evolution of the different states (x1, x2, x3, x4, x5, x6 and x7) of the PM and M* models. https://doi.org/10.1371/journal.pcbi.1011014.g008 Discussion In recent years, innovations in numerical methods and machine learning have been combined to improve our ability to understand complex systems. Currently, the three main classes of methods to learn equations from data are symbolic regression [65], neural-network approaches [13], and library-based sparse regression [12]. Recent reviews of these categories and their overlaps can be found in [66, 67]. In particular, data-driven model discovery methods for nonlinear dynamic systems have seen very significant advancements [22, 24]. The field has seen growth in terms of both sophistication and the range of applications. The fundamental aim remains the same: to discern the underlying mathematical models that govern the behaviour of complex, possibly high-dimensional and nonlinear systems, using measurement data. In this study we have investigated certain aspects of automatic model discovery techniques to derive mechanistic models of biological systems from time-series data. Specifically, we have focused on possible structural deficiencies of their end result, the inferred model equations. As a reference method we have chosen SINDy-PI [37], a recent sparse regression-based methodology that is particularly suited for computational biology due to its ability to capture complex nonlinearities and rational terms. However, it should be noted that our approach can be combined with other model discovery methods. In any case, SINDy-PI has several advantages over other model discovery methods. First, it is several orders of magnitude more robust to noise than previous approaches based on sparse regression. This means that it can learn implicit ordinary and partial differential equations and conservation laws from limited and noisy data. Second, it can discover models with very complex structure, including implicit dynamics and rational nonlinearities (such as e.g. Michaelis-Menten kinetics), which are common in biological applications. Third, it is still quite computationally efficient thanks to its parallel nature and the exploitation of a library of canonical nonlinear terms. Such a library is particularly attractive when modelling the dynamics of biological networks based on mechanistic assumptions, such as mass-action kinetics. Since by design SINDy-PI enforces parsimonious models (with the lowest complexity to support the data), it usually produces interpretable equations with excellent predictive power. However, we have shown that sometimes these models lack structural identifiability, which means that using the discovered model structure for parameter estimation might give wrong estimates, compromising its usefulness and reliability. To address this issue we have presented a methodology that, combined with SINDy-PI, facilitates the inference of identifiable and interpretable dynamic models. Our method integrates symbolic algorithms that analyse a model’s structural identifiability and observability (SIO), reparameterize it to achieve SIO if needed, and reformulate it to make it biologically interpretable. We have illustrated its use in two scenarios, with and without prior knowledge, using six challenging case studies corresponding to different kinds of biological systems, including complex regulatory mechanisms. Our results highlight additional challenges due to non-obvious issues in the relationship between model reformulation, identifiability and interpretability, and show how our approach is able to successfully surmount them. Importantly, our method is modular and can be easily integrated with other model discovery strategies. While we have demonstrated its application in combination with SINDy-PI, other methods could have been used as well. Furthermore, its calculations are entirely symbolic, i.e. they are not affected by numerical issues caused by insufficient or noisy data (which do however limit the application of the accompanying model discovery method). Future work will be devoted to model discovery in partially observed systems, where the structural identifiability problem will surely be exacerbated, and observability issues—i.e. the impossibility of inferring some of the unmeasured state variables—are to be expected. It should be noted that, as a matter of fact, our methodology is applicable to partially observed systems in its present form. However, model discovery for such systems is still in its infancy (see the recent work by [68]), hence in this study we have considered fully observed systems. Another possible area of improvement is computational efficiency. While our pipeline can be applied to systems with several states and a few dozen parameters, as demonstrated with the Glycolysis example, scaling up to larger models is challenging. The main bottleneck is currently the model reparameterization step performed with AutoRepar, which involves symbolic computations that can be very memory-consuming. We are working on improving the efficiency of the algorithms in order to alleviate its computational cost. Other important avenues of research which are currently being explored include (i) improved approaches for the design of the library of candidate functions [69], (ii) better incorporation of partial prior knowledge [70], and (iii) taking into account noisy and missing data, uncertainty quantification and applications to real-world data-sets [71–73]. Since identifiability and observability play a major role in these scenarios, we believe that our methodology will be a useful tool in these explorations. Supporting information S1 File. Supporting material document. https://doi.org/10.1371/journal.pcbi.1011014.s001 (PDF)

Nayebi, Aran;Kong, Nathan C. L.;Zhuang, Chengxu;Gardner, Justin L.;Norcia, Anthony M.;Yamins, Daniel L. K.

doi: 10.1371/journal.pcbi.1011506pmid: 37782673

Introduction The mouse has become an indispensable model organism in systems neuroscience, allowing unprecedented genetic and experimental control at the level of cell-type specificity in individual circuits [1]. Moreover, studies of mouse visual behavior have revealed a multitude of abilities, ranging from stimulus-reward associations, to goal-directed navigation, and object-centric discriminations [2]. Thus, the mouse animal model bridges fine-grained, low-level experimental control with high-level behavior. Understanding the principles underlying the structure and function of the mouse visual system and how it relates to more commonly studied visual organisms such as macaque, is therefore important. Computational models are an effective tool for understanding how mouse visual cortex is capable of supporting such behaviors and for providing a normative account for its structure and function. They allow us to identify the key ingredients leading to the model with the best quantitative agreement with the neural data. We can also assess the functional similarities and differences between rodent and primate visual cortex, which otherwise would be hard to capture in the absence of a model from prior literature, beyond failures to be homologous in higher visual areas past V1. Furthermore, these models provide a natural starting point for understanding higher-level processing downstream of the visual system, such as in memory and its role during the navigation of rich visual environments [3–7]. Without an explicit model of a visual system, it is difficult to disentangle the contributions to neural response variance of the visual system from those of higher-level cognitive phenomena. Understanding the computations underlying higher cognition and motor control in rodents will therefore critically depend on an understanding of the upstream sensory areas that they depend on. Deep convolutional neural networks (CNNs) are a class of models that have had success as predictive models of the human and non-human primate ventral visual stream (e.g., [8–13]). In contrast, such models have been poor predictors of neural responses in mouse visual cortex [14, 15]. Our hypothesis is that this failure can be understood via and be remedied by the goal-driven modeling approach [16]. This approach posits that normative models in neuroscience should pay careful attention to the objective functions (i.e., behavior), architectures (i.e., neural circuit), and data stream (i.e., visual input). These structural and functional ingredients should be finely tuned to the biology and the ecology of the organism under study. In this work, we build a substantially improved model of mouse visual cortex by better aligning the objective function, architecture, and visual input with those of the mouse visual system. Firstly, from an objective function point of view, the primate ventral stream models are trained in a supervised manner on ImageNet [17, 18], which is an image set containing over one million images belonging to one thousand, mostly human-relevant, semantic categories [19]. While such a dataset is an important technical tool for machine learning, it is highly implausible as a biological model particularly for rodents, who do not receive such category labels over development. Instead, we find that models trained using self-supervised, contrastive algorithms provide the best correspondence to mouse visual responses. Interestingly, this situation is different than in primates, where prior worked showed that the two were roughly equivalent [20]. Secondly, in terms of the architecture, these primate ventral visual stream models are too deep to be plausible models of the mouse visual system, since mouse visual cortex is known to be more parallel and much shallower than primate visual cortex [21–24]. By varying the number of linear-nonlinear layers in the models, we find that models with fewer linear-nonlinear layers can achieve neural predictivity performance that is better or on par with very deep models. Finally, mice are known to have lower visual acuity than that of primates [25, 26], suggesting that the resolution of the inputs to mouse models should be lower than that of the inputs to primate models. Indeed, we find that model fidelity can be improved by training them on lower-resolution images. Ultimately, the confluence of these ingredients, leads to a model, known as “Contrastive AlexNet” (first four layers), that best matches mouse visual cortex thus far. We then address the question of why the Contrastive AlexNet is better at neural predictivity from an ecological point of view, especially the role of contrastive, self-supervised learning, which is novel and not expected by the known physiology and behavioral experiments in mouse. To address this question, we use Contrastive AlexNet to assess out-of-distribution generalization from its original training environment, including using the visual encoder as the front-end of a bio-mechanically realistic virtual rodent operating in an environment that supports spatially-extended reward-based navigation. We show that visual representations of this model lead to improved transfer performance over its supervised counterpart across environments, illustrating the congruence between the task-transfer performance and improved neural fidelity of the computational model. Taken together, our best models of the mouse visual system suggest that it is a shallower, general-purpose system operating on comparatively low-resolution inputs. These identified factors therefore provide interpretable insight into the confluence of constraints that may have given rise to the system in the first place, suggesting that these factors were crucially important given the ecological niche in which the mouse is situated, and the resource limitations to which it is subject. Results Determining the animal-to-animal mapping transform Prior models of mouse visual cortex can be improved by varying three ingredients to better match the biology and the ecology of mouse visual cortex. Before model development, however, we must determine the appropriate procedure by which to evaluate models. As in prior work on modeling primate visual cortex, we “map” model responses to biological responses and the ability of the model responses to recapitulate biological responses determines the model’s neural fidelity [8, 9, 17, 20]. How should artificial neural network responses be mapped to biological neural responses? What firing patterns of mouse visual areas are common across multiple animals, and thus worthy of computational explanation? A natural approach would be to map artificial neural network features to mouse neural responses in the same manner that different animals can be mapped to each other. Specifically, we aimed to identify the best performing class of similarity transforms needed to map the firing patterns of one animal’s neural population to that of another, which we denote as the “inter-animal consistency”. We took inspiration from methods that have proven useful in modeling human and non-human primate visual, auditory, and motor cortices [16, 27–29]. As with other cortical areas, this transform class likely cannot be so strict as to require fixed neuron-to-neuron mappings between cells. However, the transform class for each visual area also cannot be so loose as to allow an unconstrained nonlinear mapping, since the model already yields an image-computable nonlinear response. We explored a variety of linear mapping transform classes (fit with different constraints) between the population responses for each mouse visual area (Fig 1A). The mouse visual responses to natural scenes were collected previously using both two-photon calcium imaging and Neuropixels by the Allen Institute [15, 22] from areas V1 (VISp), LM (VISl), AL (VISal), RL (VISrl), AM (VISam), and PM (VISpm) in mouse visual cortex (see number of units and specimens for each dataset in Table 1 and further details in the “Neural Response Datasets” section). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Inter-animal neural response consistency across mouse visual areas. A. Inter-animal consistency was computed using different linear maps, showing that PLS regression provides the highest consistency. Horizontal bars at the top are the median and s.e.m. of the internal consistencies of the neurons in each visual area. Refer to Table 1 for N units per visual area. B. The fraction of maximum split-half reliability is plotted as a function of time (in 10-ms time bins) for each visual area. https://doi.org/10.1371/journal.pcbi.1011506.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Descriptive statistics of the neural datasets. Total number of units and specimens for each visual area for the calcium imaging and Neuropixels datasets. https://doi.org/10.1371/journal.pcbi.1011506.t001 We focused on the natural scene stimuli, consisting of 118 images, each presented 50 times (i.e., 50 trials per image). For all methods, the corresponding mapping was trained on 50% of all the natural scene images, and evaluated on the remaining held-out set of images. We also included representational similarity analyses (RSA; [30]) as a baseline measure of population-wide similarity across animals, corresponding to no selection of individual units, unlike the other mapping transforms. For the strictest mapping transform (One-to-One), each target unit was mapped to the single most correlated unit in the source animal. Overall, the One-to-One mapping tended to yield the lowest inter-animal consistency among the maps considered. However, Ridge regression (L2-regularized) and Partial Least Squares (PLS) regression were more effective at the inter-animal mapping, yielding the most consistent fits across visual areas, with PLS regression providing the highest inter-animal consistency. We therefore used PLS regression in the evaluation of a candidate model’s ability to predict neural responses. This mapping transform confers the additional benefit of enabling direct comparison to prior primate ventral stream results (which also used this mapping [8, 17]) in order to better understand ecological differences between the two visual systems across species. Under all mapping transforms, a log-linear extrapolation analysis (S5 Fig) reveals that as the number of units increases, the inter-animal consistency approaches 1.0 more rapidly for the Neuropixels dataset, than the calcium imaging dataset, which were obtained from the average of the Δf/F trace—indicating the higher reliability of the Neuropixels data across all visual areas. We further noticed a large difference between the inter-animal consistency obtained via RSA and the consistencies achieved by any of the other mapping transforms for the responses in RL of the calcium imaging dataset (green in Fig 1A). This difference, however, was not observed for responses in RL in the Neuropixels dataset. This discrepancy suggested that there was a high degree of population-level heterogeneity in the responses collected from the calcium imaging dataset, which may be attributed to the fact that the two-photon field-of-view for RL spanned the boundary between the visual and somatosensory cortex, as originally noted by de Vries et al. [15]. We therefore excluded RL in the calcium imaging dataset from further analyses, following Siegle et al. [31], who systematically compared these two datasets. Thus, this analysis provided insight into the experiments from which the data were collected, and allowed us to ascertain the level of neural response variance that is common across animals and that therefore should be “explained” by candidate models. For these reasons above, we present our main results on the newer and more reliable Neuropixels dataset, as it is in a better position to separate models—with similar results on the calcium imaging dataset presented in the Supporting Information. Modeling mouse visual cortex Building quantitatively accurate models. With this mapping and evaluation procedure, we can then develop models to better match mouse visual responses. The overall conclusion that the mouse visual system is most consistent with an artificial neural network model that is self-supervised, low-resolution, and comparatively shallow. This conclusion holds more generally as well on the earlier calcium imaging dataset, along with non-regression-based comparisons like RSA (cf. S2, S3 and S4 Figs). Our best models attained neural predictivity of 90% of the inter-animal consistency, much better than the prior high-resolution, deep, and task-specific model (VGG16), which attained 56.27% of this ceiling (Fig 2A; cf. rightmost column of Table 2 for the unnormalized quantities). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Substantially improving neural response predictivity of models of mouse visual cortex. A. The median and s.e.m. (across units) neural predictivity difference with the prior-used, primate model of Supervised VGG16 trained on 224 px inputs (“Primate Model Baseline”, used in [14, 15, 36]), under PLS regression, across units in all mouse visual areas (N = 1731 units in total). Absolute neural predictivity (always computed from the model layer that best predicts a given visual area) for each model can be found in Table 2. Our best model is denoted as “AlexNet (IR)” on the far left. “Single Stream”, “Dual Stream”, and “Six Stream” are novel architectures we developed based on the first four layers of AlexNet, but additionally incorporates dense skip connections, known from the feedforward connectivity of the mouse connectome [34, 35], as well as multiple parallel streams (schematized in S1 Fig). CPC denotes contrastive predictive coding [32, 37]. All models, except for the “Primate Model Baseline”, are trained on 64 px inputs. We also note that all the models are trained using ImageNet, except for CPC (purple), Depth Prediction (orange), and the CIFAR-10-labeled black bars. B. Training a model on a contrastive objective improves neural predictivity across all visual areas. For each visual area, the neural predictivity values are plotted across all model layers for an untrained AlexNet, Supervised (ImageNet) AlexNet and Contrastive AlexNet (ImageNet, instance recognition)—the latter’s first four layers form the best model of mouse visual cortex. Shaded regions denote mean and s.e.m. across units. https://doi.org/10.1371/journal.pcbi.1011506.g002 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. ImageNet top-1 validation set accuracy via linear transfer or via supervised training and neural predictivity for each model. We summarize here the top-1 accuracy for each self-supervised and supervised model on ImageNet as well as their noise-corrected neural predictivity obtained via the PLS map (aggregated across all visual areas). These values are plotted in Fig 2C and S2 Fig. Unless otherwise stated, each model is trained and validated on 64 × 64 pixels images. https://doi.org/10.1371/journal.pcbi.1011506.t002 We also attain neural predictivity improvements over prior work [32, 33] (cf. the purple and green bars in Fig 2A), especially the latter “MouseNet” of Shi et al. [33], which attempts to map details of the mouse connectome [34, 35] onto a CNN architecture. There are two green bars in Fig 2A since we also built our own variant of MouseNet where everything is the same except that image categories are read off of at the penultimate layer of the model (rather than the concatenation of the earlier layers as originally proposed). We thought this might aid the original MouseNet’s task performance and neural predictivity, since it can be difficult to train linear layers when the input dimensionality is very large. Our best models also outperform the neural predictivity of “MouseNet” even when it is trained with a self-supervised objective (leftmost red vs. “MouseNet” teal bars in Fig 2A). This is another conceptual motivation for our structural-and-functional goal-driven approach, as the higher-level constraints are easier to interrogate than to incorporate and assume individual biological details, as this can be a very under constrained procedure. In the subsequent subsections, we distill the three factors that contributed to models with improved correspondence to the mouse visual areas: objective function, input resolution, and architecture. Objective function: Training models on self-supervised, contrastive objectives, instead of supervised objectives, improves correspondence to mouse visual areas. The success in modeling the human and the non-human primate visual system has largely been driven by convolutional neural networks trained in a supervised manner on ImageNet [19] to perform object categorization [9, 13]. This suggests that models trained with category-label supervision learn useful visual representations that are well-matched to those of the primate ventral visual stream [17, 20]. Thus, although biologically implausible for the rodent, category-label supervision is a useful starting point for building baseline models and indeed, models trained in this manner are much improved over the prior primate model of VGG16 (black bars in Fig 2A). We refer to this model as the “Primate Model Baseline”, since although many different models have been used to predict neural activity in the primate ventral stream, this model in particular was the de facto CNN used in the initial goal-driven modeling studies of mouse visual cortex [14, 15, 36]. This choice also helps explicitly illustrate how the three factors that we study, which deviate in visual acuity, model depth, and functional objective from the primate ventral stream, quantitatively improve these models greatly. These improved, category-label-supervised models, however, cannot be an explanation for how the mouse visual system developed in the first place. In particular, it is unclear whether or not rodents can perform well on large-scale object recognition tasks when trained, such as tasks where there are hundreds of labels. For example, they obtain approximately 70% on a two-alternative forced-choice object classification task [38]. Furthermore, the categories of the ImageNet dataset are human-centric and therefore not relevant for rodents In fact, the affordances of primates involve being able to manipulate objects flexibly with their hands (unlike rodents). Therefore, the ImageNet categories may be more relevant to non-human primates as an ethological proxy than it is to rodents. We therefore turned to more ecological supervision signals and self-supervised objective functions, where high-level category labeling is not necessary. These objectives could possibly lead to models with improved biological plausibility and may provide more general goals for the models, based on natural image statistics, beyond the semantic specifics of (human-centric) object categorization. Among the more ecological supervised signals, we consider categorization of comparatively lower-variation and lower-resolution images with fewer labels (CIFAR-10-labeled black bars in Fig 2A; [39]), and depth prediction (orange bars in Fig 2A; [40]), as a visual proxy for whisking [41]. Here, we note that in Fig 2A, all models except for those indicated by CIFAR-10 (black), CPC (purple), or Depth Prediction (orange), are trained on ImageNet images. Therefore, even if we remove the category labels from training, many of the images themselves (mainly from both ImageNet and CIFAR-10) are human-centric so future models could be trained using images that are more plausible for mice. Turning to self-supervision, early self-supervised objectives include sparse autoencoding (pink bars in Fig 2A; [42]), instantiated as a sparsity penalty in the latent space of the image reconstruction loss, which has been shown to be successful at producing Gabor-like functions reminiscent of the experimental findings in the work of Hubel and Wiesel [43]. These objectives, however, have not led to quantitatively accurate models of higher visual cortex [8, 20]. Further developments in computer vision have led to other self-supervised algorithms, motivated by the idea that “non-semantic” features are highly related to the higher-level, semantic features (i.e., category labels), such as predicting the rotation angle of an image (blue bars in Fig 2A; [44]). Although these objectives are very simple, models optimized on them do not result in “powerful” visual representations for downstream tasks. We trained models on these objectives and showed that although they improve neural predictivity over the prior primate model (VGG16), they do not outperform category-label-supervised models (Fig 2A; compare pink, blue, and orange with black). Further developments in self-supervised learning provided a new class of contrastive objectives. These objectives are much more powerful than prior self-supervised objectives described above, as it has been shown that models trained with contrastive objectives leads to visual representations that can support strong performance on downstream object categorization tasks. At a high level, the goal of these contrastive objectives is to learn a representational space where embeddings of augmentations for one image (i.e., embeddings for two transformations of the same image) are more “similar” to each other than to embeddings of other images. We trained models using the family of contrastive objective functions including: Instance Recognition (IR; [45]), a Simple Framework for Contrastive Learning (SimCLR; [46]), Momentum Contrast (MoCov2; [47]), Simple Siamese representation learning (SimSiam; [48]), Barlow Twins [49], and Variance-Invariance-Covariance Regularization (VICReg; [50]). Note that we are using the term “contrastive” broadly to encompass methods that learn embeddings which are robust to augmentations, even if they do not explicitly rely on negative batch examples—as they have to contrast against something to avoid representational collapse. For example, SimSiam relies on asymmetric representations via a stop gradient; Barlow Twins relies on regularizing with the cross correlation matrix’s off diagonal elements; and VICReg uses the variance and covariance of each embedding to ensure samples in the batch are different. Models trained with these contrastive objectives (red bars in Fig 2A) resulted in higher neural predictivity across all the visual areas than models trained on supervised object categorization, depth prediction, and less-powerful self-supervised algorithms (black, orange, purple, pink, and blue bars in Fig 2A). We hone in on the contribution of the contrastive objective function (over a supervised objective, and with a fixed dataset of ImageNet) to neural predictivity by fixing the architecture to be AlexNet, while varying the objective function (shown in S9 Fig left). We find that across all objective functions, training AlexNet using instance recognition leads to the highest neural predictivity. We additionally note that the improvement in neural predictivity extends beyond the augmentation used for each objective function, as shown in S7 Fig. When the image augmentations intended for contrastive losses is used with supervised losses, neural predictivity does not improve. Across all the visual areas, there is an improvement in neural predictivity simply by using a powerful contrastive algorithm (red vs. black in Fig 2B and S9 Fig left). Not only is there an improvement in neural predictivity, but also an improvement in hierarchical correspondence to the mouse visual hierarchy. Using the brain hierarchy score developed by Nonaka et al. [51], we observe that contrastive models outperform their supervised counterparts in matching the mouse visual hierarchy (S10 Fig). The first four layers of Contrastive AlexNet (red; Fig 2B), where neural predictivity is maximal across visual areas, forms our best model for mouse visual cortex. Data stream: Training models on images of lower resolution improves correspondence to mouse visual areas. The visual acuity of mice is known to be lower than the visual acuity of primates [25, 26]. Thus, more accurate models of mouse visual cortex must be trained and evaluated at image resolutions that are lower than those used in the training of models of the primate visual system. We investigated how neural predictivity of two strong contrastive models varied as a function of the image resolution at which they were trained. Two models were used in the exploration of image resolution’s effects on neural predictivity. Contrastive AlexNet was trained with image resolutions that varied from 64 × 64 pixels to 224 × 224 pixels, as 64 × 64 pixels was the minimum image size for AlexNet due to its architecture. The image resolution upper bound of 224 × 224 pixels is the image resolution that is typically used to train neural network models of the primate ventral visual stream [17]. We also investigated image resolution’s effects using a novel model architecture we developed, known as “Contrastive StreamNet”, as its architecture enables us to explore a lower range of image resolutions than the original AlexNet. This model was based on the first four layers of AlexNet, but additionally incorporates dense skip connections, known from the feedforward connectivity of the mouse connectome [34, 35], as well as multiple parallel streams (schematized in S1 Fig). We trained it using a contrastive objective function (instance recognition) at image resolutions that varied from 32 × 32 pixels to 224 × 224 pixels. Training models using resolutions lower than what is used for models of primate visual cortex improves neural predictivity across all visual areas, but not beyond a certain resolution, where neural predictivity decreases (Fig 3). Although the input resolution of 64 × 64 pixels may not be optimal for every architecture, it was the resolution that we used to train all the models. This was motivated by the observation that the upper bound on mouse visual acuity is 0.5 cycles / degree [25], corresponding to 2 pixels / cycle × 0.5 cycles / degree = 1 pixel / degree. Prior retinotopic map studies [52] estimate a visual coverage range in V1 of 60–90 degrees, and we found 64 × 64 pixels to be roughly optimal for the models (Fig 3) and was also used by Shi et al. [36], by Bakhtiari et al. [32], and in the MouseNet of Shi et al. [33]. Although downsampling training images is a reasonable proxy for the mouse retina, as has been done in prior modeling work, more investigation into appropriate image transformations may be needed. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Lower-resolution training leads to improved neural response predictivity (Neuropixels dataset). Contrastive AlexNet and Contrastive StreamNet (Dual Stream) were trained using images of increasing resolutions. For AlexNet, the minimum resolution was 64 × 64 pixels. For Dual StreamNet, the minimum resolution was 32 × 32 pixels. The median and s.e.m. neural predictivity across all units of all visual areas is plotted against the image resolution at which the models were trained. Reducing the image resolution (but not beyond a certain point) during representation learning improves the match to all the visual areas. The conversion from image resolution in pixels to visual field coverage in degrees is under the assumption that the upper bound on mouse visual acuity is 0.5 cycles per degree [25] and that the Nyquist limit is 2 pixels per cycle, leading to a conversion ratio of 1 pixel per degree. https://doi.org/10.1371/journal.pcbi.1011506.g003 Overall, these data show that optimization using a simple change in the image statistics (i.e., data stream) is crucial to obtain improved models of mouse visual encoding. This suggests that mouse visual encoding is the result of “task-optimization” at a lower resolution than what is typically used for primate ventral stream models. Architecture: Shallow models suffice for improving correspondence to mouse visual cortex. Anatomically, the mouse visual system has a shallower hierarchy relative to that of the primate visual system (see, e.g., [21–24]). Furthermore, the reliability of the neural responses for each visual area provides additional support for the relatively shallow functional hierarchy (Fig 1B). Thus, more biologically-plausible models of mouse visual cortex should have fewer linear-nonlinear layers than those of primate visual cortex. By plotting a model’s neural predictivity against its number of linear-nonlinear operations, we indeed found that very deep models do not outperform shallow models (i.e., models with less than 12 linear-nonlinear operations), despite changes across loss functions and input resolutions (Fig 4). In addition, if we fix the objective function to be “instance recognition”, we can clearly observe that AlexNet, which consists of eight linear-nonlinear layers, has the highest neural predictivity compared to ResNets and VGG16 (S9 Fig right). Furthermore, models with only four convolutional layers (StreamNets; Single, Dual, or Six Streams) perform as well as or better than models with many more convolutional layers (e.g., compare Dual Stream with ResNet101, ResNet152, VGG16, or MouseNets in S9 Fig right). This observation is in direct contrast with prior observations in the primate visual system whereby networks with fewer than 18 linear-nonlinear operations predict macaque visual responses less well than models with at least 50 linear-nonlinear operations [17]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Neural predictivity vs. model depth. A model’s median neural predictivity across all units from each visual area is plotted against its depth (number of linear-nonlinear layers; in log-scale). Models with fewer linear-nonlinear layers can achieve neural predictivity performances that outperform or are on par with those of models with many more linear-nonlinear layers. The “Primate Model Baseline” denotes a supervised VGG16 trained on 224 px inputs, used in prior work [14, 15, 36]. https://doi.org/10.1371/journal.pcbi.1011506.g004 Mouse visual cortex as a general-purpose visual system Our results show that a model optimized on a self-supervised contrastive objective the most quantitatively accurate model of the mouse visual system. However, unlike supervised object categorization or other more-classical forms of self-supervised learning such as sparse autoencoding, whose ecological function is directly encoded in the loss function (e.g., predator recognition or metabolically-efficient dimension reduction), the functional utility of self-supervision via a contrastive objective is not as apparent. This naturally raises the question of what behavioral function(s) optimizing a contrastive self-supervision objective might enable for the mouse from an ecological fitness viewpoint. Analyzing the spectrum of models described in the above section, we first observed that performance on ImageNet categorization is uncorrelated with improved neural predictivity for mouse visual cortex (Fig 5; cf. middle column of Table 2 for the exact ImageNet performance values), unlike the well-known correlation in primates (Fig 5; inset). In seeking an interpretation of the biological function of the contrastive self-supervised objective, we were thus prompted to consider behaviors beyond object-centric categorization tasks. We hypothesized that since self-supervised loss functions are typically most effective in task-agnostic stimulus domains where goal-specific labels are unavailable, optimizing for such an objective might enable the rodent to transfer well to complex multi-faceted ecological tasks in novel domains affording few targeted opportunities for hyper-specialization. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Neural predictivity is not correlated with object categorization performance on ImageNet. Each model’s (transfer or supervised) object categorization performance on ImageNet is plotted against its median neural predictivity across all units from all visual areas. All ImageNet performance values can be found in Table 2. Inset. Primate ventral visual stream neural predictivity from BrainScore is correlated with ImageNet categorization accuracy (adapted from Schrimpf et al. [17]). This relationship is in stark contrast to our finding in mouse visual cortex where higher ImageNet categorization accuracy is not associated with higher neural predictivity. Color scheme as in Fig 2A. See S2 Fig. for neural predictivity on the calcium imaging dataset. https://doi.org/10.1371/journal.pcbi.1011506.g005 To test this hypothesis, we used a recently-developed “virtual rodent” framework (adapted from Merel et al. [53] and also used by Lindsay et al. [54]), in which a biomechanically-validated mouse model is placed in a simulated 3D maze-like environment (Fig 6A). The primary purpose of the experiments with the virtual rodent are not to necessarily make a specific statement about rodent movement repertoires, but mainly to how well our self-supervised visual encoder enables control of a high-dimensional body with high-dimensional continuous inputs—a problem that many (if not all) animals have to solve. Of course, given that we were trying to better understand why self-supervised methods are better predictors of specifically mouse visual cortical neurons, we wanted a reasonable ecological task for that species (e.g., navigation) and that its affordances were somewhat similar to that of an actual rodent via the biomechanical realism of its body. In particular, if we used our shallow, lower acuity, self-supervised visual encoder for controlling a virtual monkey simulation for a task that a monkey is adapted to (e.g., object manipulation), we would not expect this to work as well, given that such a task likely requires high visual acuity and good object recognition abilities at a minimum. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Evaluating the generality of learned visual representations. A. Each row shows an example of an episode used to train the reinforcement learning (RL) policy in an offline fashion [56]. The episodes used to train the virtual rodent (adapted from Merel et al. [53]) were previously generated and are part of a larger suite of RL tasks for benchmarking algorithms [57]. In this task (“DM Locomotion Rodent”), the goal of the agent is to navigate the maze to collect as many rewards as possible (blue orbs shown in the first row). B. A schematic of the RL agent. The egocentric visual input is fed into the visual backbone of the model, which is fixed to be the first four convolutional layers of either the contrastive or the supervised variants of AlexNet. The output of the visual encoder is then concatenated with the virtual rodent’s proprioceptive inputs before being fed into the (recurrent) critic and policy heads. The parameters of the visual encoder are not trained, while the parameters of the critic head and the policy head are trained. Virtual rodent schematic from Fig 1B of Merel et al. [53]. C. A schematic of the out-of-distribution generalization procedure. Visual encoders are either trained in a supervised or self-supervised manner on ImageNet [19] or the Maze environment [57], and then evaluated on reward-based navigation or datasets consisting of object properties (category, pose, position, and size) from Hong et al. [55], and different textures [58]. https://doi.org/10.1371/journal.pcbi.1011506.g006 We used the model that best corresponds to mouse visual responses as the visual system of the simulated mouse, coupled to a simple actor-critic reinforcement-learning architecture (Fig 6B). We then trained the simulated mouse in several visual contexts with varying objective functions, and evaluated those models’ ability to transfer to a variety of tasks in novel environments, including both a reward-based navigation task, as well as several object-centric visual categorization and estimation tasks (Fig 6C). We first evaluated a simulated mouse whose visual system was pretrained with ImageNet images in terms of its ability to transfer to reward-based navigation in the Maze environment, training just the reinforcement learning portion of the network on the navigation task with the pretrained visual system held constant. As a supervised control, we performed the same transfer training procedure using a visual front-end created using supervised (ImageNet) object categorization pretraining. We found that the simulated mouse with the contrastive self-supervised visual representation was able to reliably obtain substantially higher navigation rewards than its counterpart with category-supervised visual representation (Fig 7A; cf. Table 3 for the exact quantities of rewards attained). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Self-supervised, contrastive visual representations better support transfer performance on downstream, out-of-distribution tasks. Models trained in a contrastive manner (using either ImageNet or the egocentric maze inputs; red and blue respectively) lead to better transfer on out-of-distribution downstream tasks than models trained in a supervised manner (i.e., models supervised on labels or on rewards; black and purple respectively). A. Models trained on ImageNet, tested on reward-based navigation. B. Models trained on egocentric maze inputs (“Contrastive Maze”, blue) or supervised on rewards (i.e., reward-based navigation; “Supervised Maze”, purple), tested on visual scene understanding tasks: pose, position, and size estimation, and object and texture classification. C. Median and s.e.m. neural predictivity across units in the Neuropixels dataset. https://doi.org/10.1371/journal.pcbi.1011506.g007 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Average reward and s.e.m. across 600 episodes obtained by the RL agent using each of the visual backbones. https://doi.org/10.1371/journal.pcbi.1011506.t003 Conversely, we trained the simulated mouse directly in the Maze environment. In one variant, we trained the visual system on the contrastive self-supervised objective (but on the images of the Maze environment). In a second variant, we trained the agent end-to-end on the reward navigation task itself—the equivalent of “supervision” in the Maze environment. We then tested both models on their ability to transfer to out-of-sample visual categorization and estimation tasks. Again, we found that the self-supervised variant transfers significantly better than its “supervised” counterpart (blue vs. purple bars in Fig 7B). Returning to analyses of neural predictivity, we found that both self-supervised models (trained in either environment) were better matches to mouse visual cortex neurons than the supervised counterparts in their respective environments (red and blue vs. black and purple in Fig 7C). This result illustrates the congruence between general out-of-distribution task-transfer performance and improved neural fidelity of the computational model. Furthermore, Lindsay et al. [54] found that less powerful self-supervised representation learners such as CPC and autoencoding did not match mouse visual responses as well as their RL-trained counterpart (in terms of representational similarity). This is consistent with our finding that CPC and autoencoding in Fig 3A themselves do not match neural responses as well as contrastive self-supervised methods (red vs. pink and purple bars). It is also noteworthy that, comparing models by training environment rather than objective function, the ImageNet-trained models produce more neurally-consistent models than the Maze-trained models, both for supervised and self-supervised objectives (red vs. blue and black vs. purple in Fig 7C). Using the contrastive self-supervised objective function is by itself enough to raise the Maze-trained model to the predictivity level of the ImageNet-trained supervised model, but the training environment makes a significant contribution. This suggests that while the task domain and biomechanical model of the Maze environment are realistic, future work will likely need to improve the realism of the simulated image distribution. Overall, these results suggest that contrastive embedding methods have achieved a generalized improvement in the quality of the visual representations they create, enabling a diverse range of visual behaviors, providing evidence for their potential as computational models of mouse visual cortex. While we expect the calculation of rewards to be performed outside of the visual system, we expect the mouse visual system would support the visual aspects of the transfer tasks we consider, as we find that higher model areas best support these scene understanding transfer tasks (S8 Fig). This is not unlike the primate, where downstream visual areas support a variety of visual recognition tasks [55]. Determining the animal-to-animal mapping transform Prior models of mouse visual cortex can be improved by varying three ingredients to better match the biology and the ecology of mouse visual cortex. Before model development, however, we must determine the appropriate procedure by which to evaluate models. As in prior work on modeling primate visual cortex, we “map” model responses to biological responses and the ability of the model responses to recapitulate biological responses determines the model’s neural fidelity [8, 9, 17, 20]. How should artificial neural network responses be mapped to biological neural responses? What firing patterns of mouse visual areas are common across multiple animals, and thus worthy of computational explanation? A natural approach would be to map artificial neural network features to mouse neural responses in the same manner that different animals can be mapped to each other. Specifically, we aimed to identify the best performing class of similarity transforms needed to map the firing patterns of one animal’s neural population to that of another, which we denote as the “inter-animal consistency”. We took inspiration from methods that have proven useful in modeling human and non-human primate visual, auditory, and motor cortices [16, 27–29]. As with other cortical areas, this transform class likely cannot be so strict as to require fixed neuron-to-neuron mappings between cells. However, the transform class for each visual area also cannot be so loose as to allow an unconstrained nonlinear mapping, since the model already yields an image-computable nonlinear response. We explored a variety of linear mapping transform classes (fit with different constraints) between the population responses for each mouse visual area (Fig 1A). The mouse visual responses to natural scenes were collected previously using both two-photon calcium imaging and Neuropixels by the Allen Institute [15, 22] from areas V1 (VISp), LM (VISl), AL (VISal), RL (VISrl), AM (VISam), and PM (VISpm) in mouse visual cortex (see number of units and specimens for each dataset in Table 1 and further details in the “Neural Response Datasets” section). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Inter-animal neural response consistency across mouse visual areas. A. Inter-animal consistency was computed using different linear maps, showing that PLS regression provides the highest consistency. Horizontal bars at the top are the median and s.e.m. of the internal consistencies of the neurons in each visual area. Refer to Table 1 for N units per visual area. B. The fraction of maximum split-half reliability is plotted as a function of time (in 10-ms time bins) for each visual area. https://doi.org/10.1371/journal.pcbi.1011506.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Descriptive statistics of the neural datasets. Total number of units and specimens for each visual area for the calcium imaging and Neuropixels datasets. https://doi.org/10.1371/journal.pcbi.1011506.t001 We focused on the natural scene stimuli, consisting of 118 images, each presented 50 times (i.e., 50 trials per image). For all methods, the corresponding mapping was trained on 50% of all the natural scene images, and evaluated on the remaining held-out set of images. We also included representational similarity analyses (RSA; [30]) as a baseline measure of population-wide similarity across animals, corresponding to no selection of individual units, unlike the other mapping transforms. For the strictest mapping transform (One-to-One), each target unit was mapped to the single most correlated unit in the source animal. Overall, the One-to-One mapping tended to yield the lowest inter-animal consistency among the maps considered. However, Ridge regression (L2-regularized) and Partial Least Squares (PLS) regression were more effective at the inter-animal mapping, yielding the most consistent fits across visual areas, with PLS regression providing the highest inter-animal consistency. We therefore used PLS regression in the evaluation of a candidate model’s ability to predict neural responses. This mapping transform confers the additional benefit of enabling direct comparison to prior primate ventral stream results (which also used this mapping [8, 17]) in order to better understand ecological differences between the two visual systems across species. Under all mapping transforms, a log-linear extrapolation analysis (S5 Fig) reveals that as the number of units increases, the inter-animal consistency approaches 1.0 more rapidly for the Neuropixels dataset, than the calcium imaging dataset, which were obtained from the average of the Δf/F trace—indicating the higher reliability of the Neuropixels data across all visual areas. We further noticed a large difference between the inter-animal consistency obtained via RSA and the consistencies achieved by any of the other mapping transforms for the responses in RL of the calcium imaging dataset (green in Fig 1A). This difference, however, was not observed for responses in RL in the Neuropixels dataset. This discrepancy suggested that there was a high degree of population-level heterogeneity in the responses collected from the calcium imaging dataset, which may be attributed to the fact that the two-photon field-of-view for RL spanned the boundary between the visual and somatosensory cortex, as originally noted by de Vries et al. [15]. We therefore excluded RL in the calcium imaging dataset from further analyses, following Siegle et al. [31], who systematically compared these two datasets. Thus, this analysis provided insight into the experiments from which the data were collected, and allowed us to ascertain the level of neural response variance that is common across animals and that therefore should be “explained” by candidate models. For these reasons above, we present our main results on the newer and more reliable Neuropixels dataset, as it is in a better position to separate models—with similar results on the calcium imaging dataset presented in the Supporting Information. Modeling mouse visual cortex Building quantitatively accurate models. With this mapping and evaluation procedure, we can then develop models to better match mouse visual responses. The overall conclusion that the mouse visual system is most consistent with an artificial neural network model that is self-supervised, low-resolution, and comparatively shallow. This conclusion holds more generally as well on the earlier calcium imaging dataset, along with non-regression-based comparisons like RSA (cf. S2, S3 and S4 Figs). Our best models attained neural predictivity of 90% of the inter-animal consistency, much better than the prior high-resolution, deep, and task-specific model (VGG16), which attained 56.27% of this ceiling (Fig 2A; cf. rightmost column of Table 2 for the unnormalized quantities). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Substantially improving neural response predictivity of models of mouse visual cortex. A. The median and s.e.m. (across units) neural predictivity difference with the prior-used, primate model of Supervised VGG16 trained on 224 px inputs (“Primate Model Baseline”, used in [14, 15, 36]), under PLS regression, across units in all mouse visual areas (N = 1731 units in total). Absolute neural predictivity (always computed from the model layer that best predicts a given visual area) for each model can be found in Table 2. Our best model is denoted as “AlexNet (IR)” on the far left. “Single Stream”, “Dual Stream”, and “Six Stream” are novel architectures we developed based on the first four layers of AlexNet, but additionally incorporates dense skip connections, known from the feedforward connectivity of the mouse connectome [34, 35], as well as multiple parallel streams (schematized in S1 Fig). CPC denotes contrastive predictive coding [32, 37]. All models, except for the “Primate Model Baseline”, are trained on 64 px inputs. We also note that all the models are trained using ImageNet, except for CPC (purple), Depth Prediction (orange), and the CIFAR-10-labeled black bars. B. Training a model on a contrastive objective improves neural predictivity across all visual areas. For each visual area, the neural predictivity values are plotted across all model layers for an untrained AlexNet, Supervised (ImageNet) AlexNet and Contrastive AlexNet (ImageNet, instance recognition)—the latter’s first four layers form the best model of mouse visual cortex. Shaded regions denote mean and s.e.m. across units. https://doi.org/10.1371/journal.pcbi.1011506.g002 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. ImageNet top-1 validation set accuracy via linear transfer or via supervised training and neural predictivity for each model. We summarize here the top-1 accuracy for each self-supervised and supervised model on ImageNet as well as their noise-corrected neural predictivity obtained via the PLS map (aggregated across all visual areas). These values are plotted in Fig 2C and S2 Fig. Unless otherwise stated, each model is trained and validated on 64 × 64 pixels images. https://doi.org/10.1371/journal.pcbi.1011506.t002 We also attain neural predictivity improvements over prior work [32, 33] (cf. the purple and green bars in Fig 2A), especially the latter “MouseNet” of Shi et al. [33], which attempts to map details of the mouse connectome [34, 35] onto a CNN architecture. There are two green bars in Fig 2A since we also built our own variant of MouseNet where everything is the same except that image categories are read off of at the penultimate layer of the model (rather than the concatenation of the earlier layers as originally proposed). We thought this might aid the original MouseNet’s task performance and neural predictivity, since it can be difficult to train linear layers when the input dimensionality is very large. Our best models also outperform the neural predictivity of “MouseNet” even when it is trained with a self-supervised objective (leftmost red vs. “MouseNet” teal bars in Fig 2A). This is another conceptual motivation for our structural-and-functional goal-driven approach, as the higher-level constraints are easier to interrogate than to incorporate and assume individual biological details, as this can be a very under constrained procedure. In the subsequent subsections, we distill the three factors that contributed to models with improved correspondence to the mouse visual areas: objective function, input resolution, and architecture. Objective function: Training models on self-supervised, contrastive objectives, instead of supervised objectives, improves correspondence to mouse visual areas. The success in modeling the human and the non-human primate visual system has largely been driven by convolutional neural networks trained in a supervised manner on ImageNet [19] to perform object categorization [9, 13]. This suggests that models trained with category-label supervision learn useful visual representations that are well-matched to those of the primate ventral visual stream [17, 20]. Thus, although biologically implausible for the rodent, category-label supervision is a useful starting point for building baseline models and indeed, models trained in this manner are much improved over the prior primate model of VGG16 (black bars in Fig 2A). We refer to this model as the “Primate Model Baseline”, since although many different models have been used to predict neural activity in the primate ventral stream, this model in particular was the de facto CNN used in the initial goal-driven modeling studies of mouse visual cortex [14, 15, 36]. This choice also helps explicitly illustrate how the three factors that we study, which deviate in visual acuity, model depth, and functional objective from the primate ventral stream, quantitatively improve these models greatly. These improved, category-label-supervised models, however, cannot be an explanation for how the mouse visual system developed in the first place. In particular, it is unclear whether or not rodents can perform well on large-scale object recognition tasks when trained, such as tasks where there are hundreds of labels. For example, they obtain approximately 70% on a two-alternative forced-choice object classification task [38]. Furthermore, the categories of the ImageNet dataset are human-centric and therefore not relevant for rodents In fact, the affordances of primates involve being able to manipulate objects flexibly with their hands (unlike rodents). Therefore, the ImageNet categories may be more relevant to non-human primates as an ethological proxy than it is to rodents. We therefore turned to more ecological supervision signals and self-supervised objective functions, where high-level category labeling is not necessary. These objectives could possibly lead to models with improved biological plausibility and may provide more general goals for the models, based on natural image statistics, beyond the semantic specifics of (human-centric) object categorization. Among the more ecological supervised signals, we consider categorization of comparatively lower-variation and lower-resolution images with fewer labels (CIFAR-10-labeled black bars in Fig 2A; [39]), and depth prediction (orange bars in Fig 2A; [40]), as a visual proxy for whisking [41]. Here, we note that in Fig 2A, all models except for those indicated by CIFAR-10 (black), CPC (purple), or Depth Prediction (orange), are trained on ImageNet images. Therefore, even if we remove the category labels from training, many of the images themselves (mainly from both ImageNet and CIFAR-10) are human-centric so future models could be trained using images that are more plausible for mice. Turning to self-supervision, early self-supervised objectives include sparse autoencoding (pink bars in Fig 2A; [42]), instantiated as a sparsity penalty in the latent space of the image reconstruction loss, which has been shown to be successful at producing Gabor-like functions reminiscent of the experimental findings in the work of Hubel and Wiesel [43]. These objectives, however, have not led to quantitatively accurate models of higher visual cortex [8, 20]. Further developments in computer vision have led to other self-supervised algorithms, motivated by the idea that “non-semantic” features are highly related to the higher-level, semantic features (i.e., category labels), such as predicting the rotation angle of an image (blue bars in Fig 2A; [44]). Although these objectives are very simple, models optimized on them do not result in “powerful” visual representations for downstream tasks. We trained models on these objectives and showed that although they improve neural predictivity over the prior primate model (VGG16), they do not outperform category-label-supervised models (Fig 2A; compare pink, blue, and orange with black). Further developments in self-supervised learning provided a new class of contrastive objectives. These objectives are much more powerful than prior self-supervised objectives described above, as it has been shown that models trained with contrastive objectives leads to visual representations that can support strong performance on downstream object categorization tasks. At a high level, the goal of these contrastive objectives is to learn a representational space where embeddings of augmentations for one image (i.e., embeddings for two transformations of the same image) are more “similar” to each other than to embeddings of other images. We trained models using the family of contrastive objective functions including: Instance Recognition (IR; [45]), a Simple Framework for Contrastive Learning (SimCLR; [46]), Momentum Contrast (MoCov2; [47]), Simple Siamese representation learning (SimSiam; [48]), Barlow Twins [49], and Variance-Invariance-Covariance Regularization (VICReg; [50]). Note that we are using the term “contrastive” broadly to encompass methods that learn embeddings which are robust to augmentations, even if they do not explicitly rely on negative batch examples—as they have to contrast against something to avoid representational collapse. For example, SimSiam relies on asymmetric representations via a stop gradient; Barlow Twins relies on regularizing with the cross correlation matrix’s off diagonal elements; and VICReg uses the variance and covariance of each embedding to ensure samples in the batch are different. Models trained with these contrastive objectives (red bars in Fig 2A) resulted in higher neural predictivity across all the visual areas than models trained on supervised object categorization, depth prediction, and less-powerful self-supervised algorithms (black, orange, purple, pink, and blue bars in Fig 2A). We hone in on the contribution of the contrastive objective function (over a supervised objective, and with a fixed dataset of ImageNet) to neural predictivity by fixing the architecture to be AlexNet, while varying the objective function (shown in S9 Fig left). We find that across all objective functions, training AlexNet using instance recognition leads to the highest neural predictivity. We additionally note that the improvement in neural predictivity extends beyond the augmentation used for each objective function, as shown in S7 Fig. When the image augmentations intended for contrastive losses is used with supervised losses, neural predictivity does not improve. Across all the visual areas, there is an improvement in neural predictivity simply by using a powerful contrastive algorithm (red vs. black in Fig 2B and S9 Fig left). Not only is there an improvement in neural predictivity, but also an improvement in hierarchical correspondence to the mouse visual hierarchy. Using the brain hierarchy score developed by Nonaka et al. [51], we observe that contrastive models outperform their supervised counterparts in matching the mouse visual hierarchy (S10 Fig). The first four layers of Contrastive AlexNet (red; Fig 2B), where neural predictivity is maximal across visual areas, forms our best model for mouse visual cortex. Data stream: Training models on images of lower resolution improves correspondence to mouse visual areas. The visual acuity of mice is known to be lower than the visual acuity of primates [25, 26]. Thus, more accurate models of mouse visual cortex must be trained and evaluated at image resolutions that are lower than those used in the training of models of the primate visual system. We investigated how neural predictivity of two strong contrastive models varied as a function of the image resolution at which they were trained. Two models were used in the exploration of image resolution’s effects on neural predictivity. Contrastive AlexNet was trained with image resolutions that varied from 64 × 64 pixels to 224 × 224 pixels, as 64 × 64 pixels was the minimum image size for AlexNet due to its architecture. The image resolution upper bound of 224 × 224 pixels is the image resolution that is typically used to train neural network models of the primate ventral visual stream [17]. We also investigated image resolution’s effects using a novel model architecture we developed, known as “Contrastive StreamNet”, as its architecture enables us to explore a lower range of image resolutions than the original AlexNet. This model was based on the first four layers of AlexNet, but additionally incorporates dense skip connections, known from the feedforward connectivity of the mouse connectome [34, 35], as well as multiple parallel streams (schematized in S1 Fig). We trained it using a contrastive objective function (instance recognition) at image resolutions that varied from 32 × 32 pixels to 224 × 224 pixels. Training models using resolutions lower than what is used for models of primate visual cortex improves neural predictivity across all visual areas, but not beyond a certain resolution, where neural predictivity decreases (Fig 3). Although the input resolution of 64 × 64 pixels may not be optimal for every architecture, it was the resolution that we used to train all the models. This was motivated by the observation that the upper bound on mouse visual acuity is 0.5 cycles / degree [25], corresponding to 2 pixels / cycle × 0.5 cycles / degree = 1 pixel / degree. Prior retinotopic map studies [52] estimate a visual coverage range in V1 of 60–90 degrees, and we found 64 × 64 pixels to be roughly optimal for the models (Fig 3) and was also used by Shi et al. [36], by Bakhtiari et al. [32], and in the MouseNet of Shi et al. [33]. Although downsampling training images is a reasonable proxy for the mouse retina, as has been done in prior modeling work, more investigation into appropriate image transformations may be needed. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Lower-resolution training leads to improved neural response predictivity (Neuropixels dataset). Contrastive AlexNet and Contrastive StreamNet (Dual Stream) were trained using images of increasing resolutions. For AlexNet, the minimum resolution was 64 × 64 pixels. For Dual StreamNet, the minimum resolution was 32 × 32 pixels. The median and s.e.m. neural predictivity across all units of all visual areas is plotted against the image resolution at which the models were trained. Reducing the image resolution (but not beyond a certain point) during representation learning improves the match to all the visual areas. The conversion from image resolution in pixels to visual field coverage in degrees is under the assumption that the upper bound on mouse visual acuity is 0.5 cycles per degree [25] and that the Nyquist limit is 2 pixels per cycle, leading to a conversion ratio of 1 pixel per degree. https://doi.org/10.1371/journal.pcbi.1011506.g003 Overall, these data show that optimization using a simple change in the image statistics (i.e., data stream) is crucial to obtain improved models of mouse visual encoding. This suggests that mouse visual encoding is the result of “task-optimization” at a lower resolution than what is typically used for primate ventral stream models. Architecture: Shallow models suffice for improving correspondence to mouse visual cortex. Anatomically, the mouse visual system has a shallower hierarchy relative to that of the primate visual system (see, e.g., [21–24]). Furthermore, the reliability of the neural responses for each visual area provides additional support for the relatively shallow functional hierarchy (Fig 1B). Thus, more biologically-plausible models of mouse visual cortex should have fewer linear-nonlinear layers than those of primate visual cortex. By plotting a model’s neural predictivity against its number of linear-nonlinear operations, we indeed found that very deep models do not outperform shallow models (i.e., models with less than 12 linear-nonlinear operations), despite changes across loss functions and input resolutions (Fig 4). In addition, if we fix the objective function to be “instance recognition”, we can clearly observe that AlexNet, which consists of eight linear-nonlinear layers, has the highest neural predictivity compared to ResNets and VGG16 (S9 Fig right). Furthermore, models with only four convolutional layers (StreamNets; Single, Dual, or Six Streams) perform as well as or better than models with many more convolutional layers (e.g., compare Dual Stream with ResNet101, ResNet152, VGG16, or MouseNets in S9 Fig right). This observation is in direct contrast with prior observations in the primate visual system whereby networks with fewer than 18 linear-nonlinear operations predict macaque visual responses less well than models with at least 50 linear-nonlinear operations [17]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Neural predictivity vs. model depth. A model’s median neural predictivity across all units from each visual area is plotted against its depth (number of linear-nonlinear layers; in log-scale). Models with fewer linear-nonlinear layers can achieve neural predictivity performances that outperform or are on par with those of models with many more linear-nonlinear layers. The “Primate Model Baseline” denotes a supervised VGG16 trained on 224 px inputs, used in prior work [14, 15, 36]. https://doi.org/10.1371/journal.pcbi.1011506.g004 Building quantitatively accurate models. With this mapping and evaluation procedure, we can then develop models to better match mouse visual responses. The overall conclusion that the mouse visual system is most consistent with an artificial neural network model that is self-supervised, low-resolution, and comparatively shallow. This conclusion holds more generally as well on the earlier calcium imaging dataset, along with non-regression-based comparisons like RSA (cf. S2, S3 and S4 Figs). Our best models attained neural predictivity of 90% of the inter-animal consistency, much better than the prior high-resolution, deep, and task-specific model (VGG16), which attained 56.27% of this ceiling (Fig 2A; cf. rightmost column of Table 2 for the unnormalized quantities). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Substantially improving neural response predictivity of models of mouse visual cortex. A. The median and s.e.m. (across units) neural predictivity difference with the prior-used, primate model of Supervised VGG16 trained on 224 px inputs (“Primate Model Baseline”, used in [14, 15, 36]), under PLS regression, across units in all mouse visual areas (N = 1731 units in total). Absolute neural predictivity (always computed from the model layer that best predicts a given visual area) for each model can be found in Table 2. Our best model is denoted as “AlexNet (IR)” on the far left. “Single Stream”, “Dual Stream”, and “Six Stream” are novel architectures we developed based on the first four layers of AlexNet, but additionally incorporates dense skip connections, known from the feedforward connectivity of the mouse connectome [34, 35], as well as multiple parallel streams (schematized in S1 Fig). CPC denotes contrastive predictive coding [32, 37]. All models, except for the “Primate Model Baseline”, are trained on 64 px inputs. We also note that all the models are trained using ImageNet, except for CPC (purple), Depth Prediction (orange), and the CIFAR-10-labeled black bars. B. Training a model on a contrastive objective improves neural predictivity across all visual areas. For each visual area, the neural predictivity values are plotted across all model layers for an untrained AlexNet, Supervised (ImageNet) AlexNet and Contrastive AlexNet (ImageNet, instance recognition)—the latter’s first four layers form the best model of mouse visual cortex. Shaded regions denote mean and s.e.m. across units. https://doi.org/10.1371/journal.pcbi.1011506.g002 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. ImageNet top-1 validation set accuracy via linear transfer or via supervised training and neural predictivity for each model. We summarize here the top-1 accuracy for each self-supervised and supervised model on ImageNet as well as their noise-corrected neural predictivity obtained via the PLS map (aggregated across all visual areas). These values are plotted in Fig 2C and S2 Fig. Unless otherwise stated, each model is trained and validated on 64 × 64 pixels images. https://doi.org/10.1371/journal.pcbi.1011506.t002 We also attain neural predictivity improvements over prior work [32, 33] (cf. the purple and green bars in Fig 2A), especially the latter “MouseNet” of Shi et al. [33], which attempts to map details of the mouse connectome [34, 35] onto a CNN architecture. There are two green bars in Fig 2A since we also built our own variant of MouseNet where everything is the same except that image categories are read off of at the penultimate layer of the model (rather than the concatenation of the earlier layers as originally proposed). We thought this might aid the original MouseNet’s task performance and neural predictivity, since it can be difficult to train linear layers when the input dimensionality is very large. Our best models also outperform the neural predictivity of “MouseNet” even when it is trained with a self-supervised objective (leftmost red vs. “MouseNet” teal bars in Fig 2A). This is another conceptual motivation for our structural-and-functional goal-driven approach, as the higher-level constraints are easier to interrogate than to incorporate and assume individual biological details, as this can be a very under constrained procedure. In the subsequent subsections, we distill the three factors that contributed to models with improved correspondence to the mouse visual areas: objective function, input resolution, and architecture. Objective function: Training models on self-supervised, contrastive objectives, instead of supervised objectives, improves correspondence to mouse visual areas. The success in modeling the human and the non-human primate visual system has largely been driven by convolutional neural networks trained in a supervised manner on ImageNet [19] to perform object categorization [9, 13]. This suggests that models trained with category-label supervision learn useful visual representations that are well-matched to those of the primate ventral visual stream [17, 20]. Thus, although biologically implausible for the rodent, category-label supervision is a useful starting point for building baseline models and indeed, models trained in this manner are much improved over the prior primate model of VGG16 (black bars in Fig 2A). We refer to this model as the “Primate Model Baseline”, since although many different models have been used to predict neural activity in the primate ventral stream, this model in particular was the de facto CNN used in the initial goal-driven modeling studies of mouse visual cortex [14, 15, 36]. This choice also helps explicitly illustrate how the three factors that we study, which deviate in visual acuity, model depth, and functional objective from the primate ventral stream, quantitatively improve these models greatly. These improved, category-label-supervised models, however, cannot be an explanation for how the mouse visual system developed in the first place. In particular, it is unclear whether or not rodents can perform well on large-scale object recognition tasks when trained, such as tasks where there are hundreds of labels. For example, they obtain approximately 70% on a two-alternative forced-choice object classification task [38]. Furthermore, the categories of the ImageNet dataset are human-centric and therefore not relevant for rodents In fact, the affordances of primates involve being able to manipulate objects flexibly with their hands (unlike rodents). Therefore, the ImageNet categories may be more relevant to non-human primates as an ethological proxy than it is to rodents. We therefore turned to more ecological supervision signals and self-supervised objective functions, where high-level category labeling is not necessary. These objectives could possibly lead to models with improved biological plausibility and may provide more general goals for the models, based on natural image statistics, beyond the semantic specifics of (human-centric) object categorization. Among the more ecological supervised signals, we consider categorization of comparatively lower-variation and lower-resolution images with fewer labels (CIFAR-10-labeled black bars in Fig 2A; [39]), and depth prediction (orange bars in Fig 2A; [40]), as a visual proxy for whisking [41]. Here, we note that in Fig 2A, all models except for those indicated by CIFAR-10 (black), CPC (purple), or Depth Prediction (orange), are trained on ImageNet images. Therefore, even if we remove the category labels from training, many of the images themselves (mainly from both ImageNet and CIFAR-10) are human-centric so future models could be trained using images that are more plausible for mice. Turning to self-supervision, early self-supervised objectives include sparse autoencoding (pink bars in Fig 2A; [42]), instantiated as a sparsity penalty in the latent space of the image reconstruction loss, which has been shown to be successful at producing Gabor-like functions reminiscent of the experimental findings in the work of Hubel and Wiesel [43]. These objectives, however, have not led to quantitatively accurate models of higher visual cortex [8, 20]. Further developments in computer vision have led to other self-supervised algorithms, motivated by the idea that “non-semantic” features are highly related to the higher-level, semantic features (i.e., category labels), such as predicting the rotation angle of an image (blue bars in Fig 2A; [44]). Although these objectives are very simple, models optimized on them do not result in “powerful” visual representations for downstream tasks. We trained models on these objectives and showed that although they improve neural predictivity over the prior primate model (VGG16), they do not outperform category-label-supervised models (Fig 2A; compare pink, blue, and orange with black). Further developments in self-supervised learning provided a new class of contrastive objectives. These objectives are much more powerful than prior self-supervised objectives described above, as it has been shown that models trained with contrastive objectives leads to visual representations that can support strong performance on downstream object categorization tasks. At a high level, the goal of these contrastive objectives is to learn a representational space where embeddings of augmentations for one image (i.e., embeddings for two transformations of the same image) are more “similar” to each other than to embeddings of other images. We trained models using the family of contrastive objective functions including: Instance Recognition (IR; [45]), a Simple Framework for Contrastive Learning (SimCLR; [46]), Momentum Contrast (MoCov2; [47]), Simple Siamese representation learning (SimSiam; [48]), Barlow Twins [49], and Variance-Invariance-Covariance Regularization (VICReg; [50]). Note that we are using the term “contrastive” broadly to encompass methods that learn embeddings which are robust to augmentations, even if they do not explicitly rely on negative batch examples—as they have to contrast against something to avoid representational collapse. For example, SimSiam relies on asymmetric representations via a stop gradient; Barlow Twins relies on regularizing with the cross correlation matrix’s off diagonal elements; and VICReg uses the variance and covariance of each embedding to ensure samples in the batch are different. Models trained with these contrastive objectives (red bars in Fig 2A) resulted in higher neural predictivity across all the visual areas than models trained on supervised object categorization, depth prediction, and less-powerful self-supervised algorithms (black, orange, purple, pink, and blue bars in Fig 2A). We hone in on the contribution of the contrastive objective function (over a supervised objective, and with a fixed dataset of ImageNet) to neural predictivity by fixing the architecture to be AlexNet, while varying the objective function (shown in S9 Fig left). We find that across all objective functions, training AlexNet using instance recognition leads to the highest neural predictivity. We additionally note that the improvement in neural predictivity extends beyond the augmentation used for each objective function, as shown in S7 Fig. When the image augmentations intended for contrastive losses is used with supervised losses, neural predictivity does not improve. Across all the visual areas, there is an improvement in neural predictivity simply by using a powerful contrastive algorithm (red vs. black in Fig 2B and S9 Fig left). Not only is there an improvement in neural predictivity, but also an improvement in hierarchical correspondence to the mouse visual hierarchy. Using the brain hierarchy score developed by Nonaka et al. [51], we observe that contrastive models outperform their supervised counterparts in matching the mouse visual hierarchy (S10 Fig). The first four layers of Contrastive AlexNet (red; Fig 2B), where neural predictivity is maximal across visual areas, forms our best model for mouse visual cortex. Data stream: Training models on images of lower resolution improves correspondence to mouse visual areas. The visual acuity of mice is known to be lower than the visual acuity of primates [25, 26]. Thus, more accurate models of mouse visual cortex must be trained and evaluated at image resolutions that are lower than those used in the training of models of the primate visual system. We investigated how neural predictivity of two strong contrastive models varied as a function of the image resolution at which they were trained. Two models were used in the exploration of image resolution’s effects on neural predictivity. Contrastive AlexNet was trained with image resolutions that varied from 64 × 64 pixels to 224 × 224 pixels, as 64 × 64 pixels was the minimum image size for AlexNet due to its architecture. The image resolution upper bound of 224 × 224 pixels is the image resolution that is typically used to train neural network models of the primate ventral visual stream [17]. We also investigated image resolution’s effects using a novel model architecture we developed, known as “Contrastive StreamNet”, as its architecture enables us to explore a lower range of image resolutions than the original AlexNet. This model was based on the first four layers of AlexNet, but additionally incorporates dense skip connections, known from the feedforward connectivity of the mouse connectome [34, 35], as well as multiple parallel streams (schematized in S1 Fig). We trained it using a contrastive objective function (instance recognition) at image resolutions that varied from 32 × 32 pixels to 224 × 224 pixels. Training models using resolutions lower than what is used for models of primate visual cortex improves neural predictivity across all visual areas, but not beyond a certain resolution, where neural predictivity decreases (Fig 3). Although the input resolution of 64 × 64 pixels may not be optimal for every architecture, it was the resolution that we used to train all the models. This was motivated by the observation that the upper bound on mouse visual acuity is 0.5 cycles / degree [25], corresponding to 2 pixels / cycle × 0.5 cycles / degree = 1 pixel / degree. Prior retinotopic map studies [52] estimate a visual coverage range in V1 of 60–90 degrees, and we found 64 × 64 pixels to be roughly optimal for the models (Fig 3) and was also used by Shi et al. [36], by Bakhtiari et al. [32], and in the MouseNet of Shi et al. [33]. Although downsampling training images is a reasonable proxy for the mouse retina, as has been done in prior modeling work, more investigation into appropriate image transformations may be needed. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Lower-resolution training leads to improved neural response predictivity (Neuropixels dataset). Contrastive AlexNet and Contrastive StreamNet (Dual Stream) were trained using images of increasing resolutions. For AlexNet, the minimum resolution was 64 × 64 pixels. For Dual StreamNet, the minimum resolution was 32 × 32 pixels. The median and s.e.m. neural predictivity across all units of all visual areas is plotted against the image resolution at which the models were trained. Reducing the image resolution (but not beyond a certain point) during representation learning improves the match to all the visual areas. The conversion from image resolution in pixels to visual field coverage in degrees is under the assumption that the upper bound on mouse visual acuity is 0.5 cycles per degree [25] and that the Nyquist limit is 2 pixels per cycle, leading to a conversion ratio of 1 pixel per degree. https://doi.org/10.1371/journal.pcbi.1011506.g003 Overall, these data show that optimization using a simple change in the image statistics (i.e., data stream) is crucial to obtain improved models of mouse visual encoding. This suggests that mouse visual encoding is the result of “task-optimization” at a lower resolution than what is typically used for primate ventral stream models. Architecture: Shallow models suffice for improving correspondence to mouse visual cortex. Anatomically, the mouse visual system has a shallower hierarchy relative to that of the primate visual system (see, e.g., [21–24]). Furthermore, the reliability of the neural responses for each visual area provides additional support for the relatively shallow functional hierarchy (Fig 1B). Thus, more biologically-plausible models of mouse visual cortex should have fewer linear-nonlinear layers than those of primate visual cortex. By plotting a model’s neural predictivity against its number of linear-nonlinear operations, we indeed found that very deep models do not outperform shallow models (i.e., models with less than 12 linear-nonlinear operations), despite changes across loss functions and input resolutions (Fig 4). In addition, if we fix the objective function to be “instance recognition”, we can clearly observe that AlexNet, which consists of eight linear-nonlinear layers, has the highest neural predictivity compared to ResNets and VGG16 (S9 Fig right). Furthermore, models with only four convolutional layers (StreamNets; Single, Dual, or Six Streams) perform as well as or better than models with many more convolutional layers (e.g., compare Dual Stream with ResNet101, ResNet152, VGG16, or MouseNets in S9 Fig right). This observation is in direct contrast with prior observations in the primate visual system whereby networks with fewer than 18 linear-nonlinear operations predict macaque visual responses less well than models with at least 50 linear-nonlinear operations [17]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Neural predictivity vs. model depth. A model’s median neural predictivity across all units from each visual area is plotted against its depth (number of linear-nonlinear layers; in log-scale). Models with fewer linear-nonlinear layers can achieve neural predictivity performances that outperform or are on par with those of models with many more linear-nonlinear layers. The “Primate Model Baseline” denotes a supervised VGG16 trained on 224 px inputs, used in prior work [14, 15, 36]. https://doi.org/10.1371/journal.pcbi.1011506.g004 Mouse visual cortex as a general-purpose visual system Our results show that a model optimized on a self-supervised contrastive objective the most quantitatively accurate model of the mouse visual system. However, unlike supervised object categorization or other more-classical forms of self-supervised learning such as sparse autoencoding, whose ecological function is directly encoded in the loss function (e.g., predator recognition or metabolically-efficient dimension reduction), the functional utility of self-supervision via a contrastive objective is not as apparent. This naturally raises the question of what behavioral function(s) optimizing a contrastive self-supervision objective might enable for the mouse from an ecological fitness viewpoint. Analyzing the spectrum of models described in the above section, we first observed that performance on ImageNet categorization is uncorrelated with improved neural predictivity for mouse visual cortex (Fig 5; cf. middle column of Table 2 for the exact ImageNet performance values), unlike the well-known correlation in primates (Fig 5; inset). In seeking an interpretation of the biological function of the contrastive self-supervised objective, we were thus prompted to consider behaviors beyond object-centric categorization tasks. We hypothesized that since self-supervised loss functions are typically most effective in task-agnostic stimulus domains where goal-specific labels are unavailable, optimizing for such an objective might enable the rodent to transfer well to complex multi-faceted ecological tasks in novel domains affording few targeted opportunities for hyper-specialization. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Neural predictivity is not correlated with object categorization performance on ImageNet. Each model’s (transfer or supervised) object categorization performance on ImageNet is plotted against its median neural predictivity across all units from all visual areas. All ImageNet performance values can be found in Table 2. Inset. Primate ventral visual stream neural predictivity from BrainScore is correlated with ImageNet categorization accuracy (adapted from Schrimpf et al. [17]). This relationship is in stark contrast to our finding in mouse visual cortex where higher ImageNet categorization accuracy is not associated with higher neural predictivity. Color scheme as in Fig 2A. See S2 Fig. for neural predictivity on the calcium imaging dataset. https://doi.org/10.1371/journal.pcbi.1011506.g005 To test this hypothesis, we used a recently-developed “virtual rodent” framework (adapted from Merel et al. [53] and also used by Lindsay et al. [54]), in which a biomechanically-validated mouse model is placed in a simulated 3D maze-like environment (Fig 6A). The primary purpose of the experiments with the virtual rodent are not to necessarily make a specific statement about rodent movement repertoires, but mainly to how well our self-supervised visual encoder enables control of a high-dimensional body with high-dimensional continuous inputs—a problem that many (if not all) animals have to solve. Of course, given that we were trying to better understand why self-supervised methods are better predictors of specifically mouse visual cortical neurons, we wanted a reasonable ecological task for that species (e.g., navigation) and that its affordances were somewhat similar to that of an actual rodent via the biomechanical realism of its body. In particular, if we used our shallow, lower acuity, self-supervised visual encoder for controlling a virtual monkey simulation for a task that a monkey is adapted to (e.g., object manipulation), we would not expect this to work as well, given that such a task likely requires high visual acuity and good object recognition abilities at a minimum. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Evaluating the generality of learned visual representations. A. Each row shows an example of an episode used to train the reinforcement learning (RL) policy in an offline fashion [56]. The episodes used to train the virtual rodent (adapted from Merel et al. [53]) were previously generated and are part of a larger suite of RL tasks for benchmarking algorithms [57]. In this task (“DM Locomotion Rodent”), the goal of the agent is to navigate the maze to collect as many rewards as possible (blue orbs shown in the first row). B. A schematic of the RL agent. The egocentric visual input is fed into the visual backbone of the model, which is fixed to be the first four convolutional layers of either the contrastive or the supervised variants of AlexNet. The output of the visual encoder is then concatenated with the virtual rodent’s proprioceptive inputs before being fed into the (recurrent) critic and policy heads. The parameters of the visual encoder are not trained, while the parameters of the critic head and the policy head are trained. Virtual rodent schematic from Fig 1B of Merel et al. [53]. C. A schematic of the out-of-distribution generalization procedure. Visual encoders are either trained in a supervised or self-supervised manner on ImageNet [19] or the Maze environment [57], and then evaluated on reward-based navigation or datasets consisting of object properties (category, pose, position, and size) from Hong et al. [55], and different textures [58]. https://doi.org/10.1371/journal.pcbi.1011506.g006 We used the model that best corresponds to mouse visual responses as the visual system of the simulated mouse, coupled to a simple actor-critic reinforcement-learning architecture (Fig 6B). We then trained the simulated mouse in several visual contexts with varying objective functions, and evaluated those models’ ability to transfer to a variety of tasks in novel environments, including both a reward-based navigation task, as well as several object-centric visual categorization and estimation tasks (Fig 6C). We first evaluated a simulated mouse whose visual system was pretrained with ImageNet images in terms of its ability to transfer to reward-based navigation in the Maze environment, training just the reinforcement learning portion of the network on the navigation task with the pretrained visual system held constant. As a supervised control, we performed the same transfer training procedure using a visual front-end created using supervised (ImageNet) object categorization pretraining. We found that the simulated mouse with the contrastive self-supervised visual representation was able to reliably obtain substantially higher navigation rewards than its counterpart with category-supervised visual representation (Fig 7A; cf. Table 3 for the exact quantities of rewards attained). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Self-supervised, contrastive visual representations better support transfer performance on downstream, out-of-distribution tasks. Models trained in a contrastive manner (using either ImageNet or the egocentric maze inputs; red and blue respectively) lead to better transfer on out-of-distribution downstream tasks than models trained in a supervised manner (i.e., models supervised on labels or on rewards; black and purple respectively). A. Models trained on ImageNet, tested on reward-based navigation. B. Models trained on egocentric maze inputs (“Contrastive Maze”, blue) or supervised on rewards (i.e., reward-based navigation; “Supervised Maze”, purple), tested on visual scene understanding tasks: pose, position, and size estimation, and object and texture classification. C. Median and s.e.m. neural predictivity across units in the Neuropixels dataset. https://doi.org/10.1371/journal.pcbi.1011506.g007 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Average reward and s.e.m. across 600 episodes obtained by the RL agent using each of the visual backbones. https://doi.org/10.1371/journal.pcbi.1011506.t003 Conversely, we trained the simulated mouse directly in the Maze environment. In one variant, we trained the visual system on the contrastive self-supervised objective (but on the images of the Maze environment). In a second variant, we trained the agent end-to-end on the reward navigation task itself—the equivalent of “supervision” in the Maze environment. We then tested both models on their ability to transfer to out-of-sample visual categorization and estimation tasks. Again, we found that the self-supervised variant transfers significantly better than its “supervised” counterpart (blue vs. purple bars in Fig 7B). Returning to analyses of neural predictivity, we found that both self-supervised models (trained in either environment) were better matches to mouse visual cortex neurons than the supervised counterparts in their respective environments (red and blue vs. black and purple in Fig 7C). This result illustrates the congruence between general out-of-distribution task-transfer performance and improved neural fidelity of the computational model. Furthermore, Lindsay et al. [54] found that less powerful self-supervised representation learners such as CPC and autoencoding did not match mouse visual responses as well as their RL-trained counterpart (in terms of representational similarity). This is consistent with our finding that CPC and autoencoding in Fig 3A themselves do not match neural responses as well as contrastive self-supervised methods (red vs. pink and purple bars). It is also noteworthy that, comparing models by training environment rather than objective function, the ImageNet-trained models produce more neurally-consistent models than the Maze-trained models, both for supervised and self-supervised objectives (red vs. blue and black vs. purple in Fig 7C). Using the contrastive self-supervised objective function is by itself enough to raise the Maze-trained model to the predictivity level of the ImageNet-trained supervised model, but the training environment makes a significant contribution. This suggests that while the task domain and biomechanical model of the Maze environment are realistic, future work will likely need to improve the realism of the simulated image distribution. Overall, these results suggest that contrastive embedding methods have achieved a generalized improvement in the quality of the visual representations they create, enabling a diverse range of visual behaviors, providing evidence for their potential as computational models of mouse visual cortex. While we expect the calculation of rewards to be performed outside of the visual system, we expect the mouse visual system would support the visual aspects of the transfer tasks we consider, as we find that higher model areas best support these scene understanding transfer tasks (S8 Fig). This is not unlike the primate, where downstream visual areas support a variety of visual recognition tasks [55]. Discussion In this work, we showed that comparatively shallow architectures trained with contrastive objectives operating on lower-resolution images most accurately predict static-image-evoked neural responses across multiple mouse visual areas, surpassing the predictive power of supervised methods and approaching the inter-animal consistency. The fact that these goal-driven constraints lead to a better match to visual responses, even in “behaviorally-free” data where mice are passively viewing stimuli, suggests that these constraints may be good descriptions of the evolutionary and the developmental drivers of mouse visual cortical structure and function. In the primate ventral visual stream, models trained on contrastive objectives led to neural predictivity performance that was on par with that of supervised models [20], suggesting they are a more ecologically-valid proxy for a categorization-specialized system. This is in stark contrast with our findings in our models of mouse visual cortex—we found that models trained on contrastive objectives substantially surpassed the neural predictivity of their supervised counterparts. We investigated the advantages that contrastive objectives might confer over supervised objectives for mouse visual representations and found that they provide representations that are generally improved over those obtained by supervised methods. The improvement in generality of contrastive models enabled better transfer to a diverse range of downstream behaviors in novel, out of distribution environments, including reward-based navigation in egocentric maze environments and visual scene understanding. As mentioned previously, the goal-driven modeling approach allows us to understand the principles that govern the system under study and further allows for direct comparisons across systems. Our high-fidelity model of mouse visual cortex and the principles underlying its construction can be compared with models of primate visual cortex. While the primate ventral visual stream is well-modeled by a deep hierarchical system and object-category learning, mouse as a model visual system has not had such a coherent account heretofore. Our results demonstrate that both primate and rodent visual systems are highly constrained, albeit for different functional purposes. The results summarized above further suggest that mouse visual cortex is a light-weight, shallower, low-resolution, and general-purpose visual system in contrast to the deep, high-resolution, and more categorization-dominated visual system of primates, suggested by prior work [20]. Although we have made progress in modeling the mouse visual system in three core ways (the choice of objective function, data stream, and architecture class), some limitations remain both in the modeling and in the neural data. On the architectural front, our focus in this work was on feedforward models, but there are many feedback connections from higher visual areas to lower visual areas [21]. Incorporating these architectural motifs into our models and training these models using dynamic inputs may be useful for modeling temporal dynamics in mouse visual cortex, as has been recently done in primates [29, 59, 60]. By incorporating recurrent connections in the architectures, we can probe the functionality of these feedback connections using self-supervised loss functions in scenarios with temporally-varying, dynamic inputs. For example, given that powerful self-supervised methods obtained good visual representations of static images, it would be interesting to explore a larger spectrum of self-supervised signals operating on dynamic inputs, such as in the context of forward prediction (e.g., [61–63]). Constraining the input data so that they are closer to those received by the mouse visual system was important for improved neural fidelity. Our resizing (i.e., downsampling) of the images to be smaller during training acted as a proxy for low-pass filtering. We believe that future work could investigate other appropriate low-pass filters and ecologically-relevant pixel-level transformations to apply to the original image or video stream [64, 65]. Our inter-animal consistency analyses make recommendations for qualities of experiments that are likely to provide data that will be helpful in more sharply differentiating models. Under all mapping functions, a log-linear extrapolation analysis revealed that as the number of units increases, the inter-animal consistency approaches one more rapidly for the Neuropixels dataset, than for the calcium imaging dataset, indicating the higher reliability of the Neuropixels data (S5 Fig). Moreover, when assessing inter-animal consistency, correlation values between animals were significantly higher for the training set than for the test set, indicating that the number of stimuli could be enlarged to close this generalization gap (S6A Fig). As a function of the number of stimuli, the test set inter-animal consistencies steadily increased, and would likely continue to increase substantially if the dataset had more stimuli (S6B Fig). Thus, while much focus in experimental methods has been on increasing the number of neurons in a dataset [66], our analyses indicate that increasing the number of stimuli may drastically improve model identification. Doing so would likely raise the inter-animal consistency, providing substantially more dynamic range for separating models in terms of their ability to match the data, potentially allowing us to obtain more specific conclusions about which circuit structure(s) [67, 68] and which (combinations of) objectives (e.g., [45, 46, 48]) best describe mouse visual cortex. We endeavor that our work in modeling mouse visual cortex will meaningfully drive future experimental and computational studies in mice of other sensory systems and of visually-guided behaviors. The input-domain-agnostic nature of these contrastive objectives suggest the tantalizing possibility that they might be used in other sensory systems, such as barrel cortex or the olfactory system. By building high-fidelity computational models of sensory cortex, we believe that they can be integrated with models of higher-order systems (e.g., medial temporal lobe), with the goal of providing us with greater insight into how sensory experience contributes to adaptive or maladaptive behaviors. Methods Neural response datasets We used the Allen Brain Observatory Visual Coding dataset [15, 22] collected using both two-photon calcium imaging and Neuropixels from areas V1 (VISp), LM (VISl), AL (VISal), RL (VISrl), AM (VISam), and PM (VISpm) in mouse visual cortex. We focused on the natural scene stimuli, consisting of 118 images, each presented 50 times (i.e., 50 trials per image). We list the number of units and specimens for each dataset in Table 1, after units are selected, according to the following procedure: For the calcium imaging data, we used a similar unit selection criterion as in Conwell et al. [18], where we sub-selected units that attain a Spearman-Brown corrected split-half consistency of at least 0.3 (averaged across 100 bootstrapped trials), and whose peak responses to their preferred images are not significantly modulated by the mouse’s running speed during stimulus presentation (p > 0.05). For the Neuropixels dataset, we separately averaged, for each specimen and each visual area, the temporal response (at the level of 10-ms bins up to 250 ms) on the largest contiguous time interval when the median (across the population of units in that specimen) split-half consistency reached at least 0.3. This procedure helps to select the most internally-consistent units in their temporally-averaged response, and accounts for the fact that different specimens have different time courses along which their population response becomes reliable. Finally, after subselecting units according to the above criteria for both datasets, we only keep specimens that have at least the 75th percentile number of units among all specimens for that given visual area. This final step helped to ensure we have enough internally-consistent units per specimen for the inter-animal consistency estimation (derived in the “Inter-Animal Consistency Derivation” section). Noise-corrected neural predictivity Linear regression. When we perform neural fits, we choose a random 50% set of natural scene images (59 images in total) to train the regression, and the remaining 50% to use as a test set (59 images in total), across ten train-test splits total. For Ridge regression, we use an α = 1, following the sklearn.linear_model convention. PLS regression was performed with 25 components, as in prior work (e.g., [8, 17]). When we perform regression with the One-to-One mapping, as in Fig 1B, we identify the top correlated (via Pearson correlation on the training images) unit in the source population for each target unit. Once that source unit has been identified, we then fix it for that particular train-test split, evaluated on the remaining 50% of images. Motivated by the justification given in the “Inter-Animal Consistency Derivation” section for the noise correction in the inter-animal consistency, the noise correction of the model to neural response regression is a special case of the quantity defined in the “Multiple Animals” section, where now the source animal is replaced by model features, separately fit to each target animal (from the set of available animals ). Let L be the set of model layers, let rℓ be the set of model responses at model layer ℓ ∈ L, M be the mapping, and let s be the trial-averaged pseudo-population response where the average is taken over 100 bootstrapped split-half trials, ⊕ denotes concatenation of units across animals followed by the median value across units, and denotes the Pearson correlation of the two quantities. denotes the Spearman-Brown corrected value of the original quantity (see the “Spearman-Brown Correction” section). Prior to obtaining the model features of the stimuli for linear regression, we preprocessed each stimulus using the image transforms used on the validation set during model training, resizing the shortest edge of the stimulus in both cases to 64 pixels, preserving the aspect ratio of the input stimulus. Specifically, for models trained using the ImageNet dataset, we first resized the shortest edge of the stimulus to 256 pixels, center-cropped the image to 224 × 224 pixels, and finally resized the stimulus to 64 × 64 pixels. For models trained using the CIFAR-10 dataset, this resizing yielded a 64 × 81 pixels stimulus. Representational Similarity Analysis (RSA). In line with prior work [18, 36], we also used representational similarity analysis (RSA; [30]) to compare models to neural responses, as well as to compare animals to each other. Specifically, we compared (via Pearson correlation) only the upper-right triangles of the representational dissimilarity matrices (RDMs), excluding the diagonals to avoid illusory effects [69]. For each visual area and a given model, we defined the predictivity of the model for that area to be the maximum RSA score across model layers after the suitable noise correction is applied, which is defined as follows. Let rℓ be the model responses at model layer ℓ and let s be the trial-averaged pseudo-population response (i.e., responses aggregated across specimens). The metric used here is a specific instance of Eq (10), where the single source animal is the trial-wise, deterministic model features (which have a mapping consistency of 1 as a result) and a single target animal , which is the pseudo-population response: (1) where L is the set of model layers, are the animal’s responses for two halves of the trials (and averaged across the trials dimension), the average is computed over 100 bootstrapped split-half trials, and denotes Spearman-Brown correction applied to the internal consistency quantity, , defined in the “Spearman-Brown Correction” section. If the fits are performed separately for each animal, then corresponds to each animal among those for a given visual area (defined by the set ), and we compute the median across animals : (2) Similar to the above, Spearman-Brown correction is applied to the internal consistency quantity, . Inter-animal consistency derivation Single animal pair. Suppose we have neural responses from two animals and . Let be the vector of true responses (either at a given time bin or averaged across a set of time bins) of animal on stimulus set . Of course, we only receive noisy observations of , so let be the jth set of n trials of . Finally, let M(x; y)i be the predictions of a mapping M (e.g., PLS) when trained on input x to match output y and tested on stimulus set i. For example, is the prediction of mapping M on the test set stimuli trained to match the true neural responses of animal given, as input, the true neural responses of animal on the train set stimuli. Similarly, is the prediction of mapping M on the test set stimuli trained to match the trial-average of noisy sample 1 on the train set stimuli of animal given, as input, the trial-average of noisy sample 1 on the train set stimuli of animal . With these definitions in hand, the inter-animal mapping consistency from animal to animal corresponds to the following true quantity to be estimated: (3) where is the Pearson correlation across a stimulus set. In what follows, we will argue that Eq (3) can be approximated with the following ratio of measurable quantities, where we split in half and average the noisy trial observations, indexed by 1 and by 2: (4) In words, the inter-animal consistency (i.e., the quantity on the left side of Eq (4)) corresponds to the predictivity of the mapping on the test set stimuli from animal to animal on two different (averaged) halves of noisy trials (i.e., the numerator on the right side of Eq (4)), corrected by the square root of the mapping reliability on animal ’s responses to the test set stimuli on two different halves of noisy trials multiplied by the internal consistency of animal . We justify the approximation in Eq (4) by gradually replacing the true quantities (t) by their measurable estimates (s), starting from the original quantity in Eq (3). First, we make the approximation that: (5) by the transitivity of positive correlations (which is a reasonable assumption when the number of stimuli is large). Next, by transitivity and normality assumptions in the structure of the noisy estimates and since the number of trials (n) between the two sets is the same, we have that: (6) In words, Eq (6) states that the correlation between the average of two sets of noisy observations of n trials each is approximately the square of the correlation between the true value and average of one set of n noisy trials. Therefore, combining Eqs (5) and (6), it follows that: (7) From the right side of Eq (7), we can see that we have removed , but we still need to remove the term, as this term still contains unmeasurable (i.e., true) quantities. We apply the same two steps, described above, by analogy, though these approximations may not always be true (they are, however, true for Gaussian noise): which taken together implies the following: (8) Eqs (7) and (8) together imply the final estimated quantity given in Eq (4). Multiple animals. For multiple animals, we consider the average of the true quantity for each target in in Eq (3) across source animals in the ordered pair (, ) of animals and : We also bootstrap across trials, and have multiple train/test splits, in which case the average on the right hand side of the equation includes averages across these as well. Note that each neuron in our analysis will have this single average value associated with it when it was a target animal (), averaged over source animals/subsampled source neurons, bootstrapped trials, and train/test splits. This yields a vector of these average values, which we can take median and standard error of the mean (s.e.m.) over, as we do with standard explained variance metrics. RSA. We can extend the above derivations to other commonly used metrics for comparing representations that involve correlation. Since , then the corresponding quantity in Eq (4) analogously (by transitivity of positive correlations) becomes: (9) Note that in this case, each animal (rather than neuron) in our analysis will have this single average value associated with it when it was a target animal () (since RSA is computed over images and neurons), where the average is over source animals/subsampled source neurons, bootstrapped trials, and train/test splits. This yields a vector of these average values, which we can take median and s.e.m. over, across animals . For RSA, we can use the identity mapping (since RSA is computed over neurons as well, the number of neurons between source and target animal can be different to compare them with the identity mapping). As parameters are not fit, we can choose , so that Eq (9) becomes: (10) Pooled source animal. Often times, we may not have enough neurons per animal to ensure that the estimated inter-animal consistency in our data closely matches the “true” inter-animal consistency. In order to address this issue, we holdout one animal at a time and compare it to the pseudo-population aggregated across units from the remaining animals, as opposed to computing the consistencies in a pairwise fashion. Thus, is still the target heldout animal as in the pairwise case, but now the average over is over a sole “pooled” source animal constructed from the pseudo-population of the remaining animals. Spearman-brown correction. The Spearman-Brown correction can be applied to each of the terms in the denominator individually, as they are each correlations of observations from half the trials of the same underlying process to itself (unlike the numerator). Namely, Analogously, since , then we define StreamNet architecture variants We developed shallower, multiple-streamed architectures for mouse visual cortex, shown in Fig 2A. There are three main modules in our architecture: shallow, intermediate, and deep. The shallow and deep modules each consist of one convolutional layer and the intermediate module consists of a block of two convolutional layers. Thus, the longest length of the computational graph, excluding the readout module, is four (i.e., 1 + 2 + 1). Depending on the number of parallel streams in the model, the intermediate module would contain multiple branches (in parallel), each receiving input from the shallow module. The outputs of the intermediate modules are then passed through one convolutional operation (deep module). Finally, the outputs of each parallel branch would be summed together, concatenated across the channels dimension, and used as input for the readout module. The readout module consists of an (adaptive) average pooling operation that upsamples the inputs to 6 × 6 feature maps. These feature maps are then flattened, so that each image only has a single feature vector. These feature vectors are then fed into a linear layer (i.e., fully-connected layer) either for classification or for embedding them into a lower-dimensional space for the contrastive losses. The table below describes the parameters of three model variants, each containing one (N = 1), two (N = 2), or six (N = 6) parallel branches. Note that one convolutional layer is denoted by a tuple: (number of filters, filter size, stride, padding). The first convolutional layer has max pooling of stride 2, as in AlexNet. A block of convolutional layers is denoted by a list of tuples, where each tuple in the list corresponds to a single convolutional layer. When a list of tuples is followed by “×N”, this means that the convolutional parameters for each of the N parallel branches are the same. Neural network training objectives In this section, we briefly describe the supervised and self-supervised objectives that were used to train our models. Download: PPT PowerPoint slide PNG larger image TIFF original image Supervised training objective. The loss function used in supervised training is the cross-entropy loss, defined as follows: (11) where N is the batch size, C is the number of categories for the dataset, are the model outputs (i.e., logits) for the N images, are the logits for the ith image, ci ∈ [0, C − 1] is the category index of the ith image (zero-indexed), and θ are the model parameters. Eq (11) was minimized using stochastic gradient descent (SGD) with momentum [70]. ImageNet [19]. This dataset contains approximately 1.3 million images in the train set and 50 000 images in the validation set. Each image was previously labeled into C = 1000 distinct categories. CIFAR-10 [39]. This dataset contains 50 000 images in the train set and 10 000 images in the validation set. Each image was previously labeled into C = 10 distinct categories. Depth prediction [40]. The goal of this objective is to predict the depth map of an image. We used a synthetically generated dataset of images known as PBRNet [40]. It contains approximately 500 000 images and their associated depth maps. Similar to the loss function used in the sparse autoencoder objective, we used a mean-squared loss to train the models. The output (i.e., depth map) was generated using a mirrored version of each of our StreamNet variants. In order to generate the depth map, we appended one final convolutional layer onto the output of the mirrored architecture in order to downsample the three image channels to one image channel. During training, random crops of size 224 × 224 pixels were applied to the image and depth map (which were both subsequently resized to 64 × 64 pixels). In addition, both the image and depth map were flipped horizontally with probability 0.5. Finally, prior to the application of the loss function, each depth map was normalized such that the mean and standard deviation across pixels were zero and one respectively. Each of our single-, dual-, and six-stream variants were trained using a batch size of 256 for 50 epochs using SGD with momentum of 0.9, and weight decay of 0.0001. The initial learning rate was set to 10−4 and was decayed by a factor of 10 at epochs 15, 30, and 45. Self-supervised training objectives. Sparse autoencoder [42]. The goal of this objective is to reconstruct an image from a sparse image embedding. In order to generate an image reconstruction, we used a mirrored version of each of our StreamNet variants. Concretely, the loss function was defined as follows: (12) where is the image embedding, f is the (mirrored) model, f(x) is the image reconstruction, x is a 64 × 64 pixels image, λ is the regularization coefficient, and θ are the model parameters. Our single-, dual-, and six-stream variants were trained using a batch size of 256 for 100 epochs using SGD with momentum of 0.9 and weight decay of 0.0005. The initial learning rate was set to 0.01 for the single- and dual-stream variants and was set to 0.001 for the six-stream variant. The learning rate was decayed by a factor of 10 at epochs 30, 60, and 90. For all the StreamNet variants, the embedding dimension was set to 128 and the regularization coefficient was set to 0.0005. RotNet [44]. The goal of this objective is to predict the rotation of an image. Each image of the ImageNet dataset was rotated four ways (0°, 90°, 180°, 270°) and the four rotation angles were used as “pseudo-labels” or “categories”. The cross-entropy loss was used with these pseudo-labels as the training objective (i.e., Eq (11) with C = 4). Our single-, dual-, and six-stream variants were trained using a batch size of 192 (which is effectively a batch size of 192 × 4 = 768 due to the four rotations for each image) for 50 epochs using SGD with Nesterov momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.01 was decayed by a factor of 10 at epochs 15, 30, and 45. Instance recognition [45]. The goal of this objective is to be able to differentiate between embeddings of augmentations of one image from embeddings of augmentations of other images. Thus, this objective function is an instance of the class of contrastive objective functions. A random image augmentation is first performed on each image of the ImageNet dataset (random resized cropping, random grayscale, color jitter, and random horizontal flip). Let x be an image augmentation, and f(⋅) be the model backbone composed with a one-layer linear multi-layer perceptron (MLP) of size 128. The image is then embedded onto a 128-dimensional unit-sphere as follows: Throughout model training, a memory bank containing embeddings for each image in the train set is maintained (i.e., the size of the memory bank is the same as the size of the train set). The embedding z will be “compared” to a subsample of these embeddings. Concretely, the loss function for one image x is defined as follows: (13) where is the embedding for image x that is currently stored in the memory bank, N is the size of the memory bank, m = 4096 is the number of “negative” samples used, are the negative embeddings sampled from the memory bank uniformly, Z is some normalization constant, τ = 0.07 is a temperature hyperparameter, and θ are the parameters of f. From Eq (13), we see that we want to maximize h(v), which corresponds to maximizing the similarity between v and z (recall that z is the embedding for x obtained using f). We can also see that we want to maximize 1 − h(vj) (or minimize h(vj)). This would correspond to minimizing the similarity between vj and z (recall that vj are the negative embeddings). After each iteration of training, the embeddings for the current batch are used to update the memory bank (at their corresponding positions in the memory bank) via a momentum update. Concretely, for image x, its embedding in the memory bank v is updated using its current embedding z as follows: where λ = 0.5 is the momentum coefficient. The second operation on v is used to project v back onto the 128-dimensional unit sphere. Our single-, dual-, and six-stream variants were trained using a batch size of 256 for 200 epochs using SGD with momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.03 was decayed by a factor of 10 at epochs 120 and 160. MoCov2 [47, 71]. The goal of this objective is to be able to distinguish augmentations of one image (i.e., by labeling them as “positive”) from augmentations of other images (i.e., by labeling them as “negative”). Intuitively, embeddings of different augmentations of the same image should be more “similar” to each other than to embeddings of augmentations of other images. Thus, this algorithm is another instance of the class of contrastive objective functions and is similar conceptually to instance recognition. Two image augmentations are first generated for each image in the ImageNet dataset by applying random resized cropping, color jitter, random grayscale, random Gaussian blur, and random horizontal flips. Let x1 and x2 be the two augmentations for one image. Let fq(⋅) be a query encoder, which is a model backbone composed with a two-layer non-linear MLP of dimensions 2048 and 128 respectively and let fk(⋅) be a key encoder, which has the same architecture as fq. x1 is encoded by fq and x2 is encoded by fk as follows: During each iteration of training, a dictionary of size K of image embeddings obtained from previous iterations is maintained (i.e., the dimensions of the dictionary are K × 128). The image embeddings in this dictionary are used as “negative” samples. The loss function for one image of a batch is defined as follows: (14) where θq are the parameters of fq, τ = 0.2 is a temperature hyperparameter, K = 65 536 is the number of “negative” samples, and are the embeddings of the negative samples (i.e., the augmentations for other images which are encoded using fk, and are stored in the dictionary). From Eq (14), we see that we want to maximize v ⋅ k0, which corresponds to maximizing the similarity between the embeddings of the two augmentations of an image. After each iteration of training, the dictionary of negative samples is enqueued with the embeddings from the most recent iteration, while embeddings that have been in the dictionary for the longest are dequeued. Finally, the parameters θk of fk are updated via a momentum update, as follows: where λ = 0.999 is the momentum coefficient. Note that only θq are updated with back-propagation. Our single-, dual-, and six-stream variants were trained using a batch size of 512 for 200 epochs using SGD with momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.06 was used, and the learning rate was decayed to 0.0 using a cosine schedule (with no warm-up). SimCLR [46]. The goal of this objective is conceptually similar to that of MoCov2, where the embeddings of augmentations of one image should be distinguishable from the embeddings of augmentations of other images. Thus, SimCLR is another instance of the class of contrastive objective functions. Similar to other contrastive objective functions, two image augmentations are first generated for each image in the ImageNet dataset (by using random cropping, random horizontal flips, random color jittering, random grayscaling and random Gaussian blurring). Let f(⋅) be the model backbone composed with a two-layer non-linear MLP of dimensions 2048 and 128 respectively. The two image augmentations are first embedded into a 128-dimensional space and normalized: The loss function for a single pair of augmentations of an image is defined as follows: (15) where τ = 0.1 is a temperature hyperparameter, N is the batch size, is equal to 1 if i ≠ 1 and 0 otherwise, and θ are the parameters of f. The loss defined in Eq (15) is computed for every pair of images in the batch (including their augmentations) and subsequently averaged. Our single-, dual-, and six-stream variants were trained using a batch size of 4096 for 200 epochs using layer-wise adaptive rate scaling (LARS; [72]) with momentum of 0.9, and weight decay of 10−6. An initial learning rate of 4.8 was used and decayed to 0.0 using a cosine schedule. A linear warm-up of 10 epochs was used for the learning rate with warm-up ratio of 0.0001. SimSiam [48]. The goal of this objective is to maximize the similarity between the embeddings of two augmentations of the same image. Thus, SimSiam is another instance of the class of contrastive objective functions. Two random image augmentations (i.e., random resized crop, random horizontal flip, color jitter, random grayscale, and random Gaussian blur) are first generated for each image in the ImageNet dataset. Let x1 and x2 be the two augmentations of the same image, f(⋅) be the model backbone, g(⋅) be a three-layer non-linear MLP, and h(⋅) be a two-layer non-linear MLP. The three-layer MLP has hidden dimensions of 2048, 2048, and 2048. The two-layer MLP has hidden dimensions of 512 and 2048 respectively. Let θ be the parameters for f, g, and h. The loss function for one image x of a batch is defined as follows (recall that x1 and x2 are two augmentations of one image): (16) where . Note that z1 and z2 are treated as constants in this loss function (i.e., the gradients are not back-propagated through z1 and z2). This “stop-gradient” method was key to the success of this objective function. Our single-, dual-, and six-stream variants were trained using a batch size of 512 for 100 epochs using SGD with momentum of 0.9, and weight decay of 0.0001. An initial learning rate of 0.1 was used, and the learning rate was decayed to 0.0 using a cosine schedule (with no warm-up). Barlow twins [49]. This method is inspired by Horace Barlow’s theory that sensory systems reduce redundancy in their inputs [73]. Let x1 and x2 be the two augmentations (random crops and color distortions) of the same image, f(⋅) be the model backbone, and let h(⋅) be a three-layer non-linear MLP (each of output dimension 8192). Given , where z1 = h ∘ f(x1) and z2 = h ∘ f(x2), this method proposes an objective function which tries to make the cross-correlation matrix computed from the twin embeddings z1 and z2 as close to the identity matrix as possible: (17) where b indexes batch examples and i, j index the embedding output dimension. We trained AlexNet (with 64 × 64 image inputs) with the recommended hyperparameters of λ = 0.0051, weight decay of 10−6, and batch size of 2048 with the LARS [72] optimizer employing learning rate warm-up of 10 epochs under a cosine schedule. We found that training stably completed after 58 epochs for this particular model architecture. VICReg [50]. Let x1 and x2 be the two augmentations (random crops and color distortions) of the same image, f(⋅) be the model backbone, and let h(⋅) be a three-layer non-linear MLP (each of output dimension 8192). Given , where z1 = h ∘ f(x1) and z2 = h ∘ f(x2), this method proposes an objective function that contains three terms: Invariance: minimizes the mean square distance between the embedding vectors. Variance: enforces the embedding vectors of samples within a batch to be different via a hinge loss to keep the standard deviation of each embedding variable to be above a given threshold (set to 1). Covariance: prevents informational collapse through highly correlated variables by attracting the covariances between every pair of embedding variables towards zero. We trained AlexNet (with 64 × 64 image inputs) with the recommended hyperparameters of weight decay of 10−6 and batch size of 2048 with the LARS [72] optimizer employing learning rate warm-up of 10 epochs under a cosine schedule, for 1000 training epochs total. Top-1 validation set performance Performance of primate models on 224 × 224 pixels and 64 × 64 pixels ImageNet. Here we report the top-1 validation set accuracy of models trained in a supervised manner on 64 × 64 pixels and 224 × 224 pixels ImageNet. Download: PPT PowerPoint slide PNG larger image TIFF original image Performance of StreamNet variants on 64 × 64 pixels CIFAR-10 and 64 × 64 pixels ImageNet. Here we report the top-1 validation set accuracy of our model variants trained in a supervised manner on 64 × 64 pixels CIFAR-10 and ImageNet. Download: PPT PowerPoint slide PNG larger image TIFF original image Transfer performance of StreamNet variants on 64 × 64 pixels ImageNet under linear evaluation for models trained with self-supervised objectives. In this subsection, we report the top-1 ImageNet validation set performance under linear evaluation for models trained with self-supervised objectives. After training each model on a self-supervised objective, the model backbone weights are then held fixed and a linear readout head is trained on top of the fixed model backbone. In the case where the objective function is “untrained”, model parameters were randomly initialized and held fixed while the linear readout head was trained. The image augmentations used during transfer learning were random cropping and random horizontal flipping. The linear readout for every self-supervised model was trained with the cross-entropy loss function (i.e., Eq (11) with C = 1000) for 100 epochs, which was minimized using SGD with momentum of 0.9, and weight decay of 10−9. The initial learning rate was set to 0.1 and reduced by a factor of 10 at epochs 30, 60, and 90. Reinforcement learning task A set of state-action-reward-state tuples (i.e., (st, at, rt, st+1)) were generated in prior work [53] and were used to train (in an offline fashion) the reinforcement learning (RL) agent. We used an offline RL algorithm known as critic-regularized regression [56]. Except for the visual encoder, the architecture of the RL agent was identical to that used by Wang et al. [56] (cf. Fig 3 in Wang et al. [56]). Four different visual encoders based on the AlexNet architecture were used: Contrastive ImageNet: AlexNet trained using ImageNet in a contrastive manner (Instance Recognition). Up to the first four layers are implanted into the virtual rodent as its visual system, as these were the layers that best matched mouse visual areas. This visual encoder is the best model of mouse visual cortex (Fig 2). Its weights were held fixed during training of the RL agent. Supervised ImageNet: AlexNet trained using ImageNet in a supervised manner. Up to the first four layers are implanted into the virtual rodent as its visual system, as these were the layers that best matched mouse visual areas. Its weights were held fixed during training of the RL agent. Contrastive Maze: AlexNet trained using the egocentric maze inputs from the virtual rodent reward-based navigation task. Up to the first four layers are implanted into the virtual rodent as its visual system, as these were the layers that best matched mouse visual areas. Its weights were then held fixed during training of the RL agent. Supervised Maze: The first four convolutional layers of AlexNet (ShallowNet) trained end-to-end on the virtual rodent reward-based navigation task. After training the agent until policy loss convergence by 10 000 steps (once), the agent was evaluated on 300 episodes. Each model was trained twice (i.e., two different random seeds), so each model was evaluated on 300 × 2 = 600 episodes in total and we report the average reward across all 600 episodes. Below, we report the average rewards (and s.e.m. across the 600 episodes) obtained when each of the visual backbones are used by the RL agent to perform the task. Evaluating model performance on downstream visual tasks To evaluate transfer performance on downstream visual tasks, we used the activations from the outputs of the shallow, intermediate, and deep modules of our StreamNet variants. We also included the average-pooling layer in all the variants (the model layer prior to the fully-connected readout layer). The dimensionality of the activations was then reduced to 1000 dimensions using principal components analysis (PCA), if the number of features exceeded 1000. PCA was not used if the number of features was less than or equal to 1000. A linear readout on these features was then used to perform five transfer visual tasks. For the first four object-centric visual tasks (object categorization, pose estimation, position estimation, and size estimation), we used a stimulus set that was used previously in the evaluation of neural network models of the primate visual system [17, 20, 74]. The stimulus set consists of objects in various poses (object rotations about the x, y, and z axes), positions (vertical and horizontal coordinates of the object), and sizes, each from eight categories. We then performed five-fold cross-validation on the training split of the medium and high variation image subsets (“Var3” and “Var6”, defined by Majaj et al. [75]) consisting of 3840 images, and computed the performance (metrics defined below) on the test split of the medium and high variation sets (“Var3” and “Var6”) consisting of 1280 images. Ten different category-balanced train-test splits were randomly selected, and the performance of the best model layer (averaged across train-test splits) was reported for each model. All images were resized to 64 × 64 pixels prior to fitting, to account for the visual acuity adjustment. The final non-object-centric task was texture recognition, using the Describable Textures Dataset [58]. Object categorization. We fit a linear support vector classifier to each model layer activations that were transformed via PCA. The regularization parameter, (18) was chosen by five-fold cross validation. The categories are Animals, Boats, Cars, Chairs, Faces, Fruits, Planes, and Tables. We reported the classification accuracy average across the ten train-test splits. Position estimation. We predicted both the vertical and the horizontal locations of the object center in the image. We used Ridge regression where the regularization parameter was selected from: (19) where C was selected from the list defined in (18). For each network, we reported the correlation averaged across both locations for the best model layer. Pose estimation. This task was similar to the position prediction task except that the prediction target were the z-axis (vertical axis) and the y-axis (horizontal axis) rotations, both of which ranged between −90 degrees and 90 degrees. The (0, 0, 0) angle was defined in a per-category basis and was chosen to make the (0, 0, 0) angle “semantically” consistent across different categories. We refer the reader to Hong et al. [55] for more details. We used Ridge regression with α chosen from the range in (19). Size estimation. The prediction target was the three-dimensional object scale, which was used to generate the image in the rendering process. This target varied between 0.625 to 1.6, which was a relative measure to a fixed canonical size of 1. When objects were at the canonical size, they occluded around 40% of the image on the longest axis. We used Ridge regression with α chosen from the range in (19). Texture classification. We trained linear readouts of the model layers on texture recognition using the Describable Textures Dataset [58], which consists of 5640 images organized according to 47 categories, with 120 images per category. We used ten category-balanced train-test splits, provided by their benchmark. Each split consisted of 3760 train-set images and 1880 test-set images. A linear support vector classifier was then fit with C chosen in the range (18). We reported the classification accuracy average across the ten train-test splits. Neural response datasets We used the Allen Brain Observatory Visual Coding dataset [15, 22] collected using both two-photon calcium imaging and Neuropixels from areas V1 (VISp), LM (VISl), AL (VISal), RL (VISrl), AM (VISam), and PM (VISpm) in mouse visual cortex. We focused on the natural scene stimuli, consisting of 118 images, each presented 50 times (i.e., 50 trials per image). We list the number of units and specimens for each dataset in Table 1, after units are selected, according to the following procedure: For the calcium imaging data, we used a similar unit selection criterion as in Conwell et al. [18], where we sub-selected units that attain a Spearman-Brown corrected split-half consistency of at least 0.3 (averaged across 100 bootstrapped trials), and whose peak responses to their preferred images are not significantly modulated by the mouse’s running speed during stimulus presentation (p > 0.05). For the Neuropixels dataset, we separately averaged, for each specimen and each visual area, the temporal response (at the level of 10-ms bins up to 250 ms) on the largest contiguous time interval when the median (across the population of units in that specimen) split-half consistency reached at least 0.3. This procedure helps to select the most internally-consistent units in their temporally-averaged response, and accounts for the fact that different specimens have different time courses along which their population response becomes reliable. Finally, after subselecting units according to the above criteria for both datasets, we only keep specimens that have at least the 75th percentile number of units among all specimens for that given visual area. This final step helped to ensure we have enough internally-consistent units per specimen for the inter-animal consistency estimation (derived in the “Inter-Animal Consistency Derivation” section). Noise-corrected neural predictivity Linear regression. When we perform neural fits, we choose a random 50% set of natural scene images (59 images in total) to train the regression, and the remaining 50% to use as a test set (59 images in total), across ten train-test splits total. For Ridge regression, we use an α = 1, following the sklearn.linear_model convention. PLS regression was performed with 25 components, as in prior work (e.g., [8, 17]). When we perform regression with the One-to-One mapping, as in Fig 1B, we identify the top correlated (via Pearson correlation on the training images) unit in the source population for each target unit. Once that source unit has been identified, we then fix it for that particular train-test split, evaluated on the remaining 50% of images. Motivated by the justification given in the “Inter-Animal Consistency Derivation” section for the noise correction in the inter-animal consistency, the noise correction of the model to neural response regression is a special case of the quantity defined in the “Multiple Animals” section, where now the source animal is replaced by model features, separately fit to each target animal (from the set of available animals ). Let L be the set of model layers, let rℓ be the set of model responses at model layer ℓ ∈ L, M be the mapping, and let s be the trial-averaged pseudo-population response where the average is taken over 100 bootstrapped split-half trials, ⊕ denotes concatenation of units across animals followed by the median value across units, and denotes the Pearson correlation of the two quantities. denotes the Spearman-Brown corrected value of the original quantity (see the “Spearman-Brown Correction” section). Prior to obtaining the model features of the stimuli for linear regression, we preprocessed each stimulus using the image transforms used on the validation set during model training, resizing the shortest edge of the stimulus in both cases to 64 pixels, preserving the aspect ratio of the input stimulus. Specifically, for models trained using the ImageNet dataset, we first resized the shortest edge of the stimulus to 256 pixels, center-cropped the image to 224 × 224 pixels, and finally resized the stimulus to 64 × 64 pixels. For models trained using the CIFAR-10 dataset, this resizing yielded a 64 × 81 pixels stimulus. Representational Similarity Analysis (RSA). In line with prior work [18, 36], we also used representational similarity analysis (RSA; [30]) to compare models to neural responses, as well as to compare animals to each other. Specifically, we compared (via Pearson correlation) only the upper-right triangles of the representational dissimilarity matrices (RDMs), excluding the diagonals to avoid illusory effects [69]. For each visual area and a given model, we defined the predictivity of the model for that area to be the maximum RSA score across model layers after the suitable noise correction is applied, which is defined as follows. Let rℓ be the model responses at model layer ℓ and let s be the trial-averaged pseudo-population response (i.e., responses aggregated across specimens). The metric used here is a specific instance of Eq (10), where the single source animal is the trial-wise, deterministic model features (which have a mapping consistency of 1 as a result) and a single target animal , which is the pseudo-population response: (1) where L is the set of model layers, are the animal’s responses for two halves of the trials (and averaged across the trials dimension), the average is computed over 100 bootstrapped split-half trials, and denotes Spearman-Brown correction applied to the internal consistency quantity, , defined in the “Spearman-Brown Correction” section. If the fits are performed separately for each animal, then corresponds to each animal among those for a given visual area (defined by the set ), and we compute the median across animals : (2) Similar to the above, Spearman-Brown correction is applied to the internal consistency quantity, . Linear regression. When we perform neural fits, we choose a random 50% set of natural scene images (59 images in total) to train the regression, and the remaining 50% to use as a test set (59 images in total), across ten train-test splits total. For Ridge regression, we use an α = 1, following the sklearn.linear_model convention. PLS regression was performed with 25 components, as in prior work (e.g., [8, 17]). When we perform regression with the One-to-One mapping, as in Fig 1B, we identify the top correlated (via Pearson correlation on the training images) unit in the source population for each target unit. Once that source unit has been identified, we then fix it for that particular train-test split, evaluated on the remaining 50% of images. Motivated by the justification given in the “Inter-Animal Consistency Derivation” section for the noise correction in the inter-animal consistency, the noise correction of the model to neural response regression is a special case of the quantity defined in the “Multiple Animals” section, where now the source animal is replaced by model features, separately fit to each target animal (from the set of available animals ). Let L be the set of model layers, let rℓ be the set of model responses at model layer ℓ ∈ L, M be the mapping, and let s be the trial-averaged pseudo-population response where the average is taken over 100 bootstrapped split-half trials, ⊕ denotes concatenation of units across animals followed by the median value across units, and denotes the Pearson correlation of the two quantities. denotes the Spearman-Brown corrected value of the original quantity (see the “Spearman-Brown Correction” section). Prior to obtaining the model features of the stimuli for linear regression, we preprocessed each stimulus using the image transforms used on the validation set during model training, resizing the shortest edge of the stimulus in both cases to 64 pixels, preserving the aspect ratio of the input stimulus. Specifically, for models trained using the ImageNet dataset, we first resized the shortest edge of the stimulus to 256 pixels, center-cropped the image to 224 × 224 pixels, and finally resized the stimulus to 64 × 64 pixels. For models trained using the CIFAR-10 dataset, this resizing yielded a 64 × 81 pixels stimulus. Representational Similarity Analysis (RSA). In line with prior work [18, 36], we also used representational similarity analysis (RSA; [30]) to compare models to neural responses, as well as to compare animals to each other. Specifically, we compared (via Pearson correlation) only the upper-right triangles of the representational dissimilarity matrices (RDMs), excluding the diagonals to avoid illusory effects [69]. For each visual area and a given model, we defined the predictivity of the model for that area to be the maximum RSA score across model layers after the suitable noise correction is applied, which is defined as follows. Let rℓ be the model responses at model layer ℓ and let s be the trial-averaged pseudo-population response (i.e., responses aggregated across specimens). The metric used here is a specific instance of Eq (10), where the single source animal is the trial-wise, deterministic model features (which have a mapping consistency of 1 as a result) and a single target animal , which is the pseudo-population response: (1) where L is the set of model layers, are the animal’s responses for two halves of the trials (and averaged across the trials dimension), the average is computed over 100 bootstrapped split-half trials, and denotes Spearman-Brown correction applied to the internal consistency quantity, , defined in the “Spearman-Brown Correction” section. If the fits are performed separately for each animal, then corresponds to each animal among those for a given visual area (defined by the set ), and we compute the median across animals : (2) Similar to the above, Spearman-Brown correction is applied to the internal consistency quantity, . Inter-animal consistency derivation Single animal pair. Suppose we have neural responses from two animals and . Let be the vector of true responses (either at a given time bin or averaged across a set of time bins) of animal on stimulus set . Of course, we only receive noisy observations of , so let be the jth set of n trials of . Finally, let M(x; y)i be the predictions of a mapping M (e.g., PLS) when trained on input x to match output y and tested on stimulus set i. For example, is the prediction of mapping M on the test set stimuli trained to match the true neural responses of animal given, as input, the true neural responses of animal on the train set stimuli. Similarly, is the prediction of mapping M on the test set stimuli trained to match the trial-average of noisy sample 1 on the train set stimuli of animal given, as input, the trial-average of noisy sample 1 on the train set stimuli of animal . With these definitions in hand, the inter-animal mapping consistency from animal to animal corresponds to the following true quantity to be estimated: (3) where is the Pearson correlation across a stimulus set. In what follows, we will argue that Eq (3) can be approximated with the following ratio of measurable quantities, where we split in half and average the noisy trial observations, indexed by 1 and by 2: (4) In words, the inter-animal consistency (i.e., the quantity on the left side of Eq (4)) corresponds to the predictivity of the mapping on the test set stimuli from animal to animal on two different (averaged) halves of noisy trials (i.e., the numerator on the right side of Eq (4)), corrected by the square root of the mapping reliability on animal ’s responses to the test set stimuli on two different halves of noisy trials multiplied by the internal consistency of animal . We justify the approximation in Eq (4) by gradually replacing the true quantities (t) by their measurable estimates (s), starting from the original quantity in Eq (3). First, we make the approximation that: (5) by the transitivity of positive correlations (which is a reasonable assumption when the number of stimuli is large). Next, by transitivity and normality assumptions in the structure of the noisy estimates and since the number of trials (n) between the two sets is the same, we have that: (6) In words, Eq (6) states that the correlation between the average of two sets of noisy observations of n trials each is approximately the square of the correlation between the true value and average of one set of n noisy trials. Therefore, combining Eqs (5) and (6), it follows that: (7) From the right side of Eq (7), we can see that we have removed , but we still need to remove the term, as this term still contains unmeasurable (i.e., true) quantities. We apply the same two steps, described above, by analogy, though these approximations may not always be true (they are, however, true for Gaussian noise): which taken together implies the following: (8) Eqs (7) and (8) together imply the final estimated quantity given in Eq (4). Multiple animals. For multiple animals, we consider the average of the true quantity for each target in in Eq (3) across source animals in the ordered pair (, ) of animals and : We also bootstrap across trials, and have multiple train/test splits, in which case the average on the right hand side of the equation includes averages across these as well. Note that each neuron in our analysis will have this single average value associated with it when it was a target animal (), averaged over source animals/subsampled source neurons, bootstrapped trials, and train/test splits. This yields a vector of these average values, which we can take median and standard error of the mean (s.e.m.) over, as we do with standard explained variance metrics. RSA. We can extend the above derivations to other commonly used metrics for comparing representations that involve correlation. Since , then the corresponding quantity in Eq (4) analogously (by transitivity of positive correlations) becomes: (9) Note that in this case, each animal (rather than neuron) in our analysis will have this single average value associated with it when it was a target animal () (since RSA is computed over images and neurons), where the average is over source animals/subsampled source neurons, bootstrapped trials, and train/test splits. This yields a vector of these average values, which we can take median and s.e.m. over, across animals . For RSA, we can use the identity mapping (since RSA is computed over neurons as well, the number of neurons between source and target animal can be different to compare them with the identity mapping). As parameters are not fit, we can choose , so that Eq (9) becomes: (10) Pooled source animal. Often times, we may not have enough neurons per animal to ensure that the estimated inter-animal consistency in our data closely matches the “true” inter-animal consistency. In order to address this issue, we holdout one animal at a time and compare it to the pseudo-population aggregated across units from the remaining animals, as opposed to computing the consistencies in a pairwise fashion. Thus, is still the target heldout animal as in the pairwise case, but now the average over is over a sole “pooled” source animal constructed from the pseudo-population of the remaining animals. Spearman-brown correction. The Spearman-Brown correction can be applied to each of the terms in the denominator individually, as they are each correlations of observations from half the trials of the same underlying process to itself (unlike the numerator). Namely, Analogously, since , then we define Single animal pair. Suppose we have neural responses from two animals and . Let be the vector of true responses (either at a given time bin or averaged across a set of time bins) of animal on stimulus set . Of course, we only receive noisy observations of , so let be the jth set of n trials of . Finally, let M(x; y)i be the predictions of a mapping M (e.g., PLS) when trained on input x to match output y and tested on stimulus set i. For example, is the prediction of mapping M on the test set stimuli trained to match the true neural responses of animal given, as input, the true neural responses of animal on the train set stimuli. Similarly, is the prediction of mapping M on the test set stimuli trained to match the trial-average of noisy sample 1 on the train set stimuli of animal given, as input, the trial-average of noisy sample 1 on the train set stimuli of animal . With these definitions in hand, the inter-animal mapping consistency from animal to animal corresponds to the following true quantity to be estimated: (3) where is the Pearson correlation across a stimulus set. In what follows, we will argue that Eq (3) can be approximated with the following ratio of measurable quantities, where we split in half and average the noisy trial observations, indexed by 1 and by 2: (4) In words, the inter-animal consistency (i.e., the quantity on the left side of Eq (4)) corresponds to the predictivity of the mapping on the test set stimuli from animal to animal on two different (averaged) halves of noisy trials (i.e., the numerator on the right side of Eq (4)), corrected by the square root of the mapping reliability on animal ’s responses to the test set stimuli on two different halves of noisy trials multiplied by the internal consistency of animal . We justify the approximation in Eq (4) by gradually replacing the true quantities (t) by their measurable estimates (s), starting from the original quantity in Eq (3). First, we make the approximation that: (5) by the transitivity of positive correlations (which is a reasonable assumption when the number of stimuli is large). Next, by transitivity and normality assumptions in the structure of the noisy estimates and since the number of trials (n) between the two sets is the same, we have that: (6) In words, Eq (6) states that the correlation between the average of two sets of noisy observations of n trials each is approximately the square of the correlation between the true value and average of one set of n noisy trials. Therefore, combining Eqs (5) and (6), it follows that: (7) From the right side of Eq (7), we can see that we have removed , but we still need to remove the term, as this term still contains unmeasurable (i.e., true) quantities. We apply the same two steps, described above, by analogy, though these approximations may not always be true (they are, however, true for Gaussian noise): which taken together implies the following: (8) Eqs (7) and (8) together imply the final estimated quantity given in Eq (4). Multiple animals. For multiple animals, we consider the average of the true quantity for each target in in Eq (3) across source animals in the ordered pair (, ) of animals and : We also bootstrap across trials, and have multiple train/test splits, in which case the average on the right hand side of the equation includes averages across these as well. Note that each neuron in our analysis will have this single average value associated with it when it was a target animal (), averaged over source animals/subsampled source neurons, bootstrapped trials, and train/test splits. This yields a vector of these average values, which we can take median and standard error of the mean (s.e.m.) over, as we do with standard explained variance metrics. RSA. We can extend the above derivations to other commonly used metrics for comparing representations that involve correlation. Since , then the corresponding quantity in Eq (4) analogously (by transitivity of positive correlations) becomes: (9) Note that in this case, each animal (rather than neuron) in our analysis will have this single average value associated with it when it was a target animal () (since RSA is computed over images and neurons), where the average is over source animals/subsampled source neurons, bootstrapped trials, and train/test splits. This yields a vector of these average values, which we can take median and s.e.m. over, across animals . For RSA, we can use the identity mapping (since RSA is computed over neurons as well, the number of neurons between source and target animal can be different to compare them with the identity mapping). As parameters are not fit, we can choose , so that Eq (9) becomes: (10) Pooled source animal. Often times, we may not have enough neurons per animal to ensure that the estimated inter-animal consistency in our data closely matches the “true” inter-animal consistency. In order to address this issue, we holdout one animal at a time and compare it to the pseudo-population aggregated across units from the remaining animals, as opposed to computing the consistencies in a pairwise fashion. Thus, is still the target heldout animal as in the pairwise case, but now the average over is over a sole “pooled” source animal constructed from the pseudo-population of the remaining animals. Spearman-brown correction. The Spearman-Brown correction can be applied to each of the terms in the denominator individually, as they are each correlations of observations from half the trials of the same underlying process to itself (unlike the numerator). Namely, Analogously, since , then we define StreamNet architecture variants We developed shallower, multiple-streamed architectures for mouse visual cortex, shown in Fig 2A. There are three main modules in our architecture: shallow, intermediate, and deep. The shallow and deep modules each consist of one convolutional layer and the intermediate module consists of a block of two convolutional layers. Thus, the longest length of the computational graph, excluding the readout module, is four (i.e., 1 + 2 + 1). Depending on the number of parallel streams in the model, the intermediate module would contain multiple branches (in parallel), each receiving input from the shallow module. The outputs of the intermediate modules are then passed through one convolutional operation (deep module). Finally, the outputs of each parallel branch would be summed together, concatenated across the channels dimension, and used as input for the readout module. The readout module consists of an (adaptive) average pooling operation that upsamples the inputs to 6 × 6 feature maps. These feature maps are then flattened, so that each image only has a single feature vector. These feature vectors are then fed into a linear layer (i.e., fully-connected layer) either for classification or for embedding them into a lower-dimensional space for the contrastive losses. The table below describes the parameters of three model variants, each containing one (N = 1), two (N = 2), or six (N = 6) parallel branches. Note that one convolutional layer is denoted by a tuple: (number of filters, filter size, stride, padding). The first convolutional layer has max pooling of stride 2, as in AlexNet. A block of convolutional layers is denoted by a list of tuples, where each tuple in the list corresponds to a single convolutional layer. When a list of tuples is followed by “×N”, this means that the convolutional parameters for each of the N parallel branches are the same. Neural network training objectives In this section, we briefly describe the supervised and self-supervised objectives that were used to train our models. Download: PPT PowerPoint slide PNG larger image TIFF original image Supervised training objective. The loss function used in supervised training is the cross-entropy loss, defined as follows: (11) where N is the batch size, C is the number of categories for the dataset, are the model outputs (i.e., logits) for the N images, are the logits for the ith image, ci ∈ [0, C − 1] is the category index of the ith image (zero-indexed), and θ are the model parameters. Eq (11) was minimized using stochastic gradient descent (SGD) with momentum [70]. ImageNet [19]. This dataset contains approximately 1.3 million images in the train set and 50 000 images in the validation set. Each image was previously labeled into C = 1000 distinct categories. CIFAR-10 [39]. This dataset contains 50 000 images in the train set and 10 000 images in the validation set. Each image was previously labeled into C = 10 distinct categories. Depth prediction [40]. The goal of this objective is to predict the depth map of an image. We used a synthetically generated dataset of images known as PBRNet [40]. It contains approximately 500 000 images and their associated depth maps. Similar to the loss function used in the sparse autoencoder objective, we used a mean-squared loss to train the models. The output (i.e., depth map) was generated using a mirrored version of each of our StreamNet variants. In order to generate the depth map, we appended one final convolutional layer onto the output of the mirrored architecture in order to downsample the three image channels to one image channel. During training, random crops of size 224 × 224 pixels were applied to the image and depth map (which were both subsequently resized to 64 × 64 pixels). In addition, both the image and depth map were flipped horizontally with probability 0.5. Finally, prior to the application of the loss function, each depth map was normalized such that the mean and standard deviation across pixels were zero and one respectively. Each of our single-, dual-, and six-stream variants were trained using a batch size of 256 for 50 epochs using SGD with momentum of 0.9, and weight decay of 0.0001. The initial learning rate was set to 10−4 and was decayed by a factor of 10 at epochs 15, 30, and 45. Self-supervised training objectives. Sparse autoencoder [42]. The goal of this objective is to reconstruct an image from a sparse image embedding. In order to generate an image reconstruction, we used a mirrored version of each of our StreamNet variants. Concretely, the loss function was defined as follows: (12) where is the image embedding, f is the (mirrored) model, f(x) is the image reconstruction, x is a 64 × 64 pixels image, λ is the regularization coefficient, and θ are the model parameters. Our single-, dual-, and six-stream variants were trained using a batch size of 256 for 100 epochs using SGD with momentum of 0.9 and weight decay of 0.0005. The initial learning rate was set to 0.01 for the single- and dual-stream variants and was set to 0.001 for the six-stream variant. The learning rate was decayed by a factor of 10 at epochs 30, 60, and 90. For all the StreamNet variants, the embedding dimension was set to 128 and the regularization coefficient was set to 0.0005. RotNet [44]. The goal of this objective is to predict the rotation of an image. Each image of the ImageNet dataset was rotated four ways (0°, 90°, 180°, 270°) and the four rotation angles were used as “pseudo-labels” or “categories”. The cross-entropy loss was used with these pseudo-labels as the training objective (i.e., Eq (11) with C = 4). Our single-, dual-, and six-stream variants were trained using a batch size of 192 (which is effectively a batch size of 192 × 4 = 768 due to the four rotations for each image) for 50 epochs using SGD with Nesterov momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.01 was decayed by a factor of 10 at epochs 15, 30, and 45. Instance recognition [45]. The goal of this objective is to be able to differentiate between embeddings of augmentations of one image from embeddings of augmentations of other images. Thus, this objective function is an instance of the class of contrastive objective functions. A random image augmentation is first performed on each image of the ImageNet dataset (random resized cropping, random grayscale, color jitter, and random horizontal flip). Let x be an image augmentation, and f(⋅) be the model backbone composed with a one-layer linear multi-layer perceptron (MLP) of size 128. The image is then embedded onto a 128-dimensional unit-sphere as follows: Throughout model training, a memory bank containing embeddings for each image in the train set is maintained (i.e., the size of the memory bank is the same as the size of the train set). The embedding z will be “compared” to a subsample of these embeddings. Concretely, the loss function for one image x is defined as follows: (13) where is the embedding for image x that is currently stored in the memory bank, N is the size of the memory bank, m = 4096 is the number of “negative” samples used, are the negative embeddings sampled from the memory bank uniformly, Z is some normalization constant, τ = 0.07 is a temperature hyperparameter, and θ are the parameters of f. From Eq (13), we see that we want to maximize h(v), which corresponds to maximizing the similarity between v and z (recall that z is the embedding for x obtained using f). We can also see that we want to maximize 1 − h(vj) (or minimize h(vj)). This would correspond to minimizing the similarity between vj and z (recall that vj are the negative embeddings). After each iteration of training, the embeddings for the current batch are used to update the memory bank (at their corresponding positions in the memory bank) via a momentum update. Concretely, for image x, its embedding in the memory bank v is updated using its current embedding z as follows: where λ = 0.5 is the momentum coefficient. The second operation on v is used to project v back onto the 128-dimensional unit sphere. Our single-, dual-, and six-stream variants were trained using a batch size of 256 for 200 epochs using SGD with momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.03 was decayed by a factor of 10 at epochs 120 and 160. MoCov2 [47, 71]. The goal of this objective is to be able to distinguish augmentations of one image (i.e., by labeling them as “positive”) from augmentations of other images (i.e., by labeling them as “negative”). Intuitively, embeddings of different augmentations of the same image should be more “similar” to each other than to embeddings of augmentations of other images. Thus, this algorithm is another instance of the class of contrastive objective functions and is similar conceptually to instance recognition. Two image augmentations are first generated for each image in the ImageNet dataset by applying random resized cropping, color jitter, random grayscale, random Gaussian blur, and random horizontal flips. Let x1 and x2 be the two augmentations for one image. Let fq(⋅) be a query encoder, which is a model backbone composed with a two-layer non-linear MLP of dimensions 2048 and 128 respectively and let fk(⋅) be a key encoder, which has the same architecture as fq. x1 is encoded by fq and x2 is encoded by fk as follows: During each iteration of training, a dictionary of size K of image embeddings obtained from previous iterations is maintained (i.e., the dimensions of the dictionary are K × 128). The image embeddings in this dictionary are used as “negative” samples. The loss function for one image of a batch is defined as follows: (14) where θq are the parameters of fq, τ = 0.2 is a temperature hyperparameter, K = 65 536 is the number of “negative” samples, and are the embeddings of the negative samples (i.e., the augmentations for other images which are encoded using fk, and are stored in the dictionary). From Eq (14), we see that we want to maximize v ⋅ k0, which corresponds to maximizing the similarity between the embeddings of the two augmentations of an image. After each iteration of training, the dictionary of negative samples is enqueued with the embeddings from the most recent iteration, while embeddings that have been in the dictionary for the longest are dequeued. Finally, the parameters θk of fk are updated via a momentum update, as follows: where λ = 0.999 is the momentum coefficient. Note that only θq are updated with back-propagation. Our single-, dual-, and six-stream variants were trained using a batch size of 512 for 200 epochs using SGD with momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.06 was used, and the learning rate was decayed to 0.0 using a cosine schedule (with no warm-up). SimCLR [46]. The goal of this objective is conceptually similar to that of MoCov2, where the embeddings of augmentations of one image should be distinguishable from the embeddings of augmentations of other images. Thus, SimCLR is another instance of the class of contrastive objective functions. Similar to other contrastive objective functions, two image augmentations are first generated for each image in the ImageNet dataset (by using random cropping, random horizontal flips, random color jittering, random grayscaling and random Gaussian blurring). Let f(⋅) be the model backbone composed with a two-layer non-linear MLP of dimensions 2048 and 128 respectively. The two image augmentations are first embedded into a 128-dimensional space and normalized: The loss function for a single pair of augmentations of an image is defined as follows: (15) where τ = 0.1 is a temperature hyperparameter, N is the batch size, is equal to 1 if i ≠ 1 and 0 otherwise, and θ are the parameters of f. The loss defined in Eq (15) is computed for every pair of images in the batch (including their augmentations) and subsequently averaged. Our single-, dual-, and six-stream variants were trained using a batch size of 4096 for 200 epochs using layer-wise adaptive rate scaling (LARS; [72]) with momentum of 0.9, and weight decay of 10−6. An initial learning rate of 4.8 was used and decayed to 0.0 using a cosine schedule. A linear warm-up of 10 epochs was used for the learning rate with warm-up ratio of 0.0001. SimSiam [48]. The goal of this objective is to maximize the similarity between the embeddings of two augmentations of the same image. Thus, SimSiam is another instance of the class of contrastive objective functions. Two random image augmentations (i.e., random resized crop, random horizontal flip, color jitter, random grayscale, and random Gaussian blur) are first generated for each image in the ImageNet dataset. Let x1 and x2 be the two augmentations of the same image, f(⋅) be the model backbone, g(⋅) be a three-layer non-linear MLP, and h(⋅) be a two-layer non-linear MLP. The three-layer MLP has hidden dimensions of 2048, 2048, and 2048. The two-layer MLP has hidden dimensions of 512 and 2048 respectively. Let θ be the parameters for f, g, and h. The loss function for one image x of a batch is defined as follows (recall that x1 and x2 are two augmentations of one image): (16) where . Note that z1 and z2 are treated as constants in this loss function (i.e., the gradients are not back-propagated through z1 and z2). This “stop-gradient” method was key to the success of this objective function. Our single-, dual-, and six-stream variants were trained using a batch size of 512 for 100 epochs using SGD with momentum of 0.9, and weight decay of 0.0001. An initial learning rate of 0.1 was used, and the learning rate was decayed to 0.0 using a cosine schedule (with no warm-up). Barlow twins [49]. This method is inspired by Horace Barlow’s theory that sensory systems reduce redundancy in their inputs [73]. Let x1 and x2 be the two augmentations (random crops and color distortions) of the same image, f(⋅) be the model backbone, and let h(⋅) be a three-layer non-linear MLP (each of output dimension 8192). Given , where z1 = h ∘ f(x1) and z2 = h ∘ f(x2), this method proposes an objective function which tries to make the cross-correlation matrix computed from the twin embeddings z1 and z2 as close to the identity matrix as possible: (17) where b indexes batch examples and i, j index the embedding output dimension. We trained AlexNet (with 64 × 64 image inputs) with the recommended hyperparameters of λ = 0.0051, weight decay of 10−6, and batch size of 2048 with the LARS [72] optimizer employing learning rate warm-up of 10 epochs under a cosine schedule. We found that training stably completed after 58 epochs for this particular model architecture. VICReg [50]. Let x1 and x2 be the two augmentations (random crops and color distortions) of the same image, f(⋅) be the model backbone, and let h(⋅) be a three-layer non-linear MLP (each of output dimension 8192). Given , where z1 = h ∘ f(x1) and z2 = h ∘ f(x2), this method proposes an objective function that contains three terms: Invariance: minimizes the mean square distance between the embedding vectors. Variance: enforces the embedding vectors of samples within a batch to be different via a hinge loss to keep the standard deviation of each embedding variable to be above a given threshold (set to 1). Covariance: prevents informational collapse through highly correlated variables by attracting the covariances between every pair of embedding variables towards zero. We trained AlexNet (with 64 × 64 image inputs) with the recommended hyperparameters of weight decay of 10−6 and batch size of 2048 with the LARS [72] optimizer employing learning rate warm-up of 10 epochs under a cosine schedule, for 1000 training epochs total. Supervised training objective. The loss function used in supervised training is the cross-entropy loss, defined as follows: (11) where N is the batch size, C is the number of categories for the dataset, are the model outputs (i.e., logits) for the N images, are the logits for the ith image, ci ∈ [0, C − 1] is the category index of the ith image (zero-indexed), and θ are the model parameters. Eq (11) was minimized using stochastic gradient descent (SGD) with momentum [70]. ImageNet [19]. This dataset contains approximately 1.3 million images in the train set and 50 000 images in the validation set. Each image was previously labeled into C = 1000 distinct categories. CIFAR-10 [39]. This dataset contains 50 000 images in the train set and 10 000 images in the validation set. Each image was previously labeled into C = 10 distinct categories. Depth prediction [40]. The goal of this objective is to predict the depth map of an image. We used a synthetically generated dataset of images known as PBRNet [40]. It contains approximately 500 000 images and their associated depth maps. Similar to the loss function used in the sparse autoencoder objective, we used a mean-squared loss to train the models. The output (i.e., depth map) was generated using a mirrored version of each of our StreamNet variants. In order to generate the depth map, we appended one final convolutional layer onto the output of the mirrored architecture in order to downsample the three image channels to one image channel. During training, random crops of size 224 × 224 pixels were applied to the image and depth map (which were both subsequently resized to 64 × 64 pixels). In addition, both the image and depth map were flipped horizontally with probability 0.5. Finally, prior to the application of the loss function, each depth map was normalized such that the mean and standard deviation across pixels were zero and one respectively. Each of our single-, dual-, and six-stream variants were trained using a batch size of 256 for 50 epochs using SGD with momentum of 0.9, and weight decay of 0.0001. The initial learning rate was set to 10−4 and was decayed by a factor of 10 at epochs 15, 30, and 45. Self-supervised training objectives. Sparse autoencoder [42]. The goal of this objective is to reconstruct an image from a sparse image embedding. In order to generate an image reconstruction, we used a mirrored version of each of our StreamNet variants. Concretely, the loss function was defined as follows: (12) where is the image embedding, f is the (mirrored) model, f(x) is the image reconstruction, x is a 64 × 64 pixels image, λ is the regularization coefficient, and θ are the model parameters. Our single-, dual-, and six-stream variants were trained using a batch size of 256 for 100 epochs using SGD with momentum of 0.9 and weight decay of 0.0005. The initial learning rate was set to 0.01 for the single- and dual-stream variants and was set to 0.001 for the six-stream variant. The learning rate was decayed by a factor of 10 at epochs 30, 60, and 90. For all the StreamNet variants, the embedding dimension was set to 128 and the regularization coefficient was set to 0.0005. RotNet [44]. The goal of this objective is to predict the rotation of an image. Each image of the ImageNet dataset was rotated four ways (0°, 90°, 180°, 270°) and the four rotation angles were used as “pseudo-labels” or “categories”. The cross-entropy loss was used with these pseudo-labels as the training objective (i.e., Eq (11) with C = 4). Our single-, dual-, and six-stream variants were trained using a batch size of 192 (which is effectively a batch size of 192 × 4 = 768 due to the four rotations for each image) for 50 epochs using SGD with Nesterov momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.01 was decayed by a factor of 10 at epochs 15, 30, and 45. Instance recognition [45]. The goal of this objective is to be able to differentiate between embeddings of augmentations of one image from embeddings of augmentations of other images. Thus, this objective function is an instance of the class of contrastive objective functions. A random image augmentation is first performed on each image of the ImageNet dataset (random resized cropping, random grayscale, color jitter, and random horizontal flip). Let x be an image augmentation, and f(⋅) be the model backbone composed with a one-layer linear multi-layer perceptron (MLP) of size 128. The image is then embedded onto a 128-dimensional unit-sphere as follows: Throughout model training, a memory bank containing embeddings for each image in the train set is maintained (i.e., the size of the memory bank is the same as the size of the train set). The embedding z will be “compared” to a subsample of these embeddings. Concretely, the loss function for one image x is defined as follows: (13) where is the embedding for image x that is currently stored in the memory bank, N is the size of the memory bank, m = 4096 is the number of “negative” samples used, are the negative embeddings sampled from the memory bank uniformly, Z is some normalization constant, τ = 0.07 is a temperature hyperparameter, and θ are the parameters of f. From Eq (13), we see that we want to maximize h(v), which corresponds to maximizing the similarity between v and z (recall that z is the embedding for x obtained using f). We can also see that we want to maximize 1 − h(vj) (or minimize h(vj)). This would correspond to minimizing the similarity between vj and z (recall that vj are the negative embeddings). After each iteration of training, the embeddings for the current batch are used to update the memory bank (at their corresponding positions in the memory bank) via a momentum update. Concretely, for image x, its embedding in the memory bank v is updated using its current embedding z as follows: where λ = 0.5 is the momentum coefficient. The second operation on v is used to project v back onto the 128-dimensional unit sphere. Our single-, dual-, and six-stream variants were trained using a batch size of 256 for 200 epochs using SGD with momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.03 was decayed by a factor of 10 at epochs 120 and 160. MoCov2 [47, 71]. The goal of this objective is to be able to distinguish augmentations of one image (i.e., by labeling them as “positive”) from augmentations of other images (i.e., by labeling them as “negative”). Intuitively, embeddings of different augmentations of the same image should be more “similar” to each other than to embeddings of augmentations of other images. Thus, this algorithm is another instance of the class of contrastive objective functions and is similar conceptually to instance recognition. Two image augmentations are first generated for each image in the ImageNet dataset by applying random resized cropping, color jitter, random grayscale, random Gaussian blur, and random horizontal flips. Let x1 and x2 be the two augmentations for one image. Let fq(⋅) be a query encoder, which is a model backbone composed with a two-layer non-linear MLP of dimensions 2048 and 128 respectively and let fk(⋅) be a key encoder, which has the same architecture as fq. x1 is encoded by fq and x2 is encoded by fk as follows: During each iteration of training, a dictionary of size K of image embeddings obtained from previous iterations is maintained (i.e., the dimensions of the dictionary are K × 128). The image embeddings in this dictionary are used as “negative” samples. The loss function for one image of a batch is defined as follows: (14) where θq are the parameters of fq, τ = 0.2 is a temperature hyperparameter, K = 65 536 is the number of “negative” samples, and are the embeddings of the negative samples (i.e., the augmentations for other images which are encoded using fk, and are stored in the dictionary). From Eq (14), we see that we want to maximize v ⋅ k0, which corresponds to maximizing the similarity between the embeddings of the two augmentations of an image. After each iteration of training, the dictionary of negative samples is enqueued with the embeddings from the most recent iteration, while embeddings that have been in the dictionary for the longest are dequeued. Finally, the parameters θk of fk are updated via a momentum update, as follows: where λ = 0.999 is the momentum coefficient. Note that only θq are updated with back-propagation. Our single-, dual-, and six-stream variants were trained using a batch size of 512 for 200 epochs using SGD with momentum of 0.9, and weight decay of 0.0005. An initial learning rate of 0.06 was used, and the learning rate was decayed to 0.0 using a cosine schedule (with no warm-up). SimCLR [46]. The goal of this objective is conceptually similar to that of MoCov2, where the embeddings of augmentations of one image should be distinguishable from the embeddings of augmentations of other images. Thus, SimCLR is another instance of the class of contrastive objective functions. Similar to other contrastive objective functions, two image augmentations are first generated for each image in the ImageNet dataset (by using random cropping, random horizontal flips, random color jittering, random grayscaling and random Gaussian blurring). Let f(⋅) be the model backbone composed with a two-layer non-linear MLP of dimensions 2048 and 128 respectively. The two image augmentations are first embedded into a 128-dimensional space and normalized: The loss function for a single pair of augmentations of an image is defined as follows: (15) where τ = 0.1 is a temperature hyperparameter, N is the batch size, is equal to 1 if i ≠ 1 and 0 otherwise, and θ are the parameters of f. The loss defined in Eq (15) is computed for every pair of images in the batch (including their augmentations) and subsequently averaged. Our single-, dual-, and six-stream variants were trained using a batch size of 4096 for 200 epochs using layer-wise adaptive rate scaling (LARS; [72]) with momentum of 0.9, and weight decay of 10−6. An initial learning rate of 4.8 was used and decayed to 0.0 using a cosine schedule. A linear warm-up of 10 epochs was used for the learning rate with warm-up ratio of 0.0001. SimSiam [48]. The goal of this objective is to maximize the similarity between the embeddings of two augmentations of the same image. Thus, SimSiam is another instance of the class of contrastive objective functions. Two random image augmentations (i.e., random resized crop, random horizontal flip, color jitter, random grayscale, and random Gaussian blur) are first generated for each image in the ImageNet dataset. Let x1 and x2 be the two augmentations of the same image, f(⋅) be the model backbone, g(⋅) be a three-layer non-linear MLP, and h(⋅) be a two-layer non-linear MLP. The three-layer MLP has hidden dimensions of 2048, 2048, and 2048. The two-layer MLP has hidden dimensions of 512 and 2048 respectively. Let θ be the parameters for f, g, and h. The loss function for one image x of a batch is defined as follows (recall that x1 and x2 are two augmentations of one image): (16) where . Note that z1 and z2 are treated as constants in this loss function (i.e., the gradients are not back-propagated through z1 and z2). This “stop-gradient” method was key to the success of this objective function. Our single-, dual-, and six-stream variants were trained using a batch size of 512 for 100 epochs using SGD with momentum of 0.9, and weight decay of 0.0001. An initial learning rate of 0.1 was used, and the learning rate was decayed to 0.0 using a cosine schedule (with no warm-up). Barlow twins [49]. This method is inspired by Horace Barlow’s theory that sensory systems reduce redundancy in their inputs [73]. Let x1 and x2 be the two augmentations (random crops and color distortions) of the same image, f(⋅) be the model backbone, and let h(⋅) be a three-layer non-linear MLP (each of output dimension 8192). Given , where z1 = h ∘ f(x1) and z2 = h ∘ f(x2), this method proposes an objective function which tries to make the cross-correlation matrix computed from the twin embeddings z1 and z2 as close to the identity matrix as possible: (17) where b indexes batch examples and i, j index the embedding output dimension. We trained AlexNet (with 64 × 64 image inputs) with the recommended hyperparameters of λ = 0.0051, weight decay of 10−6, and batch size of 2048 with the LARS [72] optimizer employing learning rate warm-up of 10 epochs under a cosine schedule. We found that training stably completed after 58 epochs for this particular model architecture. VICReg [50]. Let x1 and x2 be the two augmentations (random crops and color distortions) of the same image, f(⋅) be the model backbone, and let h(⋅) be a three-layer non-linear MLP (each of output dimension 8192). Given , where z1 = h ∘ f(x1) and z2 = h ∘ f(x2), this method proposes an objective function that contains three terms: Invariance: minimizes the mean square distance between the embedding vectors. Variance: enforces the embedding vectors of samples within a batch to be different via a hinge loss to keep the standard deviation of each embedding variable to be above a given threshold (set to 1). Covariance: prevents informational collapse through highly correlated variables by attracting the covariances between every pair of embedding variables towards zero. We trained AlexNet (with 64 × 64 image inputs) with the recommended hyperparameters of weight decay of 10−6 and batch size of 2048 with the LARS [72] optimizer employing learning rate warm-up of 10 epochs under a cosine schedule, for 1000 training epochs total. Top-1 validation set performance Performance of primate models on 224 × 224 pixels and 64 × 64 pixels ImageNet. Here we report the top-1 validation set accuracy of models trained in a supervised manner on 64 × 64 pixels and 224 × 224 pixels ImageNet. Download: PPT PowerPoint slide PNG larger image TIFF original image Performance of StreamNet variants on 64 × 64 pixels CIFAR-10 and 64 × 64 pixels ImageNet. Here we report the top-1 validation set accuracy of our model variants trained in a supervised manner on 64 × 64 pixels CIFAR-10 and ImageNet. Download: PPT PowerPoint slide PNG larger image TIFF original image Transfer performance of StreamNet variants on 64 × 64 pixels ImageNet under linear evaluation for models trained with self-supervised objectives. In this subsection, we report the top-1 ImageNet validation set performance under linear evaluation for models trained with self-supervised objectives. After training each model on a self-supervised objective, the model backbone weights are then held fixed and a linear readout head is trained on top of the fixed model backbone. In the case where the objective function is “untrained”, model parameters were randomly initialized and held fixed while the linear readout head was trained. The image augmentations used during transfer learning were random cropping and random horizontal flipping. The linear readout for every self-supervised model was trained with the cross-entropy loss function (i.e., Eq (11) with C = 1000) for 100 epochs, which was minimized using SGD with momentum of 0.9, and weight decay of 10−9. The initial learning rate was set to 0.1 and reduced by a factor of 10 at epochs 30, 60, and 90. Performance of primate models on 224 × 224 pixels and 64 × 64 pixels ImageNet. Here we report the top-1 validation set accuracy of models trained in a supervised manner on 64 × 64 pixels and 224 × 224 pixels ImageNet. Download: PPT PowerPoint slide PNG larger image TIFF original image Performance of StreamNet variants on 64 × 64 pixels CIFAR-10 and 64 × 64 pixels ImageNet. Here we report the top-1 validation set accuracy of our model variants trained in a supervised manner on 64 × 64 pixels CIFAR-10 and ImageNet. Download: PPT PowerPoint slide PNG larger image TIFF original image Transfer performance of StreamNet variants on 64 × 64 pixels ImageNet under linear evaluation for models trained with self-supervised objectives. In this subsection, we report the top-1 ImageNet validation set performance under linear evaluation for models trained with self-supervised objectives. After training each model on a self-supervised objective, the model backbone weights are then held fixed and a linear readout head is trained on top of the fixed model backbone. In the case where the objective function is “untrained”, model parameters were randomly initialized and held fixed while the linear readout head was trained. The image augmentations used during transfer learning were random cropping and random horizontal flipping. The linear readout for every self-supervised model was trained with the cross-entropy loss function (i.e., Eq (11) with C = 1000) for 100 epochs, which was minimized using SGD with momentum of 0.9, and weight decay of 10−9. The initial learning rate was set to 0.1 and reduced by a factor of 10 at epochs 30, 60, and 90. Reinforcement learning task A set of state-action-reward-state tuples (i.e., (st, at, rt, st+1)) were generated in prior work [53] and were used to train (in an offline fashion) the reinforcement learning (RL) agent. We used an offline RL algorithm known as critic-regularized regression [56]. Except for the visual encoder, the architecture of the RL agent was identical to that used by Wang et al. [56] (cf. Fig 3 in Wang et al. [56]). Four different visual encoders based on the AlexNet architecture were used: Contrastive ImageNet: AlexNet trained using ImageNet in a contrastive manner (Instance Recognition). Up to the first four layers are implanted into the virtual rodent as its visual system, as these were the layers that best matched mouse visual areas. This visual encoder is the best model of mouse visual cortex (Fig 2). Its weights were held fixed during training of the RL agent. Supervised ImageNet: AlexNet trained using ImageNet in a supervised manner. Up to the first four layers are implanted into the virtual rodent as its visual system, as these were the layers that best matched mouse visual areas. Its weights were held fixed during training of the RL agent. Contrastive Maze: AlexNet trained using the egocentric maze inputs from the virtual rodent reward-based navigation task. Up to the first four layers are implanted into the virtual rodent as its visual system, as these were the layers that best matched mouse visual areas. Its weights were then held fixed during training of the RL agent. Supervised Maze: The first four convolutional layers of AlexNet (ShallowNet) trained end-to-end on the virtual rodent reward-based navigation task. After training the agent until policy loss convergence by 10 000 steps (once), the agent was evaluated on 300 episodes. Each model was trained twice (i.e., two different random seeds), so each model was evaluated on 300 × 2 = 600 episodes in total and we report the average reward across all 600 episodes. Below, we report the average rewards (and s.e.m. across the 600 episodes) obtained when each of the visual backbones are used by the RL agent to perform the task. Evaluating model performance on downstream visual tasks To evaluate transfer performance on downstream visual tasks, we used the activations from the outputs of the shallow, intermediate, and deep modules of our StreamNet variants. We also included the average-pooling layer in all the variants (the model layer prior to the fully-connected readout layer). The dimensionality of the activations was then reduced to 1000 dimensions using principal components analysis (PCA), if the number of features exceeded 1000. PCA was not used if the number of features was less than or equal to 1000. A linear readout on these features was then used to perform five transfer visual tasks. For the first four object-centric visual tasks (object categorization, pose estimation, position estimation, and size estimation), we used a stimulus set that was used previously in the evaluation of neural network models of the primate visual system [17, 20, 74]. The stimulus set consists of objects in various poses (object rotations about the x, y, and z axes), positions (vertical and horizontal coordinates of the object), and sizes, each from eight categories. We then performed five-fold cross-validation on the training split of the medium and high variation image subsets (“Var3” and “Var6”, defined by Majaj et al. [75]) consisting of 3840 images, and computed the performance (metrics defined below) on the test split of the medium and high variation sets (“Var3” and “Var6”) consisting of 1280 images. Ten different category-balanced train-test splits were randomly selected, and the performance of the best model layer (averaged across train-test splits) was reported for each model. All images were resized to 64 × 64 pixels prior to fitting, to account for the visual acuity adjustment. The final non-object-centric task was texture recognition, using the Describable Textures Dataset [58]. Object categorization. We fit a linear support vector classifier to each model layer activations that were transformed via PCA. The regularization parameter, (18) was chosen by five-fold cross validation. The categories are Animals, Boats, Cars, Chairs, Faces, Fruits, Planes, and Tables. We reported the classification accuracy average across the ten train-test splits. Position estimation. We predicted both the vertical and the horizontal locations of the object center in the image. We used Ridge regression where the regularization parameter was selected from: (19) where C was selected from the list defined in (18). For each network, we reported the correlation averaged across both locations for the best model layer. Pose estimation. This task was similar to the position prediction task except that the prediction target were the z-axis (vertical axis) and the y-axis (horizontal axis) rotations, both of which ranged between −90 degrees and 90 degrees. The (0, 0, 0) angle was defined in a per-category basis and was chosen to make the (0, 0, 0) angle “semantically” consistent across different categories. We refer the reader to Hong et al. [55] for more details. We used Ridge regression with α chosen from the range in (19). Size estimation. The prediction target was the three-dimensional object scale, which was used to generate the image in the rendering process. This target varied between 0.625 to 1.6, which was a relative measure to a fixed canonical size of 1. When objects were at the canonical size, they occluded around 40% of the image on the longest axis. We used Ridge regression with α chosen from the range in (19). Texture classification. We trained linear readouts of the model layers on texture recognition using the Describable Textures Dataset [58], which consists of 5640 images organized according to 47 categories, with 120 images per category. We used ten category-balanced train-test splits, provided by their benchmark. Each split consisted of 3760 train-set images and 1880 test-set images. A linear support vector classifier was then fit with C chosen in the range (18). We reported the classification accuracy average across the ten train-test splits. Object categorization. We fit a linear support vector classifier to each model layer activations that were transformed via PCA. The regularization parameter, (18) was chosen by five-fold cross validation. The categories are Animals, Boats, Cars, Chairs, Faces, Fruits, Planes, and Tables. We reported the classification accuracy average across the ten train-test splits. Position estimation. We predicted both the vertical and the horizontal locations of the object center in the image. We used Ridge regression where the regularization parameter was selected from: (19) where C was selected from the list defined in (18). For each network, we reported the correlation averaged across both locations for the best model layer. Pose estimation. This task was similar to the position prediction task except that the prediction target were the z-axis (vertical axis) and the y-axis (horizontal axis) rotations, both of which ranged between −90 degrees and 90 degrees. The (0, 0, 0) angle was defined in a per-category basis and was chosen to make the (0, 0, 0) angle “semantically” consistent across different categories. We refer the reader to Hong et al. [55] for more details. We used Ridge regression with α chosen from the range in (19). Size estimation. The prediction target was the three-dimensional object scale, which was used to generate the image in the rendering process. This target varied between 0.625 to 1.6, which was a relative measure to a fixed canonical size of 1. When objects were at the canonical size, they occluded around 40% of the image on the longest axis. We used Ridge regression with α chosen from the range in (19). Texture classification. We trained linear readouts of the model layers on texture recognition using the Describable Textures Dataset [58], which consists of 5640 images organized according to 47 categories, with 120 images per category. We used ten category-balanced train-test splits, provided by their benchmark. Each split consisted of 3760 train-set images and 1880 test-set images. A linear support vector classifier was then fit with C chosen in the range (18). We reported the classification accuracy average across the ten train-test splits. Supporting information S1 Fig. StreamNet architecture schematic. The first four convolutional layers of AlexNet best corresponded to all the mouse visual areas) These convolutional layers were used as the basis for our StreamNet architecture variants. The number of parallel streams, N, was varied to be one (single-stream), two (dual-stream) or six (six-stream). https://doi.org/10.1371/journal.pcbi.1011506.s001 (TIF) S2 Fig. Shallow architectures trained with contrastive objective functions yield the best matches to the neural data (calcium imaging dataset). As in Fig 2, but for the calcium imaging dataset. A. The median and s.e.m. neural predictivity, using PLS regression, across neurons in all mouse visual areas except RL. N = 16228 units in total (RL is excluded, as mentioned in the “Neural Response Datasets” section). Actual neural predictivity performance can be found in Table 2. The “Primate Model Baseline” denotes a supervised VGG16 trained on 224 px inputs, used in prior work [14, 15, 36]. All models, except for the “Primate Model Baseline”, are trained on 64 px inputs. B. Each model’s performance on ImageNet is plotted against its median neural predictivity across all units from each visual area. Inset. Primate ventral visual stream neural predictivity from BrainScore is correlated with ImageNet categorization accuracy (adapted from Schrimpf et al. [17]). All ImageNet performance numbers can be found in Table 2. Color scheme as in A and Fig 2A. https://doi.org/10.1371/journal.pcbi.1011506.s002 (TIF) S3 Fig. Shallow architectures trained with contrastive objective functions yield the best matches to the neural data (RSA). A. The median and s.e.m. noise-corrected neural predictivity, using RSA, across N = 39 and N = 90 animals for the Neuropixels and calcium imaging dataset respectively (across all visual areas, with RL excluded for the calcium imaging dataset, as mentioned in the “Neural Response Datasets” section). The “Primate Model Baseline” denotes a supervised VGG16 trained on 224 px inputs, used in prior work [14, 15, 36]). All models, except for the “Primate Model Baseline”, are trained on 64 px inputs. B. We plot each model’s performance on ImageNet against its median neural predictivity, using RSA, across visual areas. All ImageNet performance numbers can be found in Table 2. Color scheme as in A and Fig 2A. https://doi.org/10.1371/journal.pcbi.1011506.s003 (TIF) S4 Fig. Structural and functional factors leading to improved neural response predictivity (calcium imaging dataset). A. As in Fig 3, our dual stream variant (red) and Contrastive AlexNet (brown) were trained using lower resolution ImageNet images and in a contrastive manner. Each image was downsampled from 224 × 224 pixels, the image size typically used to train primate ventral stream models, to various image sizes. Training models on resolutions lower than 224 × 224 pixels generally led to improved neural predictivity. The median and s.e.m. across neurons in each visual area is reported. As mentioned in the “Neural Response Datasets” section, visual area RL was removed from the calcium imaging neural predictivity results. Refer to Table 1 for N units per visual area. B. As in Fig 2B, AlexNet was either untrained, trained in a supervised manner (ImageNet) or trained in an self-supervised manner (instance recognition). We observe that the first four convolutional layers provide the best fits to the neural responses for all the visual areas while the latter three layers are not very predictive for any visual area. As mentioned in the “Neural Response Datasets” section, visual area RL was removed from the calcium imaging neural predictivity results. https://doi.org/10.1371/journal.pcbi.1011506.s004 (TIF) S5 Fig. Inter-animal consistency as a function of the number of units in the datasets for different mapping transforms. Inter-animal consistency was extrapolated across the number of units in each dataset using a log-linear function (f(n) = a log10(n) + b, where a and b are fit as parameters via least squares, and n is the sample size factor). This analysis reveals that inter-animal consistency of the Neuropixels dataset approaches 1.0 more rapidly than it does for the calcium imaging dataset. Inter-animal consistency evaluated at a sample size factor of one indicates the consistency when all the existing units in the datasets are used (i.e., inter-animal consistency values reported in Fig 1A). https://doi.org/10.1371/journal.pcbi.1011506.s005 (TIF) S6 Fig. Inter-animal consistency can increase with more stimuli. A. Inter-animal consistency under PLS regression evaluated on the train set (left bars for each visual area) and test set (right bars for each visual area), for both Neuropixels and calcium imaging datasets. The horizontal lines are the internal consistency (split-half reliability). B. Inter-animal consistency under PLS regression on the train set (dotted lines) and test set (straight lines), aggregated across visual areas. Each dot corresponds to the inter-animal consistency evaluated across 10 train-test splits, where each split is a sample of the natural scene image set corresponding to the percentage (x-axis). Note that RL is excluded for calcium imaging, as explained in the text (the “Neural Response Datasets” section). The median and s.e.m. across neurons is reported for both panels. Refer to Table 1 for N units per visual area. https://doi.org/10.1371/journal.pcbi.1011506.s006 (TIF) S7 Fig. Data augmentations alone do not lead to improved neural predictivity. Here we compare the neural predictivity of AlexNet trained in three different ways. Contrastive ImageNet is an AlexNet trained using instance recognition on ImageNet with augmentations that are part of the contrastive algorithm (random crop, random color jitter, random grayscale, random horizontal flip). Supervised ImageNet is an AlexNet trained on ImageNet in a supervised manner with a smaller set of augmentations (random crop and random horizontal flip). Supervised ImageNet (contrastive augmentations) is an AlexNet trained on ImageNet in a supervised manner with the augmentations used in the instance recognition algorithm. This control model allows us to ascertain whether the improved neural predictivity of the Contrastive ImageNet model (red) is due to the contrastive loss function itself or due to the larger set of image augmentations used during model training. In both neural response datasets, we can conclude that data augmentations alone do not contribute to improved correspondence with the mouse visual areas. https://doi.org/10.1371/journal.pcbi.1011506.s007 (TIF) S8 Fig. Out-of-distribution task performance across model layers. For the models trained on images from the maze environment, we plot their transfer performance on a set of out-of-distribution tasks (described in lower right panel of Fig 6C) across model layers. We find that intermediate model areas are better able to perform the transfer tasks and that the model layers that attain peak performance on the tasks correspond to those that best predict neural responses in the intermediate/higher mouse visual areas (see Fig 2B). https://doi.org/10.1371/journal.pcbi.1011506.s008 (TIF) S9 Fig. Neural predictivity on the Neuropixels dataset while fixing architecture (left) and objective function (right). The data in the left panel shows how neural predictivity varies when the architecture is fixed to be AlexNet, but the objective function is varied between supervised (object categorization) and self-supervised, contrastive objectives. The data in the right panel shows how neural predictivity varies when the objective function is fixed to be instance recognition, but the architecture is varied, including StreamNets, MouseNets, VGG16, ResNets, and AlexNet. https://doi.org/10.1371/journal.pcbi.1011506.s009 (TIF) S10 Fig. Brain hierarchy score. The brain hierarchy score metric of Nonaka et al. [51] was computed for a set of feedforward CNNs. Self-supervised, contrastive models (and shallower models) have a higher brain hierarchy score, computed using the mapping from model features to electrophysiological responses. https://doi.org/10.1371/journal.pcbi.1011506.s010 (TIF) Acknowledgments We thank Shahab Bakhtiari, Katherine L. Hermann, and Akshay Jagadeesh for helpful discussions, and Eshed Margalit and Xiaoxuan Jia for helpful feedback on an initial draft of the manuscript.

Meacock, Oliver J.;Durham, William M.

doi: 10.1371/journal.pcbi.1011524pmid: 37812642

Introduction Time-lapse microscopy and automated cell tracking has led to many fundamental advances in our understanding of how microorganisms sense and respond to their environment. While many studies have focused on the movement of planktonic bacteria at relatively low densities, many behaviours—including collective movement [1,2], combat [3], sharing of public goods [4] and genetic exchange [5]—typically only occur in the closely-packed assemblages in which most microbes live [6,7]. These dense communities are often studied in the laboratory using confluent monolayers of cells, which are much easier to image than three-dimensional aggregations. One method to generate such monolayers is to confine cells with a slab of agarose or polyacrylamide to form an interstitial colony [1,8–12], while more advanced microfluidic techniques [13] can also be used to confine cells to a single plane while allowing for more precise control over their chemical environment (Fig 1A). Monolayers can also form in thin films of fluid, including those arising naturally during bacterial swarming motility [2] and in assays used to study mixing induced by flagellar motility [14]. Regardless of the origin of the monolayer however, investigators face the same technical challenges when tracking densely packed cells using phase-contrast, brightfield and/or epi-fluorescence microscopy (Fig 1B). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Analysis of high-density datasets using FAST’s modular framework. (A) While bacteria naturally form monolayers in some environments, a number of different assays are used to physically confine cells to a plane in laboratory experiments. Among these are interstitial colonies, microfluidic devices that trap cells between two solid boundaries, and experimental assays that confine cells to a thin film of liquid. (B) These experiments are typically imaged using automated microscopy systems capable of collecting images in both transmitted light and fluorescent channels at specified time points. (C) FAST analyses these imaging datasets in six separate modules, each of which is used in sequence (see S1 Text). A complete specification of each module is provided in section 3 of S1 Text. https://doi.org/10.1371/journal.pcbi.1011524.g001 Tracking solitary cells at low density is relatively straightforward, and basic tracking algorithms that use cell position alone often produce excellent results. However, tracking cells that are densely packed together is notoriously difficult because the spacing between neighbouring cells becomes similar to the distance cells move between frames. These problems are exacerbated by cell motility. While some species of bacteria are non-motile at high density and spread slowly only via cell division and the secretion of surfactants [9,13], emergent patterns of collective motility driven by twitching [1], swarming [2] and gliding [15] further complicate tracking by rapidly changing the positions of cells. Consequently, dense, motile communities must be imaged at high framerates for tracking to be feasible, such that a typical experiment requires a timeseries of hundreds or thousands of images. The large size of imaging datasets, combined with the large number of cells within each image, means that the computational time required for cell tracking is often one of the main bottlenecks in a researcher’s workflow. This can be compounded if tracking must be repeated multiple times to optimise tracking parameters. A basic nearest-neighbour tracking algorithm compares the coordinates of cell centroids between subsequent frames and builds trajectories by connecting those centroids that are closest together between subsequent time points. More advanced algorithms leverage additional cell characteristics or ‘features’ to distinguish cells from one another, including metrics that measure cell shape, orientation, fluorescence levels and patterns of previous movement [16–19]. However, incorporating these additional feature measurements introduces a new problem, namely how to efficiently combine the information from different features to optimise tracking performance. In some software packages (e.g. MicrobeJ [17]), this requires the user to manually choose a large number of parameters and make qualitative judgements of the trajectories that result from each set of parameters. The combinatorial explosion in the number of possible parameters, the fact that a single parameter set can require hours to test, and the lack of a rigorous way to compare tracking results across different parameter sets, means that parameter optimisation in such a tracking algorithm is typically a highly iterative, time-consuming process. A related problem is the fact that properties of bacteria within collectives are highly dynamic, changing at both the individual level and population level over time. For example, fluorophores can accumulate within cells or bleach, while cell movement can speed up or slow down due to the secretion of extracellular factors or changes in gene expression [20,21]. In addition, the overall density of cells often increases over time as a result of cell division. This variability means that an algorithm optimised to track cells at the beginning of an experiment might struggle at later timepoints. Furthermore, experimental issues—such as changes in illumination, focus, or shifts in the field of view caused by thermal drift—can cause an abrupt deterioration of tracking accuracy. Knowing when tracking accuracy has deteriorated to an unacceptable level often lacks a rigorous basis and requires the output of tracking software to be carefully validated by eye, which is typically infeasible for high-throughput datasets. In this paper, we discuss a new approach that uses unsupervised machine learning to improve the fidelity of tracking. Our system automatically measures the statistical properties of each feature over time and then uses this data to dynamically change the relative weighting of each feature based on the information it can contribute to solving the tracking problem. It also provides users with a metric of the expected accuracy of the resulting cell trajectories, alerting them to sections of datasets that may need to be omitted from subsequent analyses. This tracking algorithm is combined with robust segmentation, feature extraction, lineage analysis and visualisation routines to make up FAST, the Feature-Assisted Segmenter/Tracker. FAST has been released as open-source software which can be run either directly within Matlab or as a stand-alone application, and has already been used in a number of publications to accurately analyse densely-packed bacterial monolayers [1,10,22,23]. In the following sections, we discuss the design approach and general structure of FAST, and then illustrate the utility of our novel cell tracking approach using synthetic datasets. Next, we discuss three case studies that illustrate the versatility of FAST, including: 1) lineage analysis of E. coli microcolonies, 2) tracking of twitching P. aeruginosa cells in a 2D monolayer, and 3) automated analysis of the Type 6 Secretion System (T6SS) in a co-culture of P. aeruginosa and V. cholerae. While these case studies focus on densely packed bacteria, FAST can also be used to analyse other types of biological samples (e.g. Fig Ag-i in S1 Text). Results Software overview Initially, we conducted a review of existing cell tracking software packages [17,24–28] to establish four key design objectives for our software: modularity, rapid user feedbacks, minimisation of user-defined parameters and extensibility (see section 2 of S1 Text for further details). We built the FAST pipeline following these design principles, resulting in a set of six modules that are used in sequence (Fig 1C). If required, external tools can be used to pre-process timelapse images—for example, to stabilise or bleach correct images—before importing them into FAST. Imaging data is loaded into FAST using Bioformats [29], ensuring compatibility with a diverse range of imaging formats. The first module of FAST is the Segmentation module, which uses standard image processing methods—including texture detection, ridge detection [30] and a watershed algorithm [31]—to identify the boundaries of individual cells. While this is typically based on brightfield or phase-contrast images, fluorescent images can also be used if cells are appropriately labelled with cytosolic or membrane-bound fluorophores. Next, the Feature Extraction module measures a range of different cell properties such as position, size and fluorescent intensity (potentially in multiple channels) using the previously extracted segmentation as a basis. The Tracking module employs our machine learning process to quantify the information associated with each feature and then calculates the relative weighting of each feature to maximise tracking fidelity. If required, a manual validation and correction sub-module also allows the user to correct any mistakes made by the tracking algorithm. The optional Division Detection module uses a closely related machine learning process to assign daughter cells to mother cells following cell division events. Two separate modules can finally be used to visualise the output of FAST: the Overlay module plots trajectories and/or the results of analyses over the top of the original images, while the Plotting module contains a range of different options to visualise extracted data. A comprehensive description of each of these modules is provided in section 3 of S1 Text. The FAST GUI guides the user through the process of analysing a single dataset. However, many applications require a large number of imaging datasets to be analysed using consistent settings. To automate this, we have implemented a batch-processing tool called doubleFAST. Once a user has performed an analysis on a single dataset using the FAST GUI, doubleFAST can then read the settings used during this initial run and automatically apply them to any number of additional datasets. This allows data from multiple experiments to be processed with minimal amounts of additional user input and ensures each has been analysed using consistent settings. Finally, we have implemented a post-processing toolbox, which contains scripts and functions to perform a number of different tasks on FAST’s output. These allow, among other things, conversion of track data to other file formats, automated detection of different genotypes, and the annotation of events such as reversals in movement direction. Users of FAST can suggest new additions to this toolbox so they can be used by the wider community. FAST’s tracking algorithm One of FAST’s principal innovations is its tracking algorithm, which automatically determines how to best combine data from a variety of different cell features to improve tracking fidelity. Although previous tracking software packages have had the option to incorporate cell characteristics other than position in their tracking routines [16,19], FAST uses a conceptual framework based on information theory that optimises this process, increasing both the power and convenience of this approach. In this section we provide a high-level overview of our main innovations and how they impact the tracking process. For more detailed derivations and explanations, please refer to section 3.4 of S1 Text. Our approach to the tracking problem is based on measurement of the amount of information that can be used to assign links between objects in subsequent frames, a measurement we call the ‘trackability’. This trackability score provides an integrated measure of how accurately we can follow objects from frame to frame, and due to its grounding in information theory has certain desirable properties such as the additivity of contributions from statistically independent features [32,33]. Measuring the trackability over time therefore provides users with a tool to predict when tracking will be stable and robust, as well as flagging portions of a dataset that might yield spurious trajectories. We define trackability in the following paragraphs and then illustrate how this metric is used in practice in our first and second case studies. We begin by assuming that N different features are measured for each object. An object’s features can be expressed as a vector with N elements, where the object index is denoted as i and time is denoted as t. Changes in an object’s features over time—corresponding to, for example, translational movement or changes in fluorescent intensity—therefore generate a trajectory through the N-dimensional feature space. The goal of our tracking algorithm is to reconstruct each object’s trajectory from the cloud of individual data points resulting from the segmentation and measurement of objects in each image. As previously noted, individuals in high-density and high-motility systems can easily move further than the typical cell-cell separation between frames, making it difficult to reconstruct their trajectories from positional information alone. By including additional features in the tracking framework, we expand the feature space from these two spatial dimensions to N feature dimensions, creating new axes along which one can potentially discriminate neighbouring individuals from each other. The trackability metric provides an estimate of how distinguishable trajectories are from each other in the feature space, and therefore how accurate tracking is likely to be. Unpredictable movement of objects tends to reduce trackability, while trackability increases if the features sample a wider range of values (i.e. if they have a larger dynamic range). To formalise this, we model an object’s instantaneous position in feature space as the random vector Xt and the change in this position between subsequent images as the random vector ΔXt. The distribution f(x) is then the probability density function (PDF) representing the chance of finding a randomly selected object at a particular position x in the absence of additional information (specifically, the position of the object at prior timepoints), while the distribution f(Δx) represents the stochastic change in an object’s position in feature space between sequential timepoints. We estimate f(Δx) by assuming that the motion of an object through feature space can be modelled as a Gaussian random walk. Explicitly, we assume that the feature vector of an object at frame t+1 can be written in terms of its prior feature vector as: (1) where ΔXt is modelled as a multivariate normal , and μt(Δx) and Σt(Δx) are respectively the mean vector and covariance matrix of the set of frame-frame feature displacements {Δxt}. We can similarly characterize f(x) using the covariance matrix of the raw object locations, Σt(x). While we can estimate Σt(x) directly from static snapshots as the covariance of the set of feature vectors {xt}, resolving {Δxt} and subsequently μt(Δx) and Σt(Δx) requires a putative set of cell trajectories, which in our algorithm are obtained via a preliminary round of tracking that uses a simple nearest-neighbour approach. To help ensure these summary statistics are measured accurately, FAST uses only the lowest displacement trajectory links from this training dataset, which are the ones most likely to be correct. Users specify the fraction F of the links with the smallest displacements to be used in the calculation of μt(Δx) and Σt(Δx). This allows the user to balance the trade-off between larger values of F, which increases the size of the training dataset, and smaller values of F which increases the quality of the training dataset. From these measurements, we estimate the amount of information available for assigning objects between frames by calculating the difference between the entropies for the two distributions, H(X) and H(ΔX) [33,34]. The trackability rt, which is measured in bits/object, can then be written as: (2) where no,t is the total number of objects present at time t and |∙| denotes the determinant of the contents. The first term captures the balance between the dynamic range of all features (|Σt(Δx)|) and the unpredictability of an object’s movement through feature space (|Σt(Δx)|), while the second and third terms rescale this quantity to express the amount of information available to the tracking algorithm per object. For a further details, including our assumptions about the form of the underlying statistical distributions, please see section 3.4.2 of S1 Text. To illustrate the trackability metric, we consider the case of a single object with a single feature, for example its position in space along a single axis, x (Fig 2A–2C). This simplifies Eq (2) to: (3) where σt(x) and σt(Δx) denote the standard deviation of their respective distributions. As expected, one observes a larger trackability score when the distribution of feature displacements, f(Δx), is more sharply peaked relative to the distribution of f(x) (Fig 2B and 2C). i.e. when the typical distance an object moves between frames is small compared to the range of values of x. Intuitively, the precision of tracking will increase as (i) the size of the random fluctuations in feature space decreases, (ii) the number of objects within a frame decreases, and (iii) the total size of the feature space the objects occupy (i.e. their dynamic ranges) increases. By taking these multiple factors into account, rt represents an integrated measurement of the risk that a given object will be incorrectly linked to a different object in a subsequent frame. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. An information-theoretic framework for automated object tracking. (A) For illustrative purposes, we consider here a theoretical dataset in which an object is characterised using a single feature, its position along the x-axis. The object’s position at three successive timepoints is denoted x1, x2, x3 (red circles), while the displacements are denoted Δx1, Δx2 (blue arrows). (B) We assume that feature displacements are drawn from a Normal distribution f(Δx), while the instantaneous object position (independent of knowledge of other timepoints) is drawn from a separate Uniform distribution f(x). The information content I of the feature is then calculated as the difference in the entropies of the two distributions, H(X) and H(ΔX), and represents our increase in certainty about the position of the object at time t+1 given knowledge about its position at time t. The trackability quantifies the total amount of information measured for each object, which increases when f(Δx) exhibits less variability relative to f(x). (C) Trackability decreases when the distribution of f(Δx) is broader (i.e. the feature becomes more ‘noisy’). Here the different colours correspond to the different distributions of f(Δx) shown in panel B. (D,E) Illustration of the feature normalization process for two features. In both D and E, the central plot indicates the joint distribution of a pair of feature displacements, while the left and bottom plots indicate the corresponding marginal distributions. In D, the random variables representing the unnormalized frame-frame displacements of the two features—ΔX and ΔY—are correlated and displaced from the origin. Using the joint distribution’s covariance matrix Σ(Δx) and mean vector μ(Δx), the feature space is transformed such that the resulting joint distribution of feature displacements is zero-centred and isotropic (E), ensuring that each feature exhibits an equivalent amount of stochastic variation between frames. https://doi.org/10.1371/journal.pcbi.1011524.g002 In addition to calculating the trackability, μt(Δx) and Σt(Δx) are also used to perform what we call ‘feature normalization’. This transformation converts the raw feature space x to a normalized feature space with a corresponding displacement distribution that is isotropic and zero-centred (Fig 2D and 2E), thus ensuring that the stochastic variation observed within each component of is equal and that any predictable motion in the feature space (e.g. a gradual increase in cell length due to growth, or a reduction in fluorescent intensity due to photobleaching) is accounted for. The metric of this transformed space is the Mahalanobis distance, a dimensionless measure of how reliably we can predict where an object will appear in the feature space at the next time point under the assumptions of our statistical framework. By providing an equitable way to combine data from different features together, this metric allows us to more accurately distinguish correct from incorrect putative links. Large distances between sequential timepoints in this space indicate a discrepancy between the predicted and observed location of an object in the next frame, and so suggest that the input data must be erroneous—for example, because an object was mis-segmented in a single frame. In contrast to previous approaches where feature weightings have to be selected or measured manually [16], feature normalization allows one to automatically optimise the contribution of multiple features without any additional user input. Finally, we use these statistical measurements to also calculate the adaptive tracking threshold βt, which automatically adjusts the stringency of the algorithm that links objects together based on how much information is available in each frame. While a larger fraction of putative links are accepted when information is relatively plentiful, only the highest certainty links are accepted when information is more limited. This dynamic adjustment of the link threshold thus allows FAST to maximise the number of trajectories in less challenging tracking conditions (e.g. slow-moving cells at low density) while minimizing the number of spurious links in more challenging conditions (e.g. fast-moving cells at high density). As different imaging datasets may require a different balance between maximising the number of correct links and minimising the number of spurious links, users select a static threshold P to balance this trade-off, which FAST then automatically converts to the time-varying threshold βt based on the instantaneous amount of information available. In summary, users must specify the tracking threshold P, the fraction of training links to retain F and the features that should be included in the final round of tracking. To help users optimise these parameters quickly, the Tracking module contains built-in tools that allow users to objectively assess tracking quality on a single pair of frames before committing to processing the entire dataset. Validating the methodology of FAST using ground-truthed datasets To demonstrate the functionality of our tracking algorithm and to illustrate how using multiple features can enhance accuracy, we used a previously described self-propelled rod (SPR) model [1,35] to generate synthetic datasets that simulate bacteria collectively moving at high-density (Fig 3A). In these simulations, cells are modelled as stiff, mutually repulsive rods that are propelled by a constant force (see Methods). This model has been shown to closely approximate bacterial collectives that propel themselves using either flagella or type IV pili, where steric interactions between neighbouring cells generate complex emergent collective behaviours [1,35]. Importantly, this approach allows us to independently control each of the different properties of the system, allowing us to both test FAST on a very large number of qualitatively different datasets and to objectively assess its performance using an automatically generated ground truth. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Validating FAST using synthetic data from a simulation of collective bacterial motility. (A) We used a self-propelled rod (SPR) model to generate noisy datasets with a verified ground truth. We then performed cell tracking with FAST using either only a single feature (the rod’s centroid at sequential timepoints; brown) or using a suite of different features (rod centroid, length, orientation and simulated fluorescent intensity; purple). Comparison of the reconstructed trajectories from FAST (B) with the ground truth data (C) allowed us to identify the errors made by the tracking algorithm (D). Errors were split into two categories: links made between objects that were incorrect (‘false positives’, solid magenta lines), and links between objects that were missed (‘false negatives’, dashed magenta lines). We also calculated the number of correct links made in each case (‘true positives’, solid orange lines). From these counts, we evaluated the performance of the tracking algorithm by calculating the F1-score (main text). (E) The value of this F1-score depends on a user-defined parameter, the tracking threshold. To objectively compare the results from the tracking algorithm when run on datasets with different properties, we calculated the tracking threshold that generated the largest F1-score for a given dataset and used this score in subsequent analyses. (F) Including all feature information substantially and consistently improved tracking performance compared to when only object positions were used (see also S2 Fig). The arrow to the right of the plot (labelled ‘Performance increase’) indicates the median F1-score for the two cases, corresponding to a ~10-fold reduction of tracking errors when all features were used. Furthermore, we found that our trackability metric was an excellent predictor of tracking accuracy for a given set of features, suggesting that it can be used to estimate the accuracy of the algorithm even when a ground truth is not available. In the different simulations we varied rod density, propulsive force, and framerate, as well the amount of noise in the measurement of position, length and fluorescence (Table 1). The black circles in F correspond to the dataset presented in E. https://doi.org/10.1371/journal.pcbi.1011524.g003 We simulated a 2D monolayer of cells constitutively expressing a fluorescent protein by initialising populations of cells whose length and fluorescent intensity were drawn from distributions obtained from experimental data (S1 Fig). Following initialization, we simulated cell movement by numerically integrating the equations of motion for each rod. Once the system had reached steady-state, we extracted measurements of rod position, orientation, fluorescence and length at evenly spaced timepoints. To simulate noisy measurements, we added Gaussian noise to each of these features, with the noise magnitude based on that observed in real experimental data (Fig 3B, Methods). Rather than specialising on datasets with specific properties, FAST’s tracking algorithm is designed to be robust to a wide diversity of different conditions by automatically compensating via feature normalization. To test this capability, we varied the parameters of our simulation by adjusting the rod density, self-propulsion force and framerate, as well as the accuracy of feature measurement by adjusting the amount of noise in the measured cell position, length and fluorescent intensity (Table 1). We tested five different values for each of these six parameters, yielding a total of 30 datasets. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Parameters of the SPR model used to generate synthetic datasets. Here we show both range of values we tested and baseline value that was used when we varied another parameter. Note that our simulations are non-dimensionalised, using the width of a single rod as the characteristic lengthscale and the time taken for an isolated rod with self-propulsion force ν = 1 to move a single rod width as the characteristic timescale. https://doi.org/10.1371/journal.pcbi.1011524.t001 The performance of the tracking algorithm was assessed by comparing its output to the ground truth (Fig 3C and 3D). Links that were present in the ground truth but missing in the reconstructions were scored as false negatives (FN), while those that were absent in the ground truth but present in the reconstructions were scored as false positives (FP). Links that were identical in both were scored as true positives (TP). We now integrated these measurements into a single metric that quantifies tracking performance, the F1-score, defined as: (4) Users of FAST specify a tracking threshold that controls the stringency of the linking process. If this threshold is too stringent, too many correct links will rejected by the algorithm, while if the threshold is not stringent enough too many incorrect links will be accepted. In practice, users need the ability to choose the tracking threshold that best suits their needs—for example, users interested in detecting very rare events will have different requirements than those interested in measuring average cell behaviour. However, for the purpose of testing the benefits of our automated tracking algorithm, we removed the tracking threshold as a factor from our analyses by using the threshold that resulted in the largest the F1-score for each dataset (Fig 3E). This maximum F1-score is an objective measure of the performance of our tracking algorithm across datasets with different characteristics, and also allows us to robustly benchmark our tracking algorithm. The simulated datasets were tracked using two different methods (Fig 3B). The first method only used the position of the rods—effectively a classical position-based tracking approach (‘Centroids’), while the second method used all four features—position, orientation, length and fluorescent intensity (‘All features’). We found that including all features led to a dramatic increase in tracking accuracy in all datasets, with a 4 to 10-fold reduction in the number of tracking errors (S2 Fig). Furthermore, our groundtruthed datasets allow us to directly relate a dataset’s trackability to the accuracy of the resulting tracks, which demonstrated that it is an excellent predictor of the F1-score regardless of the simulation parameters or level of noise in the dataset (Fig 3F). Taken together, these analyses validate our methodology and illustrate how FAST copes with challenging datasets. First, they demonstrate that our method of integrating additional feature information during tracking is reliable and effective, allowing FAST to substantially reduce tracking errors compared to alternative approaches that use cell position alone. Second, these analyses demonstrate that our trackability score can integrate multiple aspects of the dataset into a single, robust heuristic of predicted tracking accuracy. Case studies Tracking cells, divisions, and lineages in growing non-motile E. coli colonies. A major aim of a number many previous tracking packages is to automatically reconstruct cell lineages [25,28]. However, identifying cell division events and the resulting daughter cells is a particularly difficult challenge. Near-perfect tracking of individuals is essential for accurate lineage tracking because single errors can propagate and cause lineages to be misassigned at later timepoints. We tested FAST’s lineage tracking capabilities using eight separate time lapse movies of E. coli cells as they divided to form microcolonies [11]. These experiments were imaged using a combination of phase-contrast and fluorescence microscopy, and used a strain with a GFP transcriptional reporter to quantify the level of colicin Ib (cib) expression. The expression of this gene is highly variable between cells but remains stable over a single generation. This combination of population-level heterogeneity and individual-level stability means that the level of GFP expression in each cell could contribute a large amount of information to FAST’s tracking algorithm and thus substantially improve its performance. We began by applying automated tracking and division detection to a single dataset. We visualised the lineage structure of the microcolony using the Overlays module to rapidly verify the accuracy of the lineage assignment (Fig 4A and 4B, S1 Movie). This revealed that the descendants of each of the individual cells present at the beginning of the experiment formed highly elongated structures within the colony, similar to the patterns previously observed in experiments where colonies were initiated from multiple founder cells labelled with different fluorescent proteins [8]. Next, we used our batch processing tool—doubleFAST—to automate the analysis of the remainder of the eight datasets, using the parameters obtained when analysing the first microcolony. A complete breakdown of processing times by module and dataset is shown in Table 2—in total, processing of all eight datasets took 5.5 minutes, or around 40 seconds per microcolony on a standard laptop computer (Methods). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Processing times and dimensions of lineage datasets. ‘Dimensions’ indicates the size of the raw imaging dataset as x-size, y-size and duration. x- and y-sizes are recorded in pixels, while duration is recorded as the number of frames. ‘Objects’ indicates the total number of segmented objects in the manually corrected segmentations, while ‘Cells’ indicates the total number of separate cell trajectories in the manually corrected track dataset. The ‘Processing time’ section reports the amount of time taken to process each dataset by each module of FAST (in seconds) on a standard laptop computer (see Methods for details). https://doi.org/10.1371/journal.pcbi.1011524.t002 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Benchmarking FAST’s performance using an experimental dataset containing eight E. coli microcolonies. (A) Tracks showing the movement of individual cells within a typical microcolony as it expands, where colours denote the instantaneous cell length. (B) Automated lineage tracking of microcolony at three different timepoints. Cells that share the same mother in generation 1 (‘Gen 1’) are labelled with the same colour, illustrating how the spatial distribution of different lineages develops over time. Occasional black cells in the final timepoint (‘Gen 6–7’) indicate cells that were not assigned to the correct lineage. (C) Images from eight different microcolonies were processed with FAST to automatically obtain trajectories and division events. These were then compared to a manually curated ground truth to calculate F1-scores, quantifying the performance of the Segmentation module (triangles), Tracking module (filled circles) and Division Detection module (empty circles). These analyses proceeded in two parallel streams: In the ‘corrected’ stream, the inputs of the Tracking and Division Detection modules had been manually corrected, whereas in the ‘automated’ stream these inputs were obtained directly from the previous module and remained uncorrected. (D) When averaged over all timepoints that contained more than 32 cells, we found that the trackability metric was a reliable predictor of the accuracy of both the Tracking and Division Detection modules under fully automated conditions. In addition, this analysis showed that including cell length, width, fluorescent intensity and position as features in the Tracking module (‘All features’, purple) reduced the median number of errors in both the Tracking and Division Detection modules by approximately 5-fold compared to results obtained using cell position alone (‘Centroids only’, brown). (E-H) We next investigated which factors influence trackability by comparing metrics related to fluctuations in feature values–average cell speed (E), relative elongation rate (the fractional change in cell length per min) (F), coefficient of variation (COV) of GFP intensity (G) and COV of cell width (H). The statistical significance of correlations is denoted using the following symbols: * denotes p < 0.05, ** denotes p < 0.005, n.s. = not significant. These analyses were performed using the log-transformed F1-scores in D and are based on a linear regression t-test. The orange points in C-H indicate the exemplar microcolony shown in A and B. Visualisations shown in A and B were produced using FAST’s Overlays module. https://doi.org/10.1371/journal.pcbi.1011524.g004 Although these automated analyses produced highly accurate results, a few tracking and division assignment errors were observed (Fig 4B). To objectively assess the accuracy of FAST’s algorithms, we benchmarked our results by constructing manually-validated ground-truth versions of the Segmentation, Tracking and Division-Detection module outputs. In the case of the Tracking module, this task was substantially facilitated by our Track Correction GUI. Our analyses then proceeded via two separate streams: in the first, ‘corrected’ stream, we used the manually corrected output of the Segmentation module as an input to the Tracking module, and the manually corrected output of the Tracking module as an input to the Division Detection module, allowing us to assess the quality of each module independently. In the second, ‘automated’ stream, we used the uncorrected outputs of each module as the input to the next. This allows us to assess the performance of the tracking and division detection algorithms in conditions typical of large-scale analyses, where errors from earlier analyses are propagated uncorrected into later steps (Fig 4C). We found that the results of our Segmentation module were highly accurate, with a median mis-segmentation rate of 0.52%. The Tracking module was similarly accurate, with a median error rate of 1.02% when using the uncorrected segmentations as inputs. This is very similar to the error rate reported for DeLTA 2.0 [36], which was also benchmarked against microcolony datasets from van Vliet et al. [11]. We note, however, that the average error rate reported for DeLTA 2.0 is an aggregate measure that also includes datasets from additional E. coli genotypes in van Vliet et al. [11] not considered here. Manually correcting FAST’s rare mis-segmentations had a relatively small impact on the Tracking module’s performance, decreasing the median error rate to 0.71%, suggesting that the tracking algorithm’s automated system for bridging mis-segmentations performs as intended. In contrast, the Division Detection module’s performance was considerably enhanced when the input trajectories were manually corrected, with the median error rate decreasing from 4.77% to 0.63%. We therefore recommend using our track correction GUI if highly accurate lineage assignment is required. We also analysed the utility of trackability as a predictor of tracking algorithm performance, equivalent to our automated benchmarking with the SPR model (Fig 4D). As before, we found that trackability averaged over all timepoints with a relatively large number of cells (>32) was robust predictor of the tracking performance on the different microcolonies. This was even true for individual timepoints, with trackability being a much better predictor of tracking fidelity than more rudimentary metrics such as the current number of cells within the microcolony (S3 Fig). We also found that trackability was a strong predictor of how accurately divisions were detected (Fig 4D). Lastly, we measured the performance of the tracking and division detection algorithms when a large suite of features was used (cell length, width, fluorescence, and position; denoted ‘All features’) compared to when only cell positions were used (denoted ‘Centroids only’). Similar to what was previously observed in the synthetic datasets, adding extra features substantially increased the performance of our algorithms, reducing the number of tracking and division assignment errors by approximately 5-fold. Given that the eight microcolonies that we analysed used the same strain of bacteria and were grown in the same experimental conditions, we were surprised by the variations in tracking performance that we observed across the eight microcolonies. One potential cause of this variation was that the user-defined tracking parameters which were optimised for one dataset were not optimal for the other microcolonies. However, when we systematically repeated the benchmarks with different user-defined tracking parameters, we found that the set of tracking parameters that produced the most accurate results was highly conserved across the eight microcolonies (S4 Fig), suggesting this was not the primary cause. Next, we asked whether biological differences between the samples could be responsible for the observed variation in performance. To test this, we measured each microcolony’s average cell speed, relative elongation rate (the average fractional increase in cell length per minute), coefficient of variation (COV) of GFP fluorescence and the COV of cell width using our manually corrected tracks, and investigated how these metrics related to the microcolony’s trackability (Fig 4E–4H). We found that the average rate at which cells move and grow within a microcolony were significantly correlated with the microcolony’s trackability. In contrast, the amount of variability observed in the cell fluorescence and cell width did not have an appreciable impact on a microcolony’s trackability. Taken together, these results indicate that the variation in the trackability metric we observed were driven by real biological variation across the different microcolonies. In this case differences in the average growth rate of cells in each microcolony likely drive changes in the speed at which these non-motile colonies expand, and consequently impacts the fidelity of tracking and division detection. In summary, these analyses show that segmentation errors do not substantially affect the tracking performance of FAST, but errors in tracking can detrimentally impact the detection of cell division events. Moreover, these analyses of experimental data validate our previous findings from the synthetic datasets by demonstrating firstly that tracking performance can be substantially improved by including cell features other than cell position and secondly that trackability is a reliable predictor of tracking fidelity. Finally, this case study demonstrates that the two user-defined tracking parameters F and P do not need to be excessively fine-tuned to obtain reliable tracking results (S4 Fig, note the logarithmically-spaced axes). Quantifying rapid bursts of cell movement in densely packed P. aeruginosa monolayers. Many different species of bacteria generate collective motility in densely-packed communities using either flagella, Type IV pili, or via gliding [2,15,35,37]. Motility allows populations to rapidly expand into new territory, giving them a competitive advantage over non-motile genotypes [1,38]. Here we demonstrate how FAST can be used to quantify the behaviour of P. aeruginosa cells in interstitial colonies that form between agar and glass [37]. We focus on cells within the monolayer that forms directly behind the colony’s leading edge, the dynamics of which play a crucial role in the competition between genotypes in both interstitial and more classical ‘surficial’ colonies [1]. Tracking cells undergoing collective movement requires a much larger acquisition frame rate compared to non-motile cells. While expensive timing boards can be used to ‘trigger’ cameras with a high level of precision to keep the time between frames nearly constant, many high-end research microscopes lack this capability. Instead, ‘camera streaming’ is more widely available, which directly streams the camera’s output to a computer which saves frames as fast as possible. This maximises the frame rate, but images acquired via camera streaming can have slight variations in the time that elapses between subsequent frames. To illustrate how FAST can be used to handle a sequence of images collected via camera streaming, we collected a large dataset with 3,505 frames recorded at an average rate of 127 frames per second. The size is of this dataset is approximately 8 Gb and each frame contains approximately 1,700 tightly packed P. aeruginosa cells. Despite the large size of this dataset, processing with FAST could be completed on a standard laptop computer (Methods), requiring only 200 mins to segment and 355 mins to extract the features of each of the ~6 million individual objects. Tracking consists of two separate stages: the model training stage, and the link assignment stage. After completion of the model training stage, FAST automatically generates and plots the trackability of the dataset at each timepoint (Fig 5A). For our dataset, this plot revealed that the trackability dropped precipitously at some timepoints. We hypothesised that these decreases resulted from a reduction in the imaging framerate. To test this, we plotted the trackability score against ∆t, the elapsed time between frames as calculated from timestamps (Fig 5A, inset). This revealed a strong negative correlation (Pearson’s correlation coefficient = -0.705), suggesting that the longer the time between frames, the lower the trackability and the less accurate the tracking results. To avoid these timepoints—and the spurious tracking results they might generate—we used the tracking module’s built-in time window selection tool to specify a subset of frames to track (Fig 5A, green region, 750 frames). Training the tracking model for this reduced dataset took 18 minutes, while tracking and track processing took an additional 88 minutes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Tracking single cells in an interstitial P. aeruginosa colony spreading via pili-based motility. (A) We used FAST to track cells within a monolayer of P. aeruginosa undergoing collective motility using position, length and width as features. Images were collected using high speed ‘camera streaming’ with a mean frame rate of 127 fps, but variations in the elapsed time between subsequent frames, ∆t, resulted in transient reductions in trackability. (A, inset) Analysis of the timestamps associated with each frame revealed that the trackability score was negatively correlated with ∆t (r indicates Pearson’s correlation coefficient; the warmer colours denote a higher density of data points). We therefore restricted our subsequent analyses to a subset of the data in which the trackability score was relatively constant (green region) using FAST’s built-in tools. (B) We used the Overlays module to colour code cells based on their instantaneous speed. Although cells typically moved relatively slowly, very occasionally cells were observed to undergo a very rapid burst of movement (see cream coloured cell). (C) These rapid jumps can also be observed in traces of the speed of individual cells. Here we plot the instantaneous speed of three different cells over time, each in a different colour. The montages above illustrate three of these transient events, using the same colour coding shown in B. (D) To investigate these movements at the population level, we calculated instantaneous cell velocities in both the x and y direction for all cells and plotted their combined distribution. In other active systems, this distribution is approximately Gaussian. However, in our system the highly transient bursts of velocity result in heavy tails, causing them deviate from a Gaussian distribution with the same variance (dashed black line). https://doi.org/10.1371/journal.pcbi.1011524.g005 In this example, a relatively large timestep ∆t allows cells to move further between frames, which reduces predictability and therefore reduces the trackability score. More generally, however, the trackability depends on a combination of both experimental and imaging conditions, providing a robust metric to interpret datasets. For example, the trackability score can be used to quickly identify a wide range of problems which might arise during an experiment, including fluctuations in focus, illumination intensity, or inadvertent movement of the sample. Once problematic timepoints have been identified, the user can decide either to avoid them by using a subset of the images (as in this example) or to reject the entire dataset. Tracking the smaller subset of images that we specified resulted in over 1,600 separate trajectories, each at least 200 timepoints long. To visualise this large dataset, we used the Overlays module to colour individual cells in the phase-contrast image based on their instantaneous speed (Fig 5B, S2 Movie). This revealed that while most cells move at relatively slow speeds, a small number of cells undergo rapid, sporadic bursts of movement approximately 25 times faster than the average. These rapid movements are also clearly visible in measurements of the speed of individual cells over time (Fig 5C). Our results are similar to the ‘slingshots’ previously observed in solitary P. aeruginosa cells moving at the glass/liquid interface, apparently driven by pilus detachment events [39]. However, the peak speeds that we record (up to 60 μm s-1) in collectives of P. aeruginosa are approximately 20 times larger than those previously observed. While we do not know the specific reason for this difference, interactions with neighbouring cells facilitated by the high cell density, the glass/agar interstitial colony environment and our higher framerate (~13 times that of [39]) may each play a role. To illustrate how this large tracking dataset can be mined to elucidate the statistics of rare events, we constructed the instantaneous marginal velocity distribution from our tracks (i.e. the x- and y-components of the instantaneous velocity vectors) (Fig 5D). While previous studies have found that sperm and swimming bacterial cells undergoing collective movement generate marginal velocity distributions that are approximately Gaussian [35,40], we observed that bacteria collectively moving via pili-based motility exhibit much heavier tails corresponding to their occasional rapid motions. Despite the rarity of these events—less than 0.05% of our measurements had a magnitude larger than 20 μm s-1—the exceptional number of cell trajectories we obtained nevertheless allowed us to finely resolve their statistical distribution. These analyses demonstrate how FAST can be used to rapidly characterise the motility of a large number of cells in densely-packed conditions, with relatively little user input and computational effort. Automated analysis of T6SS battles. To illustrate FAST’s capabilities beyond standard cell tracking, we analysed the activity of the Type 6 Secretion System (T6SS) using FAST. The T6SS is a highly dynamic organelle composed of a molecular ‘spear’ tipped with a toxin known as an effector [41]. Bacteria that possess the T6SS can inject this effector into neighbouring cells, which kills them [42]. Firing events can be monitored by visualising the localisation of the protein that forms the contractile sheath of the T6SS needle, TssB in P. aeruginosa and VipA in Vibrio cholerae [12,43]. Here, we highlight how FAST can be used to quantify how the T6SS is regulated in co-cultures of V. cholerae and P. aeruginosa. Because of its short range, the T6SS is effective only at high cell density, which historically has made it difficult to study using automated analyses. We used FAST to analyse imaging datasets that show V. cholerae and P. aeruginosa cells expressing VipA-mCherry and TssB-mNeongreen, respectively, interacting through their respective T6SSs when mixed together in a densely-packed monolayer [12] (Fig 6A, left). We distinguished the two species from one another (Fig 6A, right) using FAST’s post-processing toolbox, which compares each cell’s intensity in the mCherry and mNeonGreen channels to automatically assign them to two distinct populations (Fig 6B). This utility allows one to rapidly compare the behaviour of different genotypes within the same sample. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Identifying different bacterial species, quantifying cell morphologies, and investigating T6SS dynamics in a densely packed collective. (A) A monolayer composed of a mixture of P. aeruginosa cells expressing TssB-mNeongreen and V. cholerae cells expressing VipA-mCherry (left). Measurements of the average intensity of each cell in the two different fluorescence channels allowed us to automatically distinguish the two species (B), an approach known as image cytometry. By colour coding each cell (P. aeruginosa, cyan, V. cholerae, red), we were able to visually confirm the accuracy of the assignment process (A, right). (C) We then quantified a novel feature, cell curvature, using FAST’s custom feature extraction framework to calculate the curvature of the segmentation backbones (inset). Splitting these measurements into the two previously assigned populations showed the comma shaped V. cholerae cells were substantially more curved than the rod-shaped P. aeruginosa cells. (D-F) We then used FAST’s plotting options to illustrate the dynamics of the T6SS. A P. aeruginosa cell fires its T6SS machinery (observed as the bright puncta of TssB), which can be visualised using the kymograph (D) and ‘cartouche’ (F) plotting options, and which also leads to an increase in the standard deviation of the mNeongreen channel (E). (G) We used this latter measurement to detect when the T6SS was active in a cell, allowing us to measure the proportion of time that each cell in the population spends in the firing state. Our analyses confirmed that when co-cultured with a strain of V. cholerae with an inactive T6SS (Δhcp1 Δhcp2), the T6SS activity of WT P. aeruginosa cells is dramatically reduced compared to when they are co-cultured with WT V. cholerae. Here the circles show averages from separate movies and horizontal lines show the mean for all samples of a given combination of genotypes. https://doi.org/10.1371/journal.pcbi.1011524.g006 The architecture of FAST allows users to easily define and quantify novel features that are not already included in FAST. To do this, users can write a short feature measurement script that can make use of the segmentation of each object in each frame, as well as the corresponding regions in each of the imaging channels—in this case, the mNeongreen, mCherry and phase-contrast signals. The Feature Extraction module then automatically stores these custom features in the same way as the built-in features, allowing them to be integrated into downstream analyses. To illustrate this capability, we extracted cell curvature as a custom feature to see if we could detect the differences in morphology of V. cholerae cells (which are comma shaped) from that of P. aeruginosa (which are rod shaped). Using a skeletonization-based approach, we extracted the morphological backbone of each cell from its segmentation and measured its curvature by finding the best-fit circle to this set of points. Consistent with expectations, our automated analysis found that the cells identified as V. cholerae were substantially more curved than those identified as P. aeruginosa (Fig 6C). Our software is also capable of generating sophisticated visualisations and analyses of the behaviour of single cells. For example, FAST allows the dynamics of T6SS firing in individual cells to be easily visualised using the kymograph option of the Plotting module (Fig 6D), as well as the ‘cartouche’ option, which extracts and aligns cropped images of the specified cells (Fig 6F). We can also quantify the dynamics of the T6SS using the feature data associated with a given cell: the standard deviation of the mNeongreen channel substantially increases during firing (Fig 6E), as the TssB becomes non-uniformly distributed through the cell as it assembles into the sheath. It is generally accepted that P. aeruginosa only fires its T6SS when triggered by the firing of the T6SS of a neighbouring cell [9,12]. Although qualitative evidence for this is strong, to our knowledge this effect has never been directly quantified, likely because of the difficulties involved with tracking densely packed cells and because the T6SS firing events are themselves relatively rare. We therefore decided to use FAST to simultaneously track a large number of densely packed cells and automatically detect when each fires its T6SS. We used doubleFAST to automate the analysis of multiple datasets from experiments in which WT P. aeruginosa cells were either co-cultured with WT V. cholerae or co-cultured with a mutant V. cholerae strain that is incapable of firing their T6SS (Δhcp1 Δhcp2). Using the standard deviation of the mNeongreen channel to distinguish when P. aeruginosa is actively firing its T6SS, we calculated the proportion of time that P. aeruginosa cells spend firing their T6SS (Fig 6G, S3 Movie). As expected, when co-cultured with the inactivated Δhcp1 Δhcp2 V. cholerae strain, P. aeruginosa dramatically reduced the proportion of time it spent firing its T6SS compared to when co-cultured with WT V. cholerae. These analyses generated 1,246 separate trajectories, of which around half (601) spanned the full 151 timepoints of the experiments. In conclusion, this case study demonstrates how FAST can automatically distinguish different species, how users can specify novel features like cell curvature, and how complex bacterial behaviours can be quantified using feature data. Software overview Initially, we conducted a review of existing cell tracking software packages [17,24–28] to establish four key design objectives for our software: modularity, rapid user feedbacks, minimisation of user-defined parameters and extensibility (see section 2 of S1 Text for further details). We built the FAST pipeline following these design principles, resulting in a set of six modules that are used in sequence (Fig 1C). If required, external tools can be used to pre-process timelapse images—for example, to stabilise or bleach correct images—before importing them into FAST. Imaging data is loaded into FAST using Bioformats [29], ensuring compatibility with a diverse range of imaging formats. The first module of FAST is the Segmentation module, which uses standard image processing methods—including texture detection, ridge detection [30] and a watershed algorithm [31]—to identify the boundaries of individual cells. While this is typically based on brightfield or phase-contrast images, fluorescent images can also be used if cells are appropriately labelled with cytosolic or membrane-bound fluorophores. Next, the Feature Extraction module measures a range of different cell properties such as position, size and fluorescent intensity (potentially in multiple channels) using the previously extracted segmentation as a basis. The Tracking module employs our machine learning process to quantify the information associated with each feature and then calculates the relative weighting of each feature to maximise tracking fidelity. If required, a manual validation and correction sub-module also allows the user to correct any mistakes made by the tracking algorithm. The optional Division Detection module uses a closely related machine learning process to assign daughter cells to mother cells following cell division events. Two separate modules can finally be used to visualise the output of FAST: the Overlay module plots trajectories and/or the results of analyses over the top of the original images, while the Plotting module contains a range of different options to visualise extracted data. A comprehensive description of each of these modules is provided in section 3 of S1 Text. The FAST GUI guides the user through the process of analysing a single dataset. However, many applications require a large number of imaging datasets to be analysed using consistent settings. To automate this, we have implemented a batch-processing tool called doubleFAST. Once a user has performed an analysis on a single dataset using the FAST GUI, doubleFAST can then read the settings used during this initial run and automatically apply them to any number of additional datasets. This allows data from multiple experiments to be processed with minimal amounts of additional user input and ensures each has been analysed using consistent settings. Finally, we have implemented a post-processing toolbox, which contains scripts and functions to perform a number of different tasks on FAST’s output. These allow, among other things, conversion of track data to other file formats, automated detection of different genotypes, and the annotation of events such as reversals in movement direction. Users of FAST can suggest new additions to this toolbox so they can be used by the wider community. FAST’s tracking algorithm One of FAST’s principal innovations is its tracking algorithm, which automatically determines how to best combine data from a variety of different cell features to improve tracking fidelity. Although previous tracking software packages have had the option to incorporate cell characteristics other than position in their tracking routines [16,19], FAST uses a conceptual framework based on information theory that optimises this process, increasing both the power and convenience of this approach. In this section we provide a high-level overview of our main innovations and how they impact the tracking process. For more detailed derivations and explanations, please refer to section 3.4 of S1 Text. Our approach to the tracking problem is based on measurement of the amount of information that can be used to assign links between objects in subsequent frames, a measurement we call the ‘trackability’. This trackability score provides an integrated measure of how accurately we can follow objects from frame to frame, and due to its grounding in information theory has certain desirable properties such as the additivity of contributions from statistically independent features [32,33]. Measuring the trackability over time therefore provides users with a tool to predict when tracking will be stable and robust, as well as flagging portions of a dataset that might yield spurious trajectories. We define trackability in the following paragraphs and then illustrate how this metric is used in practice in our first and second case studies. We begin by assuming that N different features are measured for each object. An object’s features can be expressed as a vector with N elements, where the object index is denoted as i and time is denoted as t. Changes in an object’s features over time—corresponding to, for example, translational movement or changes in fluorescent intensity—therefore generate a trajectory through the N-dimensional feature space. The goal of our tracking algorithm is to reconstruct each object’s trajectory from the cloud of individual data points resulting from the segmentation and measurement of objects in each image. As previously noted, individuals in high-density and high-motility systems can easily move further than the typical cell-cell separation between frames, making it difficult to reconstruct their trajectories from positional information alone. By including additional features in the tracking framework, we expand the feature space from these two spatial dimensions to N feature dimensions, creating new axes along which one can potentially discriminate neighbouring individuals from each other. The trackability metric provides an estimate of how distinguishable trajectories are from each other in the feature space, and therefore how accurate tracking is likely to be. Unpredictable movement of objects tends to reduce trackability, while trackability increases if the features sample a wider range of values (i.e. if they have a larger dynamic range). To formalise this, we model an object’s instantaneous position in feature space as the random vector Xt and the change in this position between subsequent images as the random vector ΔXt. The distribution f(x) is then the probability density function (PDF) representing the chance of finding a randomly selected object at a particular position x in the absence of additional information (specifically, the position of the object at prior timepoints), while the distribution f(Δx) represents the stochastic change in an object’s position in feature space between sequential timepoints. We estimate f(Δx) by assuming that the motion of an object through feature space can be modelled as a Gaussian random walk. Explicitly, we assume that the feature vector of an object at frame t+1 can be written in terms of its prior feature vector as: (1) where ΔXt is modelled as a multivariate normal , and μt(Δx) and Σt(Δx) are respectively the mean vector and covariance matrix of the set of frame-frame feature displacements {Δxt}. We can similarly characterize f(x) using the covariance matrix of the raw object locations, Σt(x). While we can estimate Σt(x) directly from static snapshots as the covariance of the set of feature vectors {xt}, resolving {Δxt} and subsequently μt(Δx) and Σt(Δx) requires a putative set of cell trajectories, which in our algorithm are obtained via a preliminary round of tracking that uses a simple nearest-neighbour approach. To help ensure these summary statistics are measured accurately, FAST uses only the lowest displacement trajectory links from this training dataset, which are the ones most likely to be correct. Users specify the fraction F of the links with the smallest displacements to be used in the calculation of μt(Δx) and Σt(Δx). This allows the user to balance the trade-off between larger values of F, which increases the size of the training dataset, and smaller values of F which increases the quality of the training dataset. From these measurements, we estimate the amount of information available for assigning objects between frames by calculating the difference between the entropies for the two distributions, H(X) and H(ΔX) [33,34]. The trackability rt, which is measured in bits/object, can then be written as: (2) where no,t is the total number of objects present at time t and |∙| denotes the determinant of the contents. The first term captures the balance between the dynamic range of all features (|Σt(Δx)|) and the unpredictability of an object’s movement through feature space (|Σt(Δx)|), while the second and third terms rescale this quantity to express the amount of information available to the tracking algorithm per object. For a further details, including our assumptions about the form of the underlying statistical distributions, please see section 3.4.2 of S1 Text. To illustrate the trackability metric, we consider the case of a single object with a single feature, for example its position in space along a single axis, x (Fig 2A–2C). This simplifies Eq (2) to: (3) where σt(x) and σt(Δx) denote the standard deviation of their respective distributions. As expected, one observes a larger trackability score when the distribution of feature displacements, f(Δx), is more sharply peaked relative to the distribution of f(x) (Fig 2B and 2C). i.e. when the typical distance an object moves between frames is small compared to the range of values of x. Intuitively, the precision of tracking will increase as (i) the size of the random fluctuations in feature space decreases, (ii) the number of objects within a frame decreases, and (iii) the total size of the feature space the objects occupy (i.e. their dynamic ranges) increases. By taking these multiple factors into account, rt represents an integrated measurement of the risk that a given object will be incorrectly linked to a different object in a subsequent frame. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. An information-theoretic framework for automated object tracking. (A) For illustrative purposes, we consider here a theoretical dataset in which an object is characterised using a single feature, its position along the x-axis. The object’s position at three successive timepoints is denoted x1, x2, x3 (red circles), while the displacements are denoted Δx1, Δx2 (blue arrows). (B) We assume that feature displacements are drawn from a Normal distribution f(Δx), while the instantaneous object position (independent of knowledge of other timepoints) is drawn from a separate Uniform distribution f(x). The information content I of the feature is then calculated as the difference in the entropies of the two distributions, H(X) and H(ΔX), and represents our increase in certainty about the position of the object at time t+1 given knowledge about its position at time t. The trackability quantifies the total amount of information measured for each object, which increases when f(Δx) exhibits less variability relative to f(x). (C) Trackability decreases when the distribution of f(Δx) is broader (i.e. the feature becomes more ‘noisy’). Here the different colours correspond to the different distributions of f(Δx) shown in panel B. (D,E) Illustration of the feature normalization process for two features. In both D and E, the central plot indicates the joint distribution of a pair of feature displacements, while the left and bottom plots indicate the corresponding marginal distributions. In D, the random variables representing the unnormalized frame-frame displacements of the two features—ΔX and ΔY—are correlated and displaced from the origin. Using the joint distribution’s covariance matrix Σ(Δx) and mean vector μ(Δx), the feature space is transformed such that the resulting joint distribution of feature displacements is zero-centred and isotropic (E), ensuring that each feature exhibits an equivalent amount of stochastic variation between frames. https://doi.org/10.1371/journal.pcbi.1011524.g002 In addition to calculating the trackability, μt(Δx) and Σt(Δx) are also used to perform what we call ‘feature normalization’. This transformation converts the raw feature space x to a normalized feature space with a corresponding displacement distribution that is isotropic and zero-centred (Fig 2D and 2E), thus ensuring that the stochastic variation observed within each component of is equal and that any predictable motion in the feature space (e.g. a gradual increase in cell length due to growth, or a reduction in fluorescent intensity due to photobleaching) is accounted for. The metric of this transformed space is the Mahalanobis distance, a dimensionless measure of how reliably we can predict where an object will appear in the feature space at the next time point under the assumptions of our statistical framework. By providing an equitable way to combine data from different features together, this metric allows us to more accurately distinguish correct from incorrect putative links. Large distances between sequential timepoints in this space indicate a discrepancy between the predicted and observed location of an object in the next frame, and so suggest that the input data must be erroneous—for example, because an object was mis-segmented in a single frame. In contrast to previous approaches where feature weightings have to be selected or measured manually [16], feature normalization allows one to automatically optimise the contribution of multiple features without any additional user input. Finally, we use these statistical measurements to also calculate the adaptive tracking threshold βt, which automatically adjusts the stringency of the algorithm that links objects together based on how much information is available in each frame. While a larger fraction of putative links are accepted when information is relatively plentiful, only the highest certainty links are accepted when information is more limited. This dynamic adjustment of the link threshold thus allows FAST to maximise the number of trajectories in less challenging tracking conditions (e.g. slow-moving cells at low density) while minimizing the number of spurious links in more challenging conditions (e.g. fast-moving cells at high density). As different imaging datasets may require a different balance between maximising the number of correct links and minimising the number of spurious links, users select a static threshold P to balance this trade-off, which FAST then automatically converts to the time-varying threshold βt based on the instantaneous amount of information available. In summary, users must specify the tracking threshold P, the fraction of training links to retain F and the features that should be included in the final round of tracking. To help users optimise these parameters quickly, the Tracking module contains built-in tools that allow users to objectively assess tracking quality on a single pair of frames before committing to processing the entire dataset. Validating the methodology of FAST using ground-truthed datasets To demonstrate the functionality of our tracking algorithm and to illustrate how using multiple features can enhance accuracy, we used a previously described self-propelled rod (SPR) model [1,35] to generate synthetic datasets that simulate bacteria collectively moving at high-density (Fig 3A). In these simulations, cells are modelled as stiff, mutually repulsive rods that are propelled by a constant force (see Methods). This model has been shown to closely approximate bacterial collectives that propel themselves using either flagella or type IV pili, where steric interactions between neighbouring cells generate complex emergent collective behaviours [1,35]. Importantly, this approach allows us to independently control each of the different properties of the system, allowing us to both test FAST on a very large number of qualitatively different datasets and to objectively assess its performance using an automatically generated ground truth. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Validating FAST using synthetic data from a simulation of collective bacterial motility. (A) We used a self-propelled rod (SPR) model to generate noisy datasets with a verified ground truth. We then performed cell tracking with FAST using either only a single feature (the rod’s centroid at sequential timepoints; brown) or using a suite of different features (rod centroid, length, orientation and simulated fluorescent intensity; purple). Comparison of the reconstructed trajectories from FAST (B) with the ground truth data (C) allowed us to identify the errors made by the tracking algorithm (D). Errors were split into two categories: links made between objects that were incorrect (‘false positives’, solid magenta lines), and links between objects that were missed (‘false negatives’, dashed magenta lines). We also calculated the number of correct links made in each case (‘true positives’, solid orange lines). From these counts, we evaluated the performance of the tracking algorithm by calculating the F1-score (main text). (E) The value of this F1-score depends on a user-defined parameter, the tracking threshold. To objectively compare the results from the tracking algorithm when run on datasets with different properties, we calculated the tracking threshold that generated the largest F1-score for a given dataset and used this score in subsequent analyses. (F) Including all feature information substantially and consistently improved tracking performance compared to when only object positions were used (see also S2 Fig). The arrow to the right of the plot (labelled ‘Performance increase’) indicates the median F1-score for the two cases, corresponding to a ~10-fold reduction of tracking errors when all features were used. Furthermore, we found that our trackability metric was an excellent predictor of tracking accuracy for a given set of features, suggesting that it can be used to estimate the accuracy of the algorithm even when a ground truth is not available. In the different simulations we varied rod density, propulsive force, and framerate, as well the amount of noise in the measurement of position, length and fluorescence (Table 1). The black circles in F correspond to the dataset presented in E. https://doi.org/10.1371/journal.pcbi.1011524.g003 We simulated a 2D monolayer of cells constitutively expressing a fluorescent protein by initialising populations of cells whose length and fluorescent intensity were drawn from distributions obtained from experimental data (S1 Fig). Following initialization, we simulated cell movement by numerically integrating the equations of motion for each rod. Once the system had reached steady-state, we extracted measurements of rod position, orientation, fluorescence and length at evenly spaced timepoints. To simulate noisy measurements, we added Gaussian noise to each of these features, with the noise magnitude based on that observed in real experimental data (Fig 3B, Methods). Rather than specialising on datasets with specific properties, FAST’s tracking algorithm is designed to be robust to a wide diversity of different conditions by automatically compensating via feature normalization. To test this capability, we varied the parameters of our simulation by adjusting the rod density, self-propulsion force and framerate, as well as the accuracy of feature measurement by adjusting the amount of noise in the measured cell position, length and fluorescent intensity (Table 1). We tested five different values for each of these six parameters, yielding a total of 30 datasets. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Parameters of the SPR model used to generate synthetic datasets. Here we show both range of values we tested and baseline value that was used when we varied another parameter. Note that our simulations are non-dimensionalised, using the width of a single rod as the characteristic lengthscale and the time taken for an isolated rod with self-propulsion force ν = 1 to move a single rod width as the characteristic timescale. https://doi.org/10.1371/journal.pcbi.1011524.t001 The performance of the tracking algorithm was assessed by comparing its output to the ground truth (Fig 3C and 3D). Links that were present in the ground truth but missing in the reconstructions were scored as false negatives (FN), while those that were absent in the ground truth but present in the reconstructions were scored as false positives (FP). Links that were identical in both were scored as true positives (TP). We now integrated these measurements into a single metric that quantifies tracking performance, the F1-score, defined as: (4) Users of FAST specify a tracking threshold that controls the stringency of the linking process. If this threshold is too stringent, too many correct links will rejected by the algorithm, while if the threshold is not stringent enough too many incorrect links will be accepted. In practice, users need the ability to choose the tracking threshold that best suits their needs—for example, users interested in detecting very rare events will have different requirements than those interested in measuring average cell behaviour. However, for the purpose of testing the benefits of our automated tracking algorithm, we removed the tracking threshold as a factor from our analyses by using the threshold that resulted in the largest the F1-score for each dataset (Fig 3E). This maximum F1-score is an objective measure of the performance of our tracking algorithm across datasets with different characteristics, and also allows us to robustly benchmark our tracking algorithm. The simulated datasets were tracked using two different methods (Fig 3B). The first method only used the position of the rods—effectively a classical position-based tracking approach (‘Centroids’), while the second method used all four features—position, orientation, length and fluorescent intensity (‘All features’). We found that including all features led to a dramatic increase in tracking accuracy in all datasets, with a 4 to 10-fold reduction in the number of tracking errors (S2 Fig). Furthermore, our groundtruthed datasets allow us to directly relate a dataset’s trackability to the accuracy of the resulting tracks, which demonstrated that it is an excellent predictor of the F1-score regardless of the simulation parameters or level of noise in the dataset (Fig 3F). Taken together, these analyses validate our methodology and illustrate how FAST copes with challenging datasets. First, they demonstrate that our method of integrating additional feature information during tracking is reliable and effective, allowing FAST to substantially reduce tracking errors compared to alternative approaches that use cell position alone. Second, these analyses demonstrate that our trackability score can integrate multiple aspects of the dataset into a single, robust heuristic of predicted tracking accuracy. Case studies Tracking cells, divisions, and lineages in growing non-motile E. coli colonies. A major aim of a number many previous tracking packages is to automatically reconstruct cell lineages [25,28]. However, identifying cell division events and the resulting daughter cells is a particularly difficult challenge. Near-perfect tracking of individuals is essential for accurate lineage tracking because single errors can propagate and cause lineages to be misassigned at later timepoints. We tested FAST’s lineage tracking capabilities using eight separate time lapse movies of E. coli cells as they divided to form microcolonies [11]. These experiments were imaged using a combination of phase-contrast and fluorescence microscopy, and used a strain with a GFP transcriptional reporter to quantify the level of colicin Ib (cib) expression. The expression of this gene is highly variable between cells but remains stable over a single generation. This combination of population-level heterogeneity and individual-level stability means that the level of GFP expression in each cell could contribute a large amount of information to FAST’s tracking algorithm and thus substantially improve its performance. We began by applying automated tracking and division detection to a single dataset. We visualised the lineage structure of the microcolony using the Overlays module to rapidly verify the accuracy of the lineage assignment (Fig 4A and 4B, S1 Movie). This revealed that the descendants of each of the individual cells present at the beginning of the experiment formed highly elongated structures within the colony, similar to the patterns previously observed in experiments where colonies were initiated from multiple founder cells labelled with different fluorescent proteins [8]. Next, we used our batch processing tool—doubleFAST—to automate the analysis of the remainder of the eight datasets, using the parameters obtained when analysing the first microcolony. A complete breakdown of processing times by module and dataset is shown in Table 2—in total, processing of all eight datasets took 5.5 minutes, or around 40 seconds per microcolony on a standard laptop computer (Methods). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Processing times and dimensions of lineage datasets. ‘Dimensions’ indicates the size of the raw imaging dataset as x-size, y-size and duration. x- and y-sizes are recorded in pixels, while duration is recorded as the number of frames. ‘Objects’ indicates the total number of segmented objects in the manually corrected segmentations, while ‘Cells’ indicates the total number of separate cell trajectories in the manually corrected track dataset. The ‘Processing time’ section reports the amount of time taken to process each dataset by each module of FAST (in seconds) on a standard laptop computer (see Methods for details). https://doi.org/10.1371/journal.pcbi.1011524.t002 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Benchmarking FAST’s performance using an experimental dataset containing eight E. coli microcolonies. (A) Tracks showing the movement of individual cells within a typical microcolony as it expands, where colours denote the instantaneous cell length. (B) Automated lineage tracking of microcolony at three different timepoints. Cells that share the same mother in generation 1 (‘Gen 1’) are labelled with the same colour, illustrating how the spatial distribution of different lineages develops over time. Occasional black cells in the final timepoint (‘Gen 6–7’) indicate cells that were not assigned to the correct lineage. (C) Images from eight different microcolonies were processed with FAST to automatically obtain trajectories and division events. These were then compared to a manually curated ground truth to calculate F1-scores, quantifying the performance of the Segmentation module (triangles), Tracking module (filled circles) and Division Detection module (empty circles). These analyses proceeded in two parallel streams: In the ‘corrected’ stream, the inputs of the Tracking and Division Detection modules had been manually corrected, whereas in the ‘automated’ stream these inputs were obtained directly from the previous module and remained uncorrected. (D) When averaged over all timepoints that contained more than 32 cells, we found that the trackability metric was a reliable predictor of the accuracy of both the Tracking and Division Detection modules under fully automated conditions. In addition, this analysis showed that including cell length, width, fluorescent intensity and position as features in the Tracking module (‘All features’, purple) reduced the median number of errors in both the Tracking and Division Detection modules by approximately 5-fold compared to results obtained using cell position alone (‘Centroids only’, brown). (E-H) We next investigated which factors influence trackability by comparing metrics related to fluctuations in feature values–average cell speed (E), relative elongation rate (the fractional change in cell length per min) (F), coefficient of variation (COV) of GFP intensity (G) and COV of cell width (H). The statistical significance of correlations is denoted using the following symbols: * denotes p < 0.05, ** denotes p < 0.005, n.s. = not significant. These analyses were performed using the log-transformed F1-scores in D and are based on a linear regression t-test. The orange points in C-H indicate the exemplar microcolony shown in A and B. Visualisations shown in A and B were produced using FAST’s Overlays module. https://doi.org/10.1371/journal.pcbi.1011524.g004 Although these automated analyses produced highly accurate results, a few tracking and division assignment errors were observed (Fig 4B). To objectively assess the accuracy of FAST’s algorithms, we benchmarked our results by constructing manually-validated ground-truth versions of the Segmentation, Tracking and Division-Detection module outputs. In the case of the Tracking module, this task was substantially facilitated by our Track Correction GUI. Our analyses then proceeded via two separate streams: in the first, ‘corrected’ stream, we used the manually corrected output of the Segmentation module as an input to the Tracking module, and the manually corrected output of the Tracking module as an input to the Division Detection module, allowing us to assess the quality of each module independently. In the second, ‘automated’ stream, we used the uncorrected outputs of each module as the input to the next. This allows us to assess the performance of the tracking and division detection algorithms in conditions typical of large-scale analyses, where errors from earlier analyses are propagated uncorrected into later steps (Fig 4C). We found that the results of our Segmentation module were highly accurate, with a median mis-segmentation rate of 0.52%. The Tracking module was similarly accurate, with a median error rate of 1.02% when using the uncorrected segmentations as inputs. This is very similar to the error rate reported for DeLTA 2.0 [36], which was also benchmarked against microcolony datasets from van Vliet et al. [11]. We note, however, that the average error rate reported for DeLTA 2.0 is an aggregate measure that also includes datasets from additional E. coli genotypes in van Vliet et al. [11] not considered here. Manually correcting FAST’s rare mis-segmentations had a relatively small impact on the Tracking module’s performance, decreasing the median error rate to 0.71%, suggesting that the tracking algorithm’s automated system for bridging mis-segmentations performs as intended. In contrast, the Division Detection module’s performance was considerably enhanced when the input trajectories were manually corrected, with the median error rate decreasing from 4.77% to 0.63%. We therefore recommend using our track correction GUI if highly accurate lineage assignment is required. We also analysed the utility of trackability as a predictor of tracking algorithm performance, equivalent to our automated benchmarking with the SPR model (Fig 4D). As before, we found that trackability averaged over all timepoints with a relatively large number of cells (>32) was robust predictor of the tracking performance on the different microcolonies. This was even true for individual timepoints, with trackability being a much better predictor of tracking fidelity than more rudimentary metrics such as the current number of cells within the microcolony (S3 Fig). We also found that trackability was a strong predictor of how accurately divisions were detected (Fig 4D). Lastly, we measured the performance of the tracking and division detection algorithms when a large suite of features was used (cell length, width, fluorescence, and position; denoted ‘All features’) compared to when only cell positions were used (denoted ‘Centroids only’). Similar to what was previously observed in the synthetic datasets, adding extra features substantially increased the performance of our algorithms, reducing the number of tracking and division assignment errors by approximately 5-fold. Given that the eight microcolonies that we analysed used the same strain of bacteria and were grown in the same experimental conditions, we were surprised by the variations in tracking performance that we observed across the eight microcolonies. One potential cause of this variation was that the user-defined tracking parameters which were optimised for one dataset were not optimal for the other microcolonies. However, when we systematically repeated the benchmarks with different user-defined tracking parameters, we found that the set of tracking parameters that produced the most accurate results was highly conserved across the eight microcolonies (S4 Fig), suggesting this was not the primary cause. Next, we asked whether biological differences between the samples could be responsible for the observed variation in performance. To test this, we measured each microcolony’s average cell speed, relative elongation rate (the average fractional increase in cell length per minute), coefficient of variation (COV) of GFP fluorescence and the COV of cell width using our manually corrected tracks, and investigated how these metrics related to the microcolony’s trackability (Fig 4E–4H). We found that the average rate at which cells move and grow within a microcolony were significantly correlated with the microcolony’s trackability. In contrast, the amount of variability observed in the cell fluorescence and cell width did not have an appreciable impact on a microcolony’s trackability. Taken together, these results indicate that the variation in the trackability metric we observed were driven by real biological variation across the different microcolonies. In this case differences in the average growth rate of cells in each microcolony likely drive changes in the speed at which these non-motile colonies expand, and consequently impacts the fidelity of tracking and division detection. In summary, these analyses show that segmentation errors do not substantially affect the tracking performance of FAST, but errors in tracking can detrimentally impact the detection of cell division events. Moreover, these analyses of experimental data validate our previous findings from the synthetic datasets by demonstrating firstly that tracking performance can be substantially improved by including cell features other than cell position and secondly that trackability is a reliable predictor of tracking fidelity. Finally, this case study demonstrates that the two user-defined tracking parameters F and P do not need to be excessively fine-tuned to obtain reliable tracking results (S4 Fig, note the logarithmically-spaced axes). Quantifying rapid bursts of cell movement in densely packed P. aeruginosa monolayers. Many different species of bacteria generate collective motility in densely-packed communities using either flagella, Type IV pili, or via gliding [2,15,35,37]. Motility allows populations to rapidly expand into new territory, giving them a competitive advantage over non-motile genotypes [1,38]. Here we demonstrate how FAST can be used to quantify the behaviour of P. aeruginosa cells in interstitial colonies that form between agar and glass [37]. We focus on cells within the monolayer that forms directly behind the colony’s leading edge, the dynamics of which play a crucial role in the competition between genotypes in both interstitial and more classical ‘surficial’ colonies [1]. Tracking cells undergoing collective movement requires a much larger acquisition frame rate compared to non-motile cells. While expensive timing boards can be used to ‘trigger’ cameras with a high level of precision to keep the time between frames nearly constant, many high-end research microscopes lack this capability. Instead, ‘camera streaming’ is more widely available, which directly streams the camera’s output to a computer which saves frames as fast as possible. This maximises the frame rate, but images acquired via camera streaming can have slight variations in the time that elapses between subsequent frames. To illustrate how FAST can be used to handle a sequence of images collected via camera streaming, we collected a large dataset with 3,505 frames recorded at an average rate of 127 frames per second. The size is of this dataset is approximately 8 Gb and each frame contains approximately 1,700 tightly packed P. aeruginosa cells. Despite the large size of this dataset, processing with FAST could be completed on a standard laptop computer (Methods), requiring only 200 mins to segment and 355 mins to extract the features of each of the ~6 million individual objects. Tracking consists of two separate stages: the model training stage, and the link assignment stage. After completion of the model training stage, FAST automatically generates and plots the trackability of the dataset at each timepoint (Fig 5A). For our dataset, this plot revealed that the trackability dropped precipitously at some timepoints. We hypothesised that these decreases resulted from a reduction in the imaging framerate. To test this, we plotted the trackability score against ∆t, the elapsed time between frames as calculated from timestamps (Fig 5A, inset). This revealed a strong negative correlation (Pearson’s correlation coefficient = -0.705), suggesting that the longer the time between frames, the lower the trackability and the less accurate the tracking results. To avoid these timepoints—and the spurious tracking results they might generate—we used the tracking module’s built-in time window selection tool to specify a subset of frames to track (Fig 5A, green region, 750 frames). Training the tracking model for this reduced dataset took 18 minutes, while tracking and track processing took an additional 88 minutes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Tracking single cells in an interstitial P. aeruginosa colony spreading via pili-based motility. (A) We used FAST to track cells within a monolayer of P. aeruginosa undergoing collective motility using position, length and width as features. Images were collected using high speed ‘camera streaming’ with a mean frame rate of 127 fps, but variations in the elapsed time between subsequent frames, ∆t, resulted in transient reductions in trackability. (A, inset) Analysis of the timestamps associated with each frame revealed that the trackability score was negatively correlated with ∆t (r indicates Pearson’s correlation coefficient; the warmer colours denote a higher density of data points). We therefore restricted our subsequent analyses to a subset of the data in which the trackability score was relatively constant (green region) using FAST’s built-in tools. (B) We used the Overlays module to colour code cells based on their instantaneous speed. Although cells typically moved relatively slowly, very occasionally cells were observed to undergo a very rapid burst of movement (see cream coloured cell). (C) These rapid jumps can also be observed in traces of the speed of individual cells. Here we plot the instantaneous speed of three different cells over time, each in a different colour. The montages above illustrate three of these transient events, using the same colour coding shown in B. (D) To investigate these movements at the population level, we calculated instantaneous cell velocities in both the x and y direction for all cells and plotted their combined distribution. In other active systems, this distribution is approximately Gaussian. However, in our system the highly transient bursts of velocity result in heavy tails, causing them deviate from a Gaussian distribution with the same variance (dashed black line). https://doi.org/10.1371/journal.pcbi.1011524.g005 In this example, a relatively large timestep ∆t allows cells to move further between frames, which reduces predictability and therefore reduces the trackability score. More generally, however, the trackability depends on a combination of both experimental and imaging conditions, providing a robust metric to interpret datasets. For example, the trackability score can be used to quickly identify a wide range of problems which might arise during an experiment, including fluctuations in focus, illumination intensity, or inadvertent movement of the sample. Once problematic timepoints have been identified, the user can decide either to avoid them by using a subset of the images (as in this example) or to reject the entire dataset. Tracking the smaller subset of images that we specified resulted in over 1,600 separate trajectories, each at least 200 timepoints long. To visualise this large dataset, we used the Overlays module to colour individual cells in the phase-contrast image based on their instantaneous speed (Fig 5B, S2 Movie). This revealed that while most cells move at relatively slow speeds, a small number of cells undergo rapid, sporadic bursts of movement approximately 25 times faster than the average. These rapid movements are also clearly visible in measurements of the speed of individual cells over time (Fig 5C). Our results are similar to the ‘slingshots’ previously observed in solitary P. aeruginosa cells moving at the glass/liquid interface, apparently driven by pilus detachment events [39]. However, the peak speeds that we record (up to 60 μm s-1) in collectives of P. aeruginosa are approximately 20 times larger than those previously observed. While we do not know the specific reason for this difference, interactions with neighbouring cells facilitated by the high cell density, the glass/agar interstitial colony environment and our higher framerate (~13 times that of [39]) may each play a role. To illustrate how this large tracking dataset can be mined to elucidate the statistics of rare events, we constructed the instantaneous marginal velocity distribution from our tracks (i.e. the x- and y-components of the instantaneous velocity vectors) (Fig 5D). While previous studies have found that sperm and swimming bacterial cells undergoing collective movement generate marginal velocity distributions that are approximately Gaussian [35,40], we observed that bacteria collectively moving via pili-based motility exhibit much heavier tails corresponding to their occasional rapid motions. Despite the rarity of these events—less than 0.05% of our measurements had a magnitude larger than 20 μm s-1—the exceptional number of cell trajectories we obtained nevertheless allowed us to finely resolve their statistical distribution. These analyses demonstrate how FAST can be used to rapidly characterise the motility of a large number of cells in densely-packed conditions, with relatively little user input and computational effort. Automated analysis of T6SS battles. To illustrate FAST’s capabilities beyond standard cell tracking, we analysed the activity of the Type 6 Secretion System (T6SS) using FAST. The T6SS is a highly dynamic organelle composed of a molecular ‘spear’ tipped with a toxin known as an effector [41]. Bacteria that possess the T6SS can inject this effector into neighbouring cells, which kills them [42]. Firing events can be monitored by visualising the localisation of the protein that forms the contractile sheath of the T6SS needle, TssB in P. aeruginosa and VipA in Vibrio cholerae [12,43]. Here, we highlight how FAST can be used to quantify how the T6SS is regulated in co-cultures of V. cholerae and P. aeruginosa. Because of its short range, the T6SS is effective only at high cell density, which historically has made it difficult to study using automated analyses. We used FAST to analyse imaging datasets that show V. cholerae and P. aeruginosa cells expressing VipA-mCherry and TssB-mNeongreen, respectively, interacting through their respective T6SSs when mixed together in a densely-packed monolayer [12] (Fig 6A, left). We distinguished the two species from one another (Fig 6A, right) using FAST’s post-processing toolbox, which compares each cell’s intensity in the mCherry and mNeonGreen channels to automatically assign them to two distinct populations (Fig 6B). This utility allows one to rapidly compare the behaviour of different genotypes within the same sample. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Identifying different bacterial species, quantifying cell morphologies, and investigating T6SS dynamics in a densely packed collective. (A) A monolayer composed of a mixture of P. aeruginosa cells expressing TssB-mNeongreen and V. cholerae cells expressing VipA-mCherry (left). Measurements of the average intensity of each cell in the two different fluorescence channels allowed us to automatically distinguish the two species (B), an approach known as image cytometry. By colour coding each cell (P. aeruginosa, cyan, V. cholerae, red), we were able to visually confirm the accuracy of the assignment process (A, right). (C) We then quantified a novel feature, cell curvature, using FAST’s custom feature extraction framework to calculate the curvature of the segmentation backbones (inset). Splitting these measurements into the two previously assigned populations showed the comma shaped V. cholerae cells were substantially more curved than the rod-shaped P. aeruginosa cells. (D-F) We then used FAST’s plotting options to illustrate the dynamics of the T6SS. A P. aeruginosa cell fires its T6SS machinery (observed as the bright puncta of TssB), which can be visualised using the kymograph (D) and ‘cartouche’ (F) plotting options, and which also leads to an increase in the standard deviation of the mNeongreen channel (E). (G) We used this latter measurement to detect when the T6SS was active in a cell, allowing us to measure the proportion of time that each cell in the population spends in the firing state. Our analyses confirmed that when co-cultured with a strain of V. cholerae with an inactive T6SS (Δhcp1 Δhcp2), the T6SS activity of WT P. aeruginosa cells is dramatically reduced compared to when they are co-cultured with WT V. cholerae. Here the circles show averages from separate movies and horizontal lines show the mean for all samples of a given combination of genotypes. https://doi.org/10.1371/journal.pcbi.1011524.g006 The architecture of FAST allows users to easily define and quantify novel features that are not already included in FAST. To do this, users can write a short feature measurement script that can make use of the segmentation of each object in each frame, as well as the corresponding regions in each of the imaging channels—in this case, the mNeongreen, mCherry and phase-contrast signals. The Feature Extraction module then automatically stores these custom features in the same way as the built-in features, allowing them to be integrated into downstream analyses. To illustrate this capability, we extracted cell curvature as a custom feature to see if we could detect the differences in morphology of V. cholerae cells (which are comma shaped) from that of P. aeruginosa (which are rod shaped). Using a skeletonization-based approach, we extracted the morphological backbone of each cell from its segmentation and measured its curvature by finding the best-fit circle to this set of points. Consistent with expectations, our automated analysis found that the cells identified as V. cholerae were substantially more curved than those identified as P. aeruginosa (Fig 6C). Our software is also capable of generating sophisticated visualisations and analyses of the behaviour of single cells. For example, FAST allows the dynamics of T6SS firing in individual cells to be easily visualised using the kymograph option of the Plotting module (Fig 6D), as well as the ‘cartouche’ option, which extracts and aligns cropped images of the specified cells (Fig 6F). We can also quantify the dynamics of the T6SS using the feature data associated with a given cell: the standard deviation of the mNeongreen channel substantially increases during firing (Fig 6E), as the TssB becomes non-uniformly distributed through the cell as it assembles into the sheath. It is generally accepted that P. aeruginosa only fires its T6SS when triggered by the firing of the T6SS of a neighbouring cell [9,12]. Although qualitative evidence for this is strong, to our knowledge this effect has never been directly quantified, likely because of the difficulties involved with tracking densely packed cells and because the T6SS firing events are themselves relatively rare. We therefore decided to use FAST to simultaneously track a large number of densely packed cells and automatically detect when each fires its T6SS. We used doubleFAST to automate the analysis of multiple datasets from experiments in which WT P. aeruginosa cells were either co-cultured with WT V. cholerae or co-cultured with a mutant V. cholerae strain that is incapable of firing their T6SS (Δhcp1 Δhcp2). Using the standard deviation of the mNeongreen channel to distinguish when P. aeruginosa is actively firing its T6SS, we calculated the proportion of time that P. aeruginosa cells spend firing their T6SS (Fig 6G, S3 Movie). As expected, when co-cultured with the inactivated Δhcp1 Δhcp2 V. cholerae strain, P. aeruginosa dramatically reduced the proportion of time it spent firing its T6SS compared to when co-cultured with WT V. cholerae. These analyses generated 1,246 separate trajectories, of which around half (601) spanned the full 151 timepoints of the experiments. In conclusion, this case study demonstrates how FAST can automatically distinguish different species, how users can specify novel features like cell curvature, and how complex bacterial behaviours can be quantified using feature data. Tracking cells, divisions, and lineages in growing non-motile E. coli colonies. A major aim of a number many previous tracking packages is to automatically reconstruct cell lineages [25,28]. However, identifying cell division events and the resulting daughter cells is a particularly difficult challenge. Near-perfect tracking of individuals is essential for accurate lineage tracking because single errors can propagate and cause lineages to be misassigned at later timepoints. We tested FAST’s lineage tracking capabilities using eight separate time lapse movies of E. coli cells as they divided to form microcolonies [11]. These experiments were imaged using a combination of phase-contrast and fluorescence microscopy, and used a strain with a GFP transcriptional reporter to quantify the level of colicin Ib (cib) expression. The expression of this gene is highly variable between cells but remains stable over a single generation. This combination of population-level heterogeneity and individual-level stability means that the level of GFP expression in each cell could contribute a large amount of information to FAST’s tracking algorithm and thus substantially improve its performance. We began by applying automated tracking and division detection to a single dataset. We visualised the lineage structure of the microcolony using the Overlays module to rapidly verify the accuracy of the lineage assignment (Fig 4A and 4B, S1 Movie). This revealed that the descendants of each of the individual cells present at the beginning of the experiment formed highly elongated structures within the colony, similar to the patterns previously observed in experiments where colonies were initiated from multiple founder cells labelled with different fluorescent proteins [8]. Next, we used our batch processing tool—doubleFAST—to automate the analysis of the remainder of the eight datasets, using the parameters obtained when analysing the first microcolony. A complete breakdown of processing times by module and dataset is shown in Table 2—in total, processing of all eight datasets took 5.5 minutes, or around 40 seconds per microcolony on a standard laptop computer (Methods). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Processing times and dimensions of lineage datasets. ‘Dimensions’ indicates the size of the raw imaging dataset as x-size, y-size and duration. x- and y-sizes are recorded in pixels, while duration is recorded as the number of frames. ‘Objects’ indicates the total number of segmented objects in the manually corrected segmentations, while ‘Cells’ indicates the total number of separate cell trajectories in the manually corrected track dataset. The ‘Processing time’ section reports the amount of time taken to process each dataset by each module of FAST (in seconds) on a standard laptop computer (see Methods for details). https://doi.org/10.1371/journal.pcbi.1011524.t002 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Benchmarking FAST’s performance using an experimental dataset containing eight E. coli microcolonies. (A) Tracks showing the movement of individual cells within a typical microcolony as it expands, where colours denote the instantaneous cell length. (B) Automated lineage tracking of microcolony at three different timepoints. Cells that share the same mother in generation 1 (‘Gen 1’) are labelled with the same colour, illustrating how the spatial distribution of different lineages develops over time. Occasional black cells in the final timepoint (‘Gen 6–7’) indicate cells that were not assigned to the correct lineage. (C) Images from eight different microcolonies were processed with FAST to automatically obtain trajectories and division events. These were then compared to a manually curated ground truth to calculate F1-scores, quantifying the performance of the Segmentation module (triangles), Tracking module (filled circles) and Division Detection module (empty circles). These analyses proceeded in two parallel streams: In the ‘corrected’ stream, the inputs of the Tracking and Division Detection modules had been manually corrected, whereas in the ‘automated’ stream these inputs were obtained directly from the previous module and remained uncorrected. (D) When averaged over all timepoints that contained more than 32 cells, we found that the trackability metric was a reliable predictor of the accuracy of both the Tracking and Division Detection modules under fully automated conditions. In addition, this analysis showed that including cell length, width, fluorescent intensity and position as features in the Tracking module (‘All features’, purple) reduced the median number of errors in both the Tracking and Division Detection modules by approximately 5-fold compared to results obtained using cell position alone (‘Centroids only’, brown). (E-H) We next investigated which factors influence trackability by comparing metrics related to fluctuations in feature values–average cell speed (E), relative elongation rate (the fractional change in cell length per min) (F), coefficient of variation (COV) of GFP intensity (G) and COV of cell width (H). The statistical significance of correlations is denoted using the following symbols: * denotes p < 0.05, ** denotes p < 0.005, n.s. = not significant. These analyses were performed using the log-transformed F1-scores in D and are based on a linear regression t-test. The orange points in C-H indicate the exemplar microcolony shown in A and B. Visualisations shown in A and B were produced using FAST’s Overlays module. https://doi.org/10.1371/journal.pcbi.1011524.g004 Although these automated analyses produced highly accurate results, a few tracking and division assignment errors were observed (Fig 4B). To objectively assess the accuracy of FAST’s algorithms, we benchmarked our results by constructing manually-validated ground-truth versions of the Segmentation, Tracking and Division-Detection module outputs. In the case of the Tracking module, this task was substantially facilitated by our Track Correction GUI. Our analyses then proceeded via two separate streams: in the first, ‘corrected’ stream, we used the manually corrected output of the Segmentation module as an input to the Tracking module, and the manually corrected output of the Tracking module as an input to the Division Detection module, allowing us to assess the quality of each module independently. In the second, ‘automated’ stream, we used the uncorrected outputs of each module as the input to the next. This allows us to assess the performance of the tracking and division detection algorithms in conditions typical of large-scale analyses, where errors from earlier analyses are propagated uncorrected into later steps (Fig 4C). We found that the results of our Segmentation module were highly accurate, with a median mis-segmentation rate of 0.52%. The Tracking module was similarly accurate, with a median error rate of 1.02% when using the uncorrected segmentations as inputs. This is very similar to the error rate reported for DeLTA 2.0 [36], which was also benchmarked against microcolony datasets from van Vliet et al. [11]. We note, however, that the average error rate reported for DeLTA 2.0 is an aggregate measure that also includes datasets from additional E. coli genotypes in van Vliet et al. [11] not considered here. Manually correcting FAST’s rare mis-segmentations had a relatively small impact on the Tracking module’s performance, decreasing the median error rate to 0.71%, suggesting that the tracking algorithm’s automated system for bridging mis-segmentations performs as intended. In contrast, the Division Detection module’s performance was considerably enhanced when the input trajectories were manually corrected, with the median error rate decreasing from 4.77% to 0.63%. We therefore recommend using our track correction GUI if highly accurate lineage assignment is required. We also analysed the utility of trackability as a predictor of tracking algorithm performance, equivalent to our automated benchmarking with the SPR model (Fig 4D). As before, we found that trackability averaged over all timepoints with a relatively large number of cells (>32) was robust predictor of the tracking performance on the different microcolonies. This was even true for individual timepoints, with trackability being a much better predictor of tracking fidelity than more rudimentary metrics such as the current number of cells within the microcolony (S3 Fig). We also found that trackability was a strong predictor of how accurately divisions were detected (Fig 4D). Lastly, we measured the performance of the tracking and division detection algorithms when a large suite of features was used (cell length, width, fluorescence, and position; denoted ‘All features’) compared to when only cell positions were used (denoted ‘Centroids only’). Similar to what was previously observed in the synthetic datasets, adding extra features substantially increased the performance of our algorithms, reducing the number of tracking and division assignment errors by approximately 5-fold. Given that the eight microcolonies that we analysed used the same strain of bacteria and were grown in the same experimental conditions, we were surprised by the variations in tracking performance that we observed across the eight microcolonies. One potential cause of this variation was that the user-defined tracking parameters which were optimised for one dataset were not optimal for the other microcolonies. However, when we systematically repeated the benchmarks with different user-defined tracking parameters, we found that the set of tracking parameters that produced the most accurate results was highly conserved across the eight microcolonies (S4 Fig), suggesting this was not the primary cause. Next, we asked whether biological differences between the samples could be responsible for the observed variation in performance. To test this, we measured each microcolony’s average cell speed, relative elongation rate (the average fractional increase in cell length per minute), coefficient of variation (COV) of GFP fluorescence and the COV of cell width using our manually corrected tracks, and investigated how these metrics related to the microcolony’s trackability (Fig 4E–4H). We found that the average rate at which cells move and grow within a microcolony were significantly correlated with the microcolony’s trackability. In contrast, the amount of variability observed in the cell fluorescence and cell width did not have an appreciable impact on a microcolony’s trackability. Taken together, these results indicate that the variation in the trackability metric we observed were driven by real biological variation across the different microcolonies. In this case differences in the average growth rate of cells in each microcolony likely drive changes in the speed at which these non-motile colonies expand, and consequently impacts the fidelity of tracking and division detection. In summary, these analyses show that segmentation errors do not substantially affect the tracking performance of FAST, but errors in tracking can detrimentally impact the detection of cell division events. Moreover, these analyses of experimental data validate our previous findings from the synthetic datasets by demonstrating firstly that tracking performance can be substantially improved by including cell features other than cell position and secondly that trackability is a reliable predictor of tracking fidelity. Finally, this case study demonstrates that the two user-defined tracking parameters F and P do not need to be excessively fine-tuned to obtain reliable tracking results (S4 Fig, note the logarithmically-spaced axes). Quantifying rapid bursts of cell movement in densely packed P. aeruginosa monolayers. Many different species of bacteria generate collective motility in densely-packed communities using either flagella, Type IV pili, or via gliding [2,15,35,37]. Motility allows populations to rapidly expand into new territory, giving them a competitive advantage over non-motile genotypes [1,38]. Here we demonstrate how FAST can be used to quantify the behaviour of P. aeruginosa cells in interstitial colonies that form between agar and glass [37]. We focus on cells within the monolayer that forms directly behind the colony’s leading edge, the dynamics of which play a crucial role in the competition between genotypes in both interstitial and more classical ‘surficial’ colonies [1]. Tracking cells undergoing collective movement requires a much larger acquisition frame rate compared to non-motile cells. While expensive timing boards can be used to ‘trigger’ cameras with a high level of precision to keep the time between frames nearly constant, many high-end research microscopes lack this capability. Instead, ‘camera streaming’ is more widely available, which directly streams the camera’s output to a computer which saves frames as fast as possible. This maximises the frame rate, but images acquired via camera streaming can have slight variations in the time that elapses between subsequent frames. To illustrate how FAST can be used to handle a sequence of images collected via camera streaming, we collected a large dataset with 3,505 frames recorded at an average rate of 127 frames per second. The size is of this dataset is approximately 8 Gb and each frame contains approximately 1,700 tightly packed P. aeruginosa cells. Despite the large size of this dataset, processing with FAST could be completed on a standard laptop computer (Methods), requiring only 200 mins to segment and 355 mins to extract the features of each of the ~6 million individual objects. Tracking consists of two separate stages: the model training stage, and the link assignment stage. After completion of the model training stage, FAST automatically generates and plots the trackability of the dataset at each timepoint (Fig 5A). For our dataset, this plot revealed that the trackability dropped precipitously at some timepoints. We hypothesised that these decreases resulted from a reduction in the imaging framerate. To test this, we plotted the trackability score against ∆t, the elapsed time between frames as calculated from timestamps (Fig 5A, inset). This revealed a strong negative correlation (Pearson’s correlation coefficient = -0.705), suggesting that the longer the time between frames, the lower the trackability and the less accurate the tracking results. To avoid these timepoints—and the spurious tracking results they might generate—we used the tracking module’s built-in time window selection tool to specify a subset of frames to track (Fig 5A, green region, 750 frames). Training the tracking model for this reduced dataset took 18 minutes, while tracking and track processing took an additional 88 minutes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Tracking single cells in an interstitial P. aeruginosa colony spreading via pili-based motility. (A) We used FAST to track cells within a monolayer of P. aeruginosa undergoing collective motility using position, length and width as features. Images were collected using high speed ‘camera streaming’ with a mean frame rate of 127 fps, but variations in the elapsed time between subsequent frames, ∆t, resulted in transient reductions in trackability. (A, inset) Analysis of the timestamps associated with each frame revealed that the trackability score was negatively correlated with ∆t (r indicates Pearson’s correlation coefficient; the warmer colours denote a higher density of data points). We therefore restricted our subsequent analyses to a subset of the data in which the trackability score was relatively constant (green region) using FAST’s built-in tools. (B) We used the Overlays module to colour code cells based on their instantaneous speed. Although cells typically moved relatively slowly, very occasionally cells were observed to undergo a very rapid burst of movement (see cream coloured cell). (C) These rapid jumps can also be observed in traces of the speed of individual cells. Here we plot the instantaneous speed of three different cells over time, each in a different colour. The montages above illustrate three of these transient events, using the same colour coding shown in B. (D) To investigate these movements at the population level, we calculated instantaneous cell velocities in both the x and y direction for all cells and plotted their combined distribution. In other active systems, this distribution is approximately Gaussian. However, in our system the highly transient bursts of velocity result in heavy tails, causing them deviate from a Gaussian distribution with the same variance (dashed black line). https://doi.org/10.1371/journal.pcbi.1011524.g005 In this example, a relatively large timestep ∆t allows cells to move further between frames, which reduces predictability and therefore reduces the trackability score. More generally, however, the trackability depends on a combination of both experimental and imaging conditions, providing a robust metric to interpret datasets. For example, the trackability score can be used to quickly identify a wide range of problems which might arise during an experiment, including fluctuations in focus, illumination intensity, or inadvertent movement of the sample. Once problematic timepoints have been identified, the user can decide either to avoid them by using a subset of the images (as in this example) or to reject the entire dataset. Tracking the smaller subset of images that we specified resulted in over 1,600 separate trajectories, each at least 200 timepoints long. To visualise this large dataset, we used the Overlays module to colour individual cells in the phase-contrast image based on their instantaneous speed (Fig 5B, S2 Movie). This revealed that while most cells move at relatively slow speeds, a small number of cells undergo rapid, sporadic bursts of movement approximately 25 times faster than the average. These rapid movements are also clearly visible in measurements of the speed of individual cells over time (Fig 5C). Our results are similar to the ‘slingshots’ previously observed in solitary P. aeruginosa cells moving at the glass/liquid interface, apparently driven by pilus detachment events [39]. However, the peak speeds that we record (up to 60 μm s-1) in collectives of P. aeruginosa are approximately 20 times larger than those previously observed. While we do not know the specific reason for this difference, interactions with neighbouring cells facilitated by the high cell density, the glass/agar interstitial colony environment and our higher framerate (~13 times that of [39]) may each play a role. To illustrate how this large tracking dataset can be mined to elucidate the statistics of rare events, we constructed the instantaneous marginal velocity distribution from our tracks (i.e. the x- and y-components of the instantaneous velocity vectors) (Fig 5D). While previous studies have found that sperm and swimming bacterial cells undergoing collective movement generate marginal velocity distributions that are approximately Gaussian [35,40], we observed that bacteria collectively moving via pili-based motility exhibit much heavier tails corresponding to their occasional rapid motions. Despite the rarity of these events—less than 0.05% of our measurements had a magnitude larger than 20 μm s-1—the exceptional number of cell trajectories we obtained nevertheless allowed us to finely resolve their statistical distribution. These analyses demonstrate how FAST can be used to rapidly characterise the motility of a large number of cells in densely-packed conditions, with relatively little user input and computational effort. Automated analysis of T6SS battles. To illustrate FAST’s capabilities beyond standard cell tracking, we analysed the activity of the Type 6 Secretion System (T6SS) using FAST. The T6SS is a highly dynamic organelle composed of a molecular ‘spear’ tipped with a toxin known as an effector [41]. Bacteria that possess the T6SS can inject this effector into neighbouring cells, which kills them [42]. Firing events can be monitored by visualising the localisation of the protein that forms the contractile sheath of the T6SS needle, TssB in P. aeruginosa and VipA in Vibrio cholerae [12,43]. Here, we highlight how FAST can be used to quantify how the T6SS is regulated in co-cultures of V. cholerae and P. aeruginosa. Because of its short range, the T6SS is effective only at high cell density, which historically has made it difficult to study using automated analyses. We used FAST to analyse imaging datasets that show V. cholerae and P. aeruginosa cells expressing VipA-mCherry and TssB-mNeongreen, respectively, interacting through their respective T6SSs when mixed together in a densely-packed monolayer [12] (Fig 6A, left). We distinguished the two species from one another (Fig 6A, right) using FAST’s post-processing toolbox, which compares each cell’s intensity in the mCherry and mNeonGreen channels to automatically assign them to two distinct populations (Fig 6B). This utility allows one to rapidly compare the behaviour of different genotypes within the same sample. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Identifying different bacterial species, quantifying cell morphologies, and investigating T6SS dynamics in a densely packed collective. (A) A monolayer composed of a mixture of P. aeruginosa cells expressing TssB-mNeongreen and V. cholerae cells expressing VipA-mCherry (left). Measurements of the average intensity of each cell in the two different fluorescence channels allowed us to automatically distinguish the two species (B), an approach known as image cytometry. By colour coding each cell (P. aeruginosa, cyan, V. cholerae, red), we were able to visually confirm the accuracy of the assignment process (A, right). (C) We then quantified a novel feature, cell curvature, using FAST’s custom feature extraction framework to calculate the curvature of the segmentation backbones (inset). Splitting these measurements into the two previously assigned populations showed the comma shaped V. cholerae cells were substantially more curved than the rod-shaped P. aeruginosa cells. (D-F) We then used FAST’s plotting options to illustrate the dynamics of the T6SS. A P. aeruginosa cell fires its T6SS machinery (observed as the bright puncta of TssB), which can be visualised using the kymograph (D) and ‘cartouche’ (F) plotting options, and which also leads to an increase in the standard deviation of the mNeongreen channel (E). (G) We used this latter measurement to detect when the T6SS was active in a cell, allowing us to measure the proportion of time that each cell in the population spends in the firing state. Our analyses confirmed that when co-cultured with a strain of V. cholerae with an inactive T6SS (Δhcp1 Δhcp2), the T6SS activity of WT P. aeruginosa cells is dramatically reduced compared to when they are co-cultured with WT V. cholerae. Here the circles show averages from separate movies and horizontal lines show the mean for all samples of a given combination of genotypes. https://doi.org/10.1371/journal.pcbi.1011524.g006 The architecture of FAST allows users to easily define and quantify novel features that are not already included in FAST. To do this, users can write a short feature measurement script that can make use of the segmentation of each object in each frame, as well as the corresponding regions in each of the imaging channels—in this case, the mNeongreen, mCherry and phase-contrast signals. The Feature Extraction module then automatically stores these custom features in the same way as the built-in features, allowing them to be integrated into downstream analyses. To illustrate this capability, we extracted cell curvature as a custom feature to see if we could detect the differences in morphology of V. cholerae cells (which are comma shaped) from that of P. aeruginosa (which are rod shaped). Using a skeletonization-based approach, we extracted the morphological backbone of each cell from its segmentation and measured its curvature by finding the best-fit circle to this set of points. Consistent with expectations, our automated analysis found that the cells identified as V. cholerae were substantially more curved than those identified as P. aeruginosa (Fig 6C). Our software is also capable of generating sophisticated visualisations and analyses of the behaviour of single cells. For example, FAST allows the dynamics of T6SS firing in individual cells to be easily visualised using the kymograph option of the Plotting module (Fig 6D), as well as the ‘cartouche’ option, which extracts and aligns cropped images of the specified cells (Fig 6F). We can also quantify the dynamics of the T6SS using the feature data associated with a given cell: the standard deviation of the mNeongreen channel substantially increases during firing (Fig 6E), as the TssB becomes non-uniformly distributed through the cell as it assembles into the sheath. It is generally accepted that P. aeruginosa only fires its T6SS when triggered by the firing of the T6SS of a neighbouring cell [9,12]. Although qualitative evidence for this is strong, to our knowledge this effect has never been directly quantified, likely because of the difficulties involved with tracking densely packed cells and because the T6SS firing events are themselves relatively rare. We therefore decided to use FAST to simultaneously track a large number of densely packed cells and automatically detect when each fires its T6SS. We used doubleFAST to automate the analysis of multiple datasets from experiments in which WT P. aeruginosa cells were either co-cultured with WT V. cholerae or co-cultured with a mutant V. cholerae strain that is incapable of firing their T6SS (Δhcp1 Δhcp2). Using the standard deviation of the mNeongreen channel to distinguish when P. aeruginosa is actively firing its T6SS, we calculated the proportion of time that P. aeruginosa cells spend firing their T6SS (Fig 6G, S3 Movie). As expected, when co-cultured with the inactivated Δhcp1 Δhcp2 V. cholerae strain, P. aeruginosa dramatically reduced the proportion of time it spent firing its T6SS compared to when co-cultured with WT V. cholerae. These analyses generated 1,246 separate trajectories, of which around half (601) spanned the full 151 timepoints of the experiments. In conclusion, this case study demonstrates how FAST can automatically distinguish different species, how users can specify novel features like cell curvature, and how complex bacterial behaviours can be quantified using feature data. Discussion Advances in experimental techniques and automated microscopy have transformed live cell imaging from a technique that relies largely on qualitative observation into a highly quantitative discipline that leverages huge amounts of data. Integral to this renaissance are computational tools that can automatically parse and annotate the large imaging datasets resulting from these experiments. Many tools have been developed for this purpose [17,24–28,44–47], each optimised for a specific research question. As we have emphasized throughout this manuscript, our own contribution is particularly well-suited for the analysis of experiments where cells are tightly packed together. We designed FAST to allow users to rapidly obtain high-quality results by minimising both the number of user-defined parameters in the Tracking module and the computational time required to process an imaging dataset containing many cells. Our tracking algorithm automatically determines how to combine data from different cell features, which prevents users from having to iteratively improve tracking results by sequentially adjusting a large number of parameters—typically a very slow and laborious process. In the case of the microcolony analyses of Fig 4, selecting appropriate values of the user-defined tracking parameters F and P took only a few minutes using FAST’s built-in tools. Subsequent benchmarking showed that the specific set of F and P parameters chosen for the first microcolony were nearly optimal for all eight datasets (S4 Fig). This robustness allows users to queue the autonomous analysis of many datasets using doubleFAST. Our approach also provides the user with a single, easy to interpret metric that allows them to rapidly identify and avoid sections of datasets that are predicted to yield low-fidelity trajectories. While the basic FAST package already outputs most of the cell features that are widely used by researchers, simple modifications to the system allow users to integrate new features into the existing analytical framework. We also note some limitations to our approach. Firstly, large-scale shifts in field of view, for example caused by thermal drift or the imprecise movement of a motorised stage, can adversely impact the tracking accuracy of FAST. While the timepoints at which these shifts occur can be pinpointed by the associated sharp reductions in trackability (and potentially excluded from subsequent analyses, similar to the process shown in Fig 5A), it is usually better to stabilise troublesome datasets with tools such as TurboReg [48] before processing them with FAST. We also note that the reliance of our approach on summary statistics such as Σt(Δx) (the covariance matrix of feature displacements) means that it can suffer from stability issues when the number of cells in the field of view is very low (≲10), as there is insufficient training data to accurately estimate these statistics. However, should this pose an issue—for example at the beginning of microcolony datasets when there are only small numbers of cells—the relatively small amounts of data involved generally make it feasible to correct these errors using the manual track correction GUI. Tracking systems typically break down in one of two ways—either individual objects cannot be distinguished, or the tracking algorithm cannot accurately link individuals between frames. Machine learning is increasingly used to solve the first of these two challenges, and a number of software packages which use deep learning methods to segment microbes at high density are now widely available [36,46,47,49,50]. However, the application of such techniques to the tracking problem has gained traction only in the last few years [36,51–53]. The approach described here is based on a statistical framework that optimises track quality while simultaneously providing a metric that predicts trajectory reliability. Our software has already proven its ability to elucidate the rich and complex behaviours that individual bacteria exhibit in densely-packed conditions [1,10,22,23]. Future work stemming from these combined experimental and analytical approaches could ultimately shed new light on how dense bacterial communities function and help us to develop novel strategies to manipulate them. Online methods Computational resources All analyses presented in this manuscript were performed with a Microsoft Surface Book 2, with an 8-core Intel i7-8650U CPU and 8 Gb of RAM. FAST was run directly in Matlab, version 2018b. SPR model The 2D SPR model used in this manuscript has been described in detail elsewhere [1,35]. In brief, we model cells as stiff rods composed of evenly-spaced Yukawa segments, mutually repulsive point potentials. We initialise the system by filling a square domain with Nr rods, evenly spaced on a lattice. Each rod i is associated with an aspect ratio ai and a fluorescence intensity Ii, randomly drawn from distributions fitted to P. aeruginosa data from an experiment visualising the T6SS [9] as shown in S1 Fig. The packing fraction of the system, ρ, is calculated as where A is the area of the simulated domain, which has doubly periodic boundary conditions. Taking the instantaneous position of a rod i as ri, its orientation as ϕi, the unit vector denoting this orientation as and the sum of the potentials between i and all other rods as Ui, we define the equations of motion for each rod as: (5A) (5B) where fT is the translational friction tensor, fθ is the rotational friction constant and ν is the size of a self-propulsion force exerted by each rod along its axis. We use the formulation presented in [35] to calculate fT and fϕ, which are in turn functions of the rod aspect ratio and the Stokesian friction coefficient, f0. We simulate the dynamics of our system by numerically integrating Eqs 5A and 5B using the midpoint method. Following an initial transient, the system reaches a statistical steady-state. At this time, we begin to measure the positions, orientations, lengths and intensities of each rod at a sampling “framerate” ΔT. We add simulated measurement noise to each of these measurements, which is drawn from a Gaussian distribution with a mean of zero. The standard deviation of the measurement noise for each feature is controlled by the parameters σr (positional noise), σϕ (orientational noise), σa (length noise) and σI (fluorescence noise). To constrain the baseline estimates of these noise parameters, we measured how each of the corresponding features fluctuated about the mean in the T6SS firing dataset [9]. The cells in this dataset are non-motile and the framerate is high enough that growth is negligible, meaning any apparent changes in position, orientation, length or fluorescence are wholly attributable to measurement noise. We varied the properties of our simulations by adjusting the values of the parameters N, v, ΔT, σr, σa and σI in different simulation runs. The values of these parameters, as well as the fixed system parameters, are provided in Table 1. Sample preparation The monolayer of P. aeruginosa presented in Fig 5 was prepared using the wild-type PAO1 strain [54]. Cells were streaked out from freezer stocks onto LB agar plates (Lennox, 20 g/l, Fisher Scientific, solidified with 1.5% (w/v) agar, Difco brand, BD) and incubated overnight at 37°C. Single colonies were picked from the resulting plates and incubated overnight in liquid culture under continuous shaking, resulting in stationary phase cultures. These were then diluted 30-fold and returned to the shaking incubator for a further two hours, yielding cultures in exponential phase. The final culture used for inoculation was prepared by adjusting the optical density at 600 nm (OD600) of the exponential phase cultures to 0.05 using fresh LB, corresponding to an approximate concentration of 12,500 cells μl-1. We prepared monolayers from these cultures using a similar protocol to that described in [37]. 1 μl of inoculation culture was spotted onto the centre of a small (2 cm x 2 cm) LB agar pad. To provide optimal conditions for observing twitching motility, the concentration of agar in these pads was 0.8%. This pad was then inverted and placed into the base of a coverslip-bottomed Petri dish (175 μm coverslip thickness, MatTek), which was then closed and incubated for 16 hr at room temperature. The ability to close the lid of the Petri dish allowed us to avoid desiccation of the sample during incubation. By the end of the incubation period, a large interstitial colony with a dense monolayer at its perimeter had formed [1]. Microscopy The high framerate movie used for the analysis of rapid motion (Fig 5) was acquired using a Zeiss Axio Observer.Z1 microscope outfitted with an Axiocam 702 camera set to ‘camera streaming’ mode and a Plan Apochromat 63x oil-immersion objective. Other datasets The datasets of dividing E. coli cells (Fig 4) were downloaded from the raw data repository associated with reference [11], https://zenodo.org/record/268921. They were then stabilised using a customised version of the TurboReg plugin [48]. The T6SS firing dataset (Fig 6) was downloaded from the Supplementary Information section of reference [12] under a Creative Commons Attribution license. FIJI’s [55] built-in AVI reader was used to extract individual frames of this movie. Computational resources All analyses presented in this manuscript were performed with a Microsoft Surface Book 2, with an 8-core Intel i7-8650U CPU and 8 Gb of RAM. FAST was run directly in Matlab, version 2018b. SPR model The 2D SPR model used in this manuscript has been described in detail elsewhere [1,35]. In brief, we model cells as stiff rods composed of evenly-spaced Yukawa segments, mutually repulsive point potentials. We initialise the system by filling a square domain with Nr rods, evenly spaced on a lattice. Each rod i is associated with an aspect ratio ai and a fluorescence intensity Ii, randomly drawn from distributions fitted to P. aeruginosa data from an experiment visualising the T6SS [9] as shown in S1 Fig. The packing fraction of the system, ρ, is calculated as where A is the area of the simulated domain, which has doubly periodic boundary conditions. Taking the instantaneous position of a rod i as ri, its orientation as ϕi, the unit vector denoting this orientation as and the sum of the potentials between i and all other rods as Ui, we define the equations of motion for each rod as: (5A) (5B) where fT is the translational friction tensor, fθ is the rotational friction constant and ν is the size of a self-propulsion force exerted by each rod along its axis. We use the formulation presented in [35] to calculate fT and fϕ, which are in turn functions of the rod aspect ratio and the Stokesian friction coefficient, f0. We simulate the dynamics of our system by numerically integrating Eqs 5A and 5B using the midpoint method. Following an initial transient, the system reaches a statistical steady-state. At this time, we begin to measure the positions, orientations, lengths and intensities of each rod at a sampling “framerate” ΔT. We add simulated measurement noise to each of these measurements, which is drawn from a Gaussian distribution with a mean of zero. The standard deviation of the measurement noise for each feature is controlled by the parameters σr (positional noise), σϕ (orientational noise), σa (length noise) and σI (fluorescence noise). To constrain the baseline estimates of these noise parameters, we measured how each of the corresponding features fluctuated about the mean in the T6SS firing dataset [9]. The cells in this dataset are non-motile and the framerate is high enough that growth is negligible, meaning any apparent changes in position, orientation, length or fluorescence are wholly attributable to measurement noise. We varied the properties of our simulations by adjusting the values of the parameters N, v, ΔT, σr, σa and σI in different simulation runs. The values of these parameters, as well as the fixed system parameters, are provided in Table 1. Sample preparation The monolayer of P. aeruginosa presented in Fig 5 was prepared using the wild-type PAO1 strain [54]. Cells were streaked out from freezer stocks onto LB agar plates (Lennox, 20 g/l, Fisher Scientific, solidified with 1.5% (w/v) agar, Difco brand, BD) and incubated overnight at 37°C. Single colonies were picked from the resulting plates and incubated overnight in liquid culture under continuous shaking, resulting in stationary phase cultures. These were then diluted 30-fold and returned to the shaking incubator for a further two hours, yielding cultures in exponential phase. The final culture used for inoculation was prepared by adjusting the optical density at 600 nm (OD600) of the exponential phase cultures to 0.05 using fresh LB, corresponding to an approximate concentration of 12,500 cells μl-1. We prepared monolayers from these cultures using a similar protocol to that described in [37]. 1 μl of inoculation culture was spotted onto the centre of a small (2 cm x 2 cm) LB agar pad. To provide optimal conditions for observing twitching motility, the concentration of agar in these pads was 0.8%. This pad was then inverted and placed into the base of a coverslip-bottomed Petri dish (175 μm coverslip thickness, MatTek), which was then closed and incubated for 16 hr at room temperature. The ability to close the lid of the Petri dish allowed us to avoid desiccation of the sample during incubation. By the end of the incubation period, a large interstitial colony with a dense monolayer at its perimeter had formed [1]. Microscopy The high framerate movie used for the analysis of rapid motion (Fig 5) was acquired using a Zeiss Axio Observer.Z1 microscope outfitted with an Axiocam 702 camera set to ‘camera streaming’ mode and a Plan Apochromat 63x oil-immersion objective. Other datasets The datasets of dividing E. coli cells (Fig 4) were downloaded from the raw data repository associated with reference [11], https://zenodo.org/record/268921. They were then stabilised using a customised version of the TurboReg plugin [48]. The T6SS firing dataset (Fig 6) was downloaded from the Supplementary Information section of reference [12] under a Creative Commons Attribution license. FIJI’s [55] built-in AVI reader was used to extract individual frames of this movie. Supporting information S1 Fig. Distributions used to specify initial conditions of the SPR model. The distributions of trajectory-averaged aspect ratios (A) and GFP intensities (B) of a dataset of non-motile P. aeruginosa cells [9]. A gamma distribution (shape parameter = 15.3, scale parameter = 0.248) and a normal distribution (mean = 63.1, standard deviation = 8.83), respectively, were fitted to these two datasets (black dotted lines). To initialise the SPR model, rod aspect ratio, ai, and simulated fluorescence intensity, Ii, were randomly drawn from these two fitted distributions, allowing us to ensure that these two features were modelled realistically in our simulations. https://doi.org/10.1371/journal.pcbi.1011524.s001 (TIFF) S2 Fig. Including additional feature information improves trackability and increases tracking fidelity. We measured the trackability (A, B) and maximum F1-scores (C, D) of synthetic high-density motility data generated using a range of different parameter combinations. Both ‘simulation parameters’ (parameters that change the properties of the SPR model we used to generate the synthetic dataset, A, C) and ‘measurement parameters’ (parameters that change the amount of measurement noise for each of the different features, B, D) were varied. See Table 1 for further details. Here we compare trackability and tracking fidelity when the tracking algorithm can only use positional information (‘Centroids’, brown) to when all feature information is available (‘All features’, purple). Arrows show the consistent increase in these two metrics when all features from the synthetic dataset are used, illustrating the robustness of our approach. https://doi.org/10.1371/journal.pcbi.1011524.s002 (TIFF) S3 Fig. The trackability metric is a more reliable predictor of tracking performance compared to the number of cells within a microcolony. We used manually corrected ground-truth datasets to calculate the performance of FAST’s tracking algorithm at each time point of the microcolony datasets shown in Fig 4, using either the full suite of features (purple) or only the cell positions (brown). For all time points that contained 32 or more cells, we then compared the resulting F1-scores to (A) the instantaneous trackability and (B) the number of cells in the microcolony. For all four regressions, the correlation between the predictor and the F1-score was significant (p < 0.05, linear regression t-test), however the strength of the correlation was much higher for trackability than cell number, as indicated by the corresponding Pearson correlation coefficients (r). https://doi.org/10.1371/journal.pcbi.1011524.s003 (TIFF) S4 Fig. Cell tracking is robust to changes in the two user-defined parameters and produces consistent results across experimental datasets. We tracked cells in each of the eight E. coli microcolonies forty-nine different times, each time with a different set of the two user-defined parameters—F, the proportion of links included in the training dataset and P, the tracking threshold. These two parameters allow the user to balance the trade-off between trajectory quality and trajectory quantity during the initial training stage and the primary tracking stage, respectively. We then compared the results of these automated analyses with a manually curated ground-truth to calculate the F1–score, which measures overall tracking fidelity. These analyses were repeating using both the full set of features (length, fluorescent intensity, width and position, above) and only the cell positions (below). In both cases, we found a wide basin of [F, P] parameter values that produced a similar level of tracking performance. Moreover, the combination of [F, P] values that produced the best results was similar between the different microcolonies, suggesting that a set of [F, P] optimised for one set of experimental images can be applied to subsequent datasets without adversely affecting tracking performance (note the logarithmic axes). The values of F and P used in the analyses of Fig 4 are shown with black squares. https://doi.org/10.1371/journal.pcbi.1011524.s004 (TIFF) S1 Movie. Tracking cells, divisions, and lineages in a growing E. coli microcolony. Growth of E. coli cells in a microcolony was imaged using phase contrast and fluorescence microscopy [11] (left). The left-hand panel shows a combination of the original phase contrast and fluorescence images, the middle panel shows the individual cell trajectories (where colours denote the instantaneous cell length), and the right-hand panel shows the “lineage tree”. In the lineage tree, the thin lines denote the trajectories of individual cells and thick lines represent division events—both are colour-coded to denote the number of generations elapsed from the original founder cell. This movie shows the same ‘automated’ analysis presented in Fig 4 A and B, in which nothing has been manually corrected. All graphics were generated by FAST’s built in Overlays module. The image datasets were obtained from a previous study [11]. Shown here is Dataset ID 140408_01_cib (see Table 2). Elapsed time: 7 hours 5 minutes. https://doi.org/10.1371/journal.pcbi.1011524.s005 (AVI) S2 Movie. Quantifying rapid, slingshot-like movements of single P. aeruginosa cells in a densely-packed monolayer. Cell segmentations are coloured-coded (using the Overlays module) by their instantaneous speed, ranging from stationary (black) to the maximum detected cell speed (light orange). Circles show time points interpolated by the tracking algorithm when cells were temporarily mis-segmented. Here we only show the trajectories longer than 200 frames that were included in our final analyses. While FAST outputs many shorter trajectories, these were generally terminated due to multiple consecutive cell mis-segmentations. Thus, to be conservative we only used very long trajectories to quantify the rare slingshotting events (Fig 5D). This movie shows a 230 frame subset of the entire 750 frame long dataset. Elapsed time: 1.809 seconds. https://doi.org/10.1371/journal.pcbi.1011524.s006 (AVI) S3 Movie. Automated detection of P. aeruginosa T6SS firing events. We automatically detect firing events of the Type 6 Secretion System (T6SS) using a function from the FAST toolbox that autonomously identifies ephemeral increases in the standard deviation of the GFP pixel intensity within a cell’s segmentation (Fig 6E), which result from the assembly and contraction of the fluorescently labelled sheath that surrounds the T6SS needle. The firing events are marked by red circles (using the Overlays module), which are centred on the cell centroid and scaled by the cell length, over greyscale images showing GFP intensity. These imaging datasets were obtained from a previous study [12]. Elapsed time: 3 minutes. https://doi.org/10.1371/journal.pcbi.1011524.s007 (AVI) S1 Text. Design goals and module specifications. Includes Fig A-E. https://doi.org/10.1371/journal.pcbi.1011524.s008 (PDF) Acknowledgments We would like to thank the numerous people who have tested and provided feedback on FAST during its development, including N.A. Costin, S.C. Booth, J.H. Wheeler, E.T. Granato, R.K. Kumar, W.P.J. Smith, C.J. Walther and H. Todorov. We would also like to thank S. van Vliet for advice regarding the E. coli microcolony image datasets.

Guan, Kaiyun;Curtis, Erin R.;Lew, Daniel J.;Elston, Timothy C.

doi: 10.1371/journal.pcbi.1011523pmid: 37782676

Introduction Cell polarity is the asymmetric distribution of cellular components along some axis. Many cells dynamically adapt the direction of polarization in response to environmental cues. Migratory cells relocate the cell “front” when tracking extracellular signals [1–3]. Pollen tubes, plant root apices, fungal hyphae, neuronal axons, and yeast cells change the direction of polarized growth in response to physical or chemical cues [4–8]. The mechanisms underlying polarity site relocation are not yet understood. The molecular machinery that controls cell polarity centers on the Rho-family GTPase Cdc42, which is highly conserved among eukaryotes [9,10]. Cdc42 acts as a molecular switch. When bound with GDP, Cdc42 is in its off state and a majority is held in the cytosol through interactions with a GDP dissociation inhibitors (GDIs). Inactive Cdc42 can transition to the membrane and dissociate from the GDI. Membrane-associated Cdc42 is activated by a guanine nucleotide exchange factor (GEF) that promotes the release of GDP to allow binding of GTP. Active GTP-Cdc42 binds various “effector” proteins that regulate the cytoskeleton. The polarity circuit of the budding yeast Saccharomyces cerevisiae has been extensively characterized. Yeast cells polarize their growth during budding and mating. Polarity is established and maintained by autocatalytic positive feedback [11,12]. A cytoplasmic protein complex containing an effector and a GEF associates with active Cdc42 at the membrane and promotes activation of neighboring Cdc42 molecules [11,13]. Because diffusion is slow in the membrane as compared to the cytosol, a cluster of Cdc42 molecules in the membrane is slow to disperse but can recruit cytoplasmic polarity factors from a wide catchment area, sustaining a polarity site. Mathematical models capturing these features recapitulate the patterns of Cdc42 localization observed in vegetative yeast cells: unpolarized cells develop clusters that rapidly coarsen to form a single polarity site that is then stably maintained [12–17]. However, during mating, yeast cells relocate their polarity sites in an apparent search for mating partners [18,19]. This observation raises the question of how the yeast polarity circuit might be modified to allow relocation. During mating, yeast cells of each mating type express G-protein-coupled receptors (GPCRs) that detect extracellular peptide pheromones released by the opposite mating type [20]. Pheromone binding to the receptor stimulates the Gα subunit to exchange GDP for GTP, which dissociates Gα from Gβγ at the membrane. Gα and Gβγ transmit signals to prepare cells for mating, including activation of pathways that regulate cell polarity. The pathway that allows cell polarity to be influenced by external pheromone gradients is mediated by the scaffold protein Far1, which binds to both Gβγ and the Cdc42 GEF [21–23]. This leads to activation of Cdc42 where there is Gβγ, which reflects the location of active GPCR [22,23]. Thus, an external gradient in pheromone concentration can be translated into an internal gradient in active Cdc42 concentration. Vesicles carrying mating-specific proteins and lipids are delivered to the cluster of Cdc42 via formin-nucleated actin cables, enabling a cell to grow toward a mating partner at the polarity site [13]. However, yeast cells mate in crowded conditions in which global pheromone gradients could be uninformative [24,25], and a shallow gradient may be difficult to interpret given the perturbing effects of molecular noise [26]. Indeed, Cdc42 clusters in mating yeast cells often form at a location that is not directed towards the partner [18,27,28]. Relocation of the polarity site is then needed to successfully mate. During an initial “indecisive phase”, multiple Cdc42 clusters in mating yeast spontaneously appear, disappear, and relocate erratically [18]. This behavior is thought to enable the search for a mating partner, and after the indecisive phase the cell develop a single stable polarity site oriented towards the chosen partner [29]. What leads to the indecisive behavior of Cdc42 clusters in mating cells? One proposal is that erratic relocation is a product of molecular-level noise acting on the yeast polarity circuit [27,30,31]. Molecular-level noise is not captured by deterministic models such as the ones solely based on reaction-diffusion equations (RDEs), which ignore the molecular nature of biochemical systems and therefore miss noise-driven phenomena. In contrast to deterministic models, stochastic models attempt to capture stochastic effects that arise from the random nature of biochemical reactions. Molecular-level noise is most accurately captured by particle-based models [17,32,33]. With particle-based simulations, we confirm that polarity site relocation in a realistic yeast polarity circuit can arise from stochastic noise, but only in a very narrow parameter regime. we find that incorporation of the pheromone-responsive Far1 pathway can dramatically expand the parameter space in which models display polarity site relocation. This same pathway can also explain why polarity eventually becomes stably oriented towards a partner. Results Noise-driven relocation of the polarity site To systematically investigate whether the core yeast polarity circuit can yield indecisive Cdc42 clustering like that seen in mating cells, we used a previously validated model developed to describe Cdc42 behavior in vegetative yeast cells [12,17,32,34] (Fig 1A and Table 1). The model species are inactive Cdc42 (which can exchange between membrane and cytosolic compartments), active Cdc42 (which is restricted to the membrane compartment), and a heterodimer of an effector and an activator of Cdc42 (here called Bem1-GEF). This complex also exchanges between membrane and cytosol, and can bind to active Cdc42. At the membrane, the complex promotes activation of inactive Cdc42. The main structural feature of the model is the positive feedback loop formed by mutual activation of Cdc42 and Bem1-GEF (Fig 1A, inset). In this feedback loop, active Cdc42 recruits Bem1-GEF from the cytoplasm and increases its GEF activity. In turn, Bem1-GEF activates Cdc42, leading to further Cdc42 recruitment from the cytosolic pool. This positive feedback loop amplifies local fluctuations in Cdc42 activity eventually leading to the formation of a polarity site [12,35]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Behavior of the core polarity circuit. A) Reaction scheme for the core polarity circuit of yeast (diagram adapted from Pablo et al. [17]). Polarity is driven by positive feedback through mutual activation (Inset). B) Two-parameter bifurcation diagram for molecular abundances of Cdc42 and Bem1-GEF (Unpolarized regime–green, transient polarity–blue, polarized–red). C) Simulation results for the transient polarity regime. Time series for Ripley’s K-function values (a measure of Cdc42 clustering) were generated for ten simulations (left panel). Distribution of K values (right panel). D) Distributions of Cdc42-GTP molecules at indicated times. Distributions are taken from the black time series in C using time points indicated with ticks on the time axis. Simulations correspond to 3000 Cdc42 and 102 Bem1-GEF molecules using uniform random distributions for Cdc42 and Bem1-GEF as initial conditions. E) Time series for number of clusters. Simulation results taken from the black time series in C (left panel). Example of cluster count from experiments (right panel). https://doi.org/10.1371/journal.pcbi.1011523.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Parameters for the core polarity circuit. https://doi.org/10.1371/journal.pcbi.1011523.t001 Our first goal was to determine if molecular-level fluctuations arising from the stochastic nature of biochemical reactions and diffusion are sufficient to promote relocation of the polarity site. To test this possibility we used the simulation platform Smoldyn [33,36] to perform particle-based simulations of the core polarity circuit, with parameters described previously [17]. We performed simulations with varying numbers of Cdc42 and Bem1-GEF molecules. These initial simulations revealed three distinct types of behavior: 1) unpolarized, in which no significant Cdc42 clusters formed (S3A Fig), 2) polarized, in which a single strong cluster was maintained (S3B Fig), and 3) transient polarity, in which clusters could spontaneously form, disappear, and re-form (Fig 1C and 1D and S1 Movie). These results suggest that molecular-level noise is able to trigger intermittent transitions between polarized and unpolarized steady states, consistent with previous work on regulatory systems [37–39]. To quantify our results, we used a normalized version of Ripley’s K-function (K) [40,41]. K measures clustering strength as the deviation of the observed particle distribution from that of a uniform distribution [17]. A value of K close to zero signifies a uniform distribution, and the larger the value of K the more polarized the distribution. We designated a threshold value of 1.5 to distinguish unpolarized (K < 1.5) from polarized (K ≥ 1.5) states for all 2D simulations (S1A Fig). In the transient polarity regime, the K values exhibit a bimodal distribution, consistent with a bistable system that switches between polarized and unpolarized states (Fig 1C, right panel) [16,37–39]. To investigate how robustly the core polarity circuit can give rise to transient polarity, we developed a method for distinguishing the three types of behavior based on the distribution of K values (Methods and S2 Fig). Specifically, we defined the transient polarity regime according to the bimodal distribution of K values (S2 Fig). Our analysis revealed that the parameter regime supporting transient polarity is limited to a small range of molecular abundances (Fig 1B), as also found in earlier work [32]. Thus, it would require very precise control for cells to exploit this parameter regime to develop transitory polarity. Moreover, the transient polarity regime exhibits transitions between long-lived unpolarized and polarized states with a single cluster, rather than multiple rapidly fluctuating clusters observed in indecisive yeast cells (Fig 1E). We conclude that the indecisive polarity behavior of mating yeast cells is unlikely to be a simple consequence of molecular noise acting on the core polarity circuit. The receptor-Far1 pathway During mating, there is an additional mechanism for bringing GEF molecules to the membrane. Specifically, following exposure to pheromone, GEF molecules in complex with Far1 (Far1-GEF) are recruited to the membrane through Far1’s interaction with Gβγ released by active receptors. Far1’s interaction with Gβγ couples the polarity circuit to the extracellular pheromone concentration. In addition, the receptor-Far1 circuit forms a second positive feedback loop, because active Cdc42 leads to the trafficking of new receptors to the membrane (Fig 2A). To gain insight into the behavior of the receptor-Far1 circuit, we considered a model consisting of Cdc42, receptors and Far1-GEF, neglecting Bem1-GEF. To simplify the model, the G proteins that mediate interaction between receptors and Far1-GEF are not modeled explicitly. That is, we assume that active receptors recruit Far1-GEF from the cytoplasm. Once at the membrane Far1-GEF promotes Cdc42 activation with the same efficacy as Bem1-GEF in complex with active Cdc42 (Fig 2A and Table 2). We also assume all pheromone receptors are active (this is analogous to the situation in which cells are exposed to saturating pheromone concentration). Pheromone receptors are delivered to the cell surface via Cdc42-oriented actin cables. The model does not explicitly take actin cables or vesicle delivery into account. Rather, we assume the rate at which receptors are delivered is determined by the local concentration of active Cdc42 (see Methods for details). Surface receptors are internalized via endocytosis, at well-characterized rates [42,43]. We further assume that receptors diffuse very slowly at the membrane, consistent with experimental findings [18]. As noted above, because Cdc42 clusters enhance local accumulation of receptors, and receptors recruit Far1-GEF to activate Cdc42, this pathway forms a mutual activation positive feedback loop (Fig 2A, inset). Thus, we anticipated that the receptor-Far1 circuit, like the Bem1-GEF circuit (Fig 1A), would have the capacity to spontaneously polarize. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Behavior of the receptor-Far1 circuit. A) Reaction scheme for the receptor-Far1 circuit. This circuit forms a second positive feedback loop through mutual activation (inset). B) Distributions of active Cdc42 (top row) and receptors (bottom row) shown for 0 min and 0.3 min. Long receptor lifetimes prevent maintenance of a polarity site. Simulations performed using 3000 Cdc42, 280 Far1-GEF, and 2500 receptor molecules. Initial conditions consisted of a polarized cluster of 3000 Cdc42-GTP and uniformly distributed receptor and Far1-GEF molecules. C) Final K values as a function of receptor membrane residence time. Values of K taken at t = 4000 secs. Data points represent averaged values of K for 30 simulations and the shaded area denotes standard deviation. Snapshots are for active Cdc42 at t = 4000 secs. https://doi.org/10.1371/journal.pcbi.1011523.g002 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Parameters for the uniformly activated receptor-Far1 pathway. https://doi.org/10.1371/journal.pcbi.1011523.t002 When we simulated this circuit using realistic receptor trafficking rates and initial particle distributions drawn from a uniform distribution, the model did not spontaneously develop a polarized cluster of Cdc42 in biologically relevant timescales. Even when the initial conditions included a strong cluster of active Cdc42 but distributed receptors, the initial clustering was rapidly lost (Fig 2B). To understand why a polarized distribution of Cdc42 was not maintained, consider that receptor trafficking occurs at a much slower rate than the rates for Cdc42 and Far1-GEF association and dissociation: the average membrane residence time of receptors is about 8 minutes [42,43] compared to a few seconds for Cdc42 [46,47] and Far1-GEF [44]. Thus, any clusters of Cdc42 dissipate before they can be reinforced by receptor traffic. Consistent with that interpretation, when the receptor residence time was decreased by increasing receptor trafficking rates the receptor-Far1 circuit did spontaneously establish polarity (Fig 2C). Our simulations indicate that in spite of its positive feedback loop (Fig 2A), the receptor-Far1 circuit by itself cannot establish a polarity site due to the mismatch between the residence times for Cdc42 (short) and receptor (long). However, it is not clear how the Far1 pathway might influence polarity behavior if the Bem1-Cdc42 circuit was also present, as is the case during the indecisive phase. To understand how these pathways interact, we modeled a system that combined the polarity circuit (Bem1-GEF and Cdc42) with the receptor-Far1 pathway (receptors and Far1-GEF). The receptor-Far1 circuit promotes polarity site relocation and generates indecisive behavior The combined polarity circuit (Fig 3A) includes two interconnected positive feedback loops, one from the core polarity circuit and the other from the receptor-Far1 circuit (Fig 3A, inset). Importantly, these feedback loops operate on different time scales, with the faster polarity circuit supporting polarization, whereas the slower receptor-Far1 circuit is unable to generate a stable polarity site. Therefore, it is not obvious how the combined system will behave. To investigate the behavior of the combined polarity circuit we performed simulations starting from two different initial conditions: a uniform random distribution of pathway components and a pre-polarized Cdc42 cluster. Similar to the polarity circuit alone (Fig 1B), the combined system exhibited one of three behaviors (Figs 3B and S4) depending on the molecular abundance of the components: an unpolarized regime with little Cdc42 clustering (S5A Fig), a polarized regime with stable and static clusters (S5B Fig), and an intermediate regime where clusters can form or disappear (Fig 3C). However, the unpolarized regime was expanded and the transition to the polarized regime required a larger amount of the Bem1-GEF complex (compare Figs 3B and 1B). In between, there was a substantially larger transient polarity regime. Interestingly, in this regime the behavior of the system appeared to be qualitatively different from the model with only the core polarity circuit. Rather than a single well-formed polarity site appearing and dispersing (Fig 1D), the addition of Far1-GEF seemed to promote multiple transient Cdc42 clusters that dynamically relocate in a manner reminiscent of indecisive polarity clustering in yeast cells (Fig 3C and S2 Movie). To quantify this observation, we employed a cluster detection algorithm based on Voronoi tessellation [48] and tracked the number of detected clusters over time (S6A Fig). From these time series we computed the fraction of time each model spent in states with 0, 1 and 2+ clusters and transition probabilities between states (S6B Fig). We then compared our simulation results directly with a similar analysis of the data presented in Clark-Cotton et al. 2021 [19] (see Methods for details). Our analysis revealed that coexisting Cdc42 clusters in the combined model were more frequent and exhibited similar behavior to polarity sites of indecisive yeast cells (S6 Fig). We also computed distributions for the dwell times in states with 0 or 1 clusters for the two models (S6C Fig). These distributions demonstrate that transitions between cluster states occur more frequently in the combined model than in the core polarity circuit alone. (We are not able to compute similar distributions from the experimental data because in the experiments, data were taken every 2 minutes, which is insufficient to resolve all transitions.) Note that indecisive behavior is also distinct from the unpolarized state in that it exhibits higher values of K with larger fluctuations and multiple coexisting clusters, which are absent in the unpolarized state. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Combined circuit allows relocation and stabilization of the polarity site. A) Reaction scheme for the combined polarity circuit. The combined model consists of dual positive feedback architecture (inset). B) Two-parameter bifurcation diagram for molecular abundances of Cdc42 and Bem1-GEF (Unpolarized regime–green, transient polarity–blue, polarized–red). C)–E): Simulations results for the transient polarity regime. Plots for K consist of 10 realizations. The distributions for active Cdc42 come from the black time series using the time points indicated by tick marks on the time axis. All simulations were performed using 3000 Cdc42, 170 Bem1-GEF, 30 Far1-GEF and 2500 receptor molecules, with different initial conditions as indicated. C) Multiple transient polarity sites form using uniform random distributions as initial conditions. In a subset of the simulations, a single polarity site is eventually formed. D) An established polarity site rapidly dissipates following the addition of randomly distributed receptors and Far1-GEF molecules. E) A polarity site is rapidly established and stably maintained for simulations started with a cluster of active receptors and uniformly distributed Cdc42 molecules. https://doi.org/10.1371/journal.pcbi.1011523.g003 When we initiated the simulations with a pre-polarized Cdc42 cluster but uniform receptors, the cluster immediately dissipated (Fig 3D and S3 Movie). Thus, even though the receptor-Far1 circuit has a positive feedback architecture, the slow timescale of receptor trafficking allows distributed receptors to overcome Bem1-GEF-maintained polarity. Note that in many of the simulations the polarity site did eventually reform (Fig 3D). In these cases the receptor distribution also became polarized. We suggest the following mechanism to explain how indecisive behavior arises in the combined model. Far1-GEF molecules recruited to the membrane by active receptors can seed multiple small Cdc42 clusters, which are then amplified by Bem1-GEF-mediated positive feedback. Without Far1-GEF, clusters would compete to yield a single winning polarity site [16,17]. However, as this competition process begins, Far1-GEF nucleates new clusters, thwarting effective competition: although some Cdc42 cluster(s) may transiently become dominant, newly seeded clusters continue to compete such that no stable cluster can develop. For this mechanism to operate, there are several requirements that the system must satisfy. First, the distributed nature of active receptors is key to the continual seeding of new clusters: if receptors were instead clustered, then the combined Bem1-GEF and Far1-GEF positive feedbacks would act synergistically to maintain a Cdc42 cluster. This was indeed the case. When simulations were run with an initially focused receptor distribution, all simulations produced a well-polarized Cdc42 distribution (Fig 3E and S4 Movie). As initial receptor distributions were varied, more dispersed receptor distributions lowered the probability that the system would remain polarized (Fig 4A). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. The Far1 pathway promotes indecisive behavior by delaying or preventing coarsening of polarity sites. A) The fraction of time polarized as a function of the degree of dispersion in the initial receptor distribution. The fraction of time was measured as the ratio of K values ≥ 1.5 to the total number of K values in the interval 2000 to 4000 secs to avoid any initial transients. Each point represents a single simulation result from 30 total simulations. Initial receptor distributions were generated from 2D Gaussian distributions. The degree of dispersion represents the standard deviation of the distribution. B) Final K values (t = 4000 secs) as a function of receptor membrane residence time. 5 ≤ n ≤ 30 simulations for each data point. Standard deviations are shaded. C) Final K values as a function of Far1-GEF molecule number. 5 ≤ n ≤ 30 simulations for each data point. D) Time to stabilization of the polarity site (K ≥ 3) as a function of number of fixed Far1-GEF molecules in the membrane. Distributions of Far1-GEF molecules are shown along the x-axis. E) Time series of K values for the case of 15 fixed Far1-GEF molecules (left panel). Right panels show distributions for active Cdc42 molecules for the black time series at times indicated with ticks on the time axis. F) Results for a 3D particle-based simulation. Cdc42T molecules shown as blue dots on the cell surface. Rate constants converted from 2D case as described in the Methods and listed in Tables 1 and 2. All simulations performed using 3000 Cdc42, 170 Bem1-GEF, 2500 receptor and unless otherwise noted 30 Far1-GEF molecules. https://doi.org/10.1371/journal.pcbi.1011523.g004 Second, slow trafficking of pheromone receptors is key to promote indecisive behavior: if trafficking rates were to be increased, then receptor internalization would deprive Far1 of the potential for seeding competing clusters, while delivery of receptors to currently dominant clusters would stabilize them. Indeed, we found that accelerating receptor traffic (reducing receptor residence time) switched the behavior of the combined model, from indecisive relocation to stable polarity (Fig 4B). Third, increasing the abundance of Far1-GEF would increase the number of newly seeded Cdc42 clusters, driving the system towards a more uniform Cdc42 distribution. This was also observed (Fig 4C). Changing either receptor trafficking rates or Far1-GEF abundance revealed parameter regimes that produced unpolarized states, polarized states, or switching between the two (Fig 4B and 4C). To directly visualize the hypothesized competition between clusters nucleated by Far1-GEF, we distributed immobile Far1-GEF molecules at specific locations in the simulation domain, so that each fixed Far1-GEF molecule could initiate a cluster of Cdc42. As the number of Far1-GEF molecules increased, the competition time between seeded Cdc42 clusters also increased, delaying the establishment of a single “winner” polarity site (Fig 4D). Cdc42 clusters relocated between different Far1-GEF “seeds” in an indecisive manner, and even when one cluster became dominant, smaller clusters continued to coexist (Fig 4E and S5 Movie). To confirm that our results held in three dimensions, we performed 3D simulations of the combined polarity circuit. Most parameters for 2D and 3D simulations were kept the same (Tables 1 and 2) so that the steady-state amount of species were similar between the two types of simulations (S7 Fig). Polarity sites in the 3D simulations relocated erratically around the cell cortex, as seen with indecisive polarity clustering in yeast cells (Fig 4F and S6 Movie). In sum, these simulations indicate that a system combining the core Bem1-GEF circuit with the receptor-Far1 circuit can exhibit indecisive polarity behavior, with clusters of Cdc42 rapidly relocating. Pheromone gradients act to stabilize the polarity site Our simulations demonstrated the Far1 pathway can destabilize polarity to promote relocation in the presence of a uniform pheromone concentration. By contrast, experimental studies found that this pathway is required for orientation and stabilization of polarity sites towards a mating partner [27,49–51]. One way to reconcile these findings is that the Far1 pathway’s effects are dependent on the pheromone concentration profile that cells experience. A yeast cell emits ~ 1400 pheromone molecules per second [52]. With this number, the maximum pheromone concentration that the partner cell is exposed to is likely to be below 10 nM [19]. Surprisingly, yeast cells secreting only 20% as much pheromone (peak concentration around 1–2 nM) are still able to stabilize their mating partner’s polarity site [53]. This is well below the concentration of uniform pheromone required for stabilization of a polarity site [51,54]. These findings suggested it may be the pheromone gradient, rather than the absolute pheromone concentration, that stabilizes the position of the polarity site [53]. To test the plausibility of this hypothesis, we included pheromone molecules in our combined polarity circuit simulations (Fig 5A and Table 3). In this situation the receptors are activated upon pheromone binding, allowing us to test whether a pheromone gradient can change polarity site behavior. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Simulating pheromone gradients. A) Reaction scheme for pheromone receptor binding. B) A 3D pheromone gradient was simulated between an emitter and receiver cell as described in Clark-Cotton et al. [19]. C and D) Time-averaged pheromone concentrations for two different emission rates (C—650 molecules/sec with a uniform background pheromone concentration of 1.5 nM, and D—150 molecules/sec). Concentrations were measured in the “rings” shown below the x-axis. Error bars denote ± std. https://doi.org/10.1371/journal.pcbi.1011523.g005 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Parameters for the pheromone-induced receptor-Far1 pathway. https://doi.org/10.1371/journal.pcbi.1011523.t003 Although the actual pheromone gradients experienced by mating yeast cells have not been experimentally visualized, previous work simulated the pheromone gradient that would be perceived by a cell with an adjacent partner secreting pheromone from its polarity site [19] (Fig 5B), assuming experimentally measured pheromone emission rates [52]. To simulate cells that are initially searching for a partner, we initialized the simulations with a uniform pheromone concentration of 1.5 nM, which resulted in relocating Cdc42 clusters (S8 Fig). Then, to simulate a situation in which the cell orients polarity (and hence pheromone secretion) towards the partner cell, we imposed a pheromone gradient that varied between 1.5–5.8 nM (Fig 5C). In many but not all simulations, this gradient led to cessation of polarity site relocation within 50 min and development of a stable cluster of Cdc42 oriented towards the pheromone source (Fig 6A and S7 Movie). We next ran simulations using a gradient that ranged from 0–1.2 nM (Fig 5D). Remarkably, in this case all simulations produced a stable polarity site located in the region of high pheromone within 30 min (Fig 6B and S8 Movie). These results are consistent with experimental findings in which cells secreting only 20% of the amount of pheromone as wildtype cells are able to stabilize the polarity sites of mating partners [53]. Polarity stabilization occurred more rapidly in 0–1.2 nM than in 1.5–5.8 nM gradients (Fig 6C), consistent with the idea that the main obstacle to polarity stabilization is provided by competition with distinct polarity clusters seeded by the background levels of pheromone. In summary, our simulations of the combined polarity circuit demonstrate that while a uniform concentration of pheromone destabilizes polarity and promotes relocation, a local pheromone gradient can stabilize polarity. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Pheromone gradients stabilize the polarity site. Simulation results using the two gradients shown in Fig 5. For both cases, cells were exposed to 1.5 nM uniform pheromone for the first 5 min. The pheromone gradient was applied starting at 5 min. A) Results for the gradient shown in Fig 5C. Time series for K for 10 simulations (Top). Distributions for active Cdc42 corresponding to the time series shown in black (Bottom). B) Same as A using gradient shown in Fig 5D. C) Time for K to 50 for the two gradients. https://doi.org/10.1371/journal.pcbi.1011523.g006 Noise-driven relocation of the polarity site To systematically investigate whether the core yeast polarity circuit can yield indecisive Cdc42 clustering like that seen in mating cells, we used a previously validated model developed to describe Cdc42 behavior in vegetative yeast cells [12,17,32,34] (Fig 1A and Table 1). The model species are inactive Cdc42 (which can exchange between membrane and cytosolic compartments), active Cdc42 (which is restricted to the membrane compartment), and a heterodimer of an effector and an activator of Cdc42 (here called Bem1-GEF). This complex also exchanges between membrane and cytosol, and can bind to active Cdc42. At the membrane, the complex promotes activation of inactive Cdc42. The main structural feature of the model is the positive feedback loop formed by mutual activation of Cdc42 and Bem1-GEF (Fig 1A, inset). In this feedback loop, active Cdc42 recruits Bem1-GEF from the cytoplasm and increases its GEF activity. In turn, Bem1-GEF activates Cdc42, leading to further Cdc42 recruitment from the cytosolic pool. This positive feedback loop amplifies local fluctuations in Cdc42 activity eventually leading to the formation of a polarity site [12,35]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Behavior of the core polarity circuit. A) Reaction scheme for the core polarity circuit of yeast (diagram adapted from Pablo et al. [17]). Polarity is driven by positive feedback through mutual activation (Inset). B) Two-parameter bifurcation diagram for molecular abundances of Cdc42 and Bem1-GEF (Unpolarized regime–green, transient polarity–blue, polarized–red). C) Simulation results for the transient polarity regime. Time series for Ripley’s K-function values (a measure of Cdc42 clustering) were generated for ten simulations (left panel). Distribution of K values (right panel). D) Distributions of Cdc42-GTP molecules at indicated times. Distributions are taken from the black time series in C using time points indicated with ticks on the time axis. Simulations correspond to 3000 Cdc42 and 102 Bem1-GEF molecules using uniform random distributions for Cdc42 and Bem1-GEF as initial conditions. E) Time series for number of clusters. Simulation results taken from the black time series in C (left panel). Example of cluster count from experiments (right panel). https://doi.org/10.1371/journal.pcbi.1011523.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Parameters for the core polarity circuit. https://doi.org/10.1371/journal.pcbi.1011523.t001 Our first goal was to determine if molecular-level fluctuations arising from the stochastic nature of biochemical reactions and diffusion are sufficient to promote relocation of the polarity site. To test this possibility we used the simulation platform Smoldyn [33,36] to perform particle-based simulations of the core polarity circuit, with parameters described previously [17]. We performed simulations with varying numbers of Cdc42 and Bem1-GEF molecules. These initial simulations revealed three distinct types of behavior: 1) unpolarized, in which no significant Cdc42 clusters formed (S3A Fig), 2) polarized, in which a single strong cluster was maintained (S3B Fig), and 3) transient polarity, in which clusters could spontaneously form, disappear, and re-form (Fig 1C and 1D and S1 Movie). These results suggest that molecular-level noise is able to trigger intermittent transitions between polarized and unpolarized steady states, consistent with previous work on regulatory systems [37–39]. To quantify our results, we used a normalized version of Ripley’s K-function (K) [40,41]. K measures clustering strength as the deviation of the observed particle distribution from that of a uniform distribution [17]. A value of K close to zero signifies a uniform distribution, and the larger the value of K the more polarized the distribution. We designated a threshold value of 1.5 to distinguish unpolarized (K < 1.5) from polarized (K ≥ 1.5) states for all 2D simulations (S1A Fig). In the transient polarity regime, the K values exhibit a bimodal distribution, consistent with a bistable system that switches between polarized and unpolarized states (Fig 1C, right panel) [16,37–39]. To investigate how robustly the core polarity circuit can give rise to transient polarity, we developed a method for distinguishing the three types of behavior based on the distribution of K values (Methods and S2 Fig). Specifically, we defined the transient polarity regime according to the bimodal distribution of K values (S2 Fig). Our analysis revealed that the parameter regime supporting transient polarity is limited to a small range of molecular abundances (Fig 1B), as also found in earlier work [32]. Thus, it would require very precise control for cells to exploit this parameter regime to develop transitory polarity. Moreover, the transient polarity regime exhibits transitions between long-lived unpolarized and polarized states with a single cluster, rather than multiple rapidly fluctuating clusters observed in indecisive yeast cells (Fig 1E). We conclude that the indecisive polarity behavior of mating yeast cells is unlikely to be a simple consequence of molecular noise acting on the core polarity circuit. The receptor-Far1 pathway During mating, there is an additional mechanism for bringing GEF molecules to the membrane. Specifically, following exposure to pheromone, GEF molecules in complex with Far1 (Far1-GEF) are recruited to the membrane through Far1’s interaction with Gβγ released by active receptors. Far1’s interaction with Gβγ couples the polarity circuit to the extracellular pheromone concentration. In addition, the receptor-Far1 circuit forms a second positive feedback loop, because active Cdc42 leads to the trafficking of new receptors to the membrane (Fig 2A). To gain insight into the behavior of the receptor-Far1 circuit, we considered a model consisting of Cdc42, receptors and Far1-GEF, neglecting Bem1-GEF. To simplify the model, the G proteins that mediate interaction between receptors and Far1-GEF are not modeled explicitly. That is, we assume that active receptors recruit Far1-GEF from the cytoplasm. Once at the membrane Far1-GEF promotes Cdc42 activation with the same efficacy as Bem1-GEF in complex with active Cdc42 (Fig 2A and Table 2). We also assume all pheromone receptors are active (this is analogous to the situation in which cells are exposed to saturating pheromone concentration). Pheromone receptors are delivered to the cell surface via Cdc42-oriented actin cables. The model does not explicitly take actin cables or vesicle delivery into account. Rather, we assume the rate at which receptors are delivered is determined by the local concentration of active Cdc42 (see Methods for details). Surface receptors are internalized via endocytosis, at well-characterized rates [42,43]. We further assume that receptors diffuse very slowly at the membrane, consistent with experimental findings [18]. As noted above, because Cdc42 clusters enhance local accumulation of receptors, and receptors recruit Far1-GEF to activate Cdc42, this pathway forms a mutual activation positive feedback loop (Fig 2A, inset). Thus, we anticipated that the receptor-Far1 circuit, like the Bem1-GEF circuit (Fig 1A), would have the capacity to spontaneously polarize. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Behavior of the receptor-Far1 circuit. A) Reaction scheme for the receptor-Far1 circuit. This circuit forms a second positive feedback loop through mutual activation (inset). B) Distributions of active Cdc42 (top row) and receptors (bottom row) shown for 0 min and 0.3 min. Long receptor lifetimes prevent maintenance of a polarity site. Simulations performed using 3000 Cdc42, 280 Far1-GEF, and 2500 receptor molecules. Initial conditions consisted of a polarized cluster of 3000 Cdc42-GTP and uniformly distributed receptor and Far1-GEF molecules. C) Final K values as a function of receptor membrane residence time. Values of K taken at t = 4000 secs. Data points represent averaged values of K for 30 simulations and the shaded area denotes standard deviation. Snapshots are for active Cdc42 at t = 4000 secs. https://doi.org/10.1371/journal.pcbi.1011523.g002 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Parameters for the uniformly activated receptor-Far1 pathway. https://doi.org/10.1371/journal.pcbi.1011523.t002 When we simulated this circuit using realistic receptor trafficking rates and initial particle distributions drawn from a uniform distribution, the model did not spontaneously develop a polarized cluster of Cdc42 in biologically relevant timescales. Even when the initial conditions included a strong cluster of active Cdc42 but distributed receptors, the initial clustering was rapidly lost (Fig 2B). To understand why a polarized distribution of Cdc42 was not maintained, consider that receptor trafficking occurs at a much slower rate than the rates for Cdc42 and Far1-GEF association and dissociation: the average membrane residence time of receptors is about 8 minutes [42,43] compared to a few seconds for Cdc42 [46,47] and Far1-GEF [44]. Thus, any clusters of Cdc42 dissipate before they can be reinforced by receptor traffic. Consistent with that interpretation, when the receptor residence time was decreased by increasing receptor trafficking rates the receptor-Far1 circuit did spontaneously establish polarity (Fig 2C). Our simulations indicate that in spite of its positive feedback loop (Fig 2A), the receptor-Far1 circuit by itself cannot establish a polarity site due to the mismatch between the residence times for Cdc42 (short) and receptor (long). However, it is not clear how the Far1 pathway might influence polarity behavior if the Bem1-Cdc42 circuit was also present, as is the case during the indecisive phase. To understand how these pathways interact, we modeled a system that combined the polarity circuit (Bem1-GEF and Cdc42) with the receptor-Far1 pathway (receptors and Far1-GEF). The receptor-Far1 circuit promotes polarity site relocation and generates indecisive behavior The combined polarity circuit (Fig 3A) includes two interconnected positive feedback loops, one from the core polarity circuit and the other from the receptor-Far1 circuit (Fig 3A, inset). Importantly, these feedback loops operate on different time scales, with the faster polarity circuit supporting polarization, whereas the slower receptor-Far1 circuit is unable to generate a stable polarity site. Therefore, it is not obvious how the combined system will behave. To investigate the behavior of the combined polarity circuit we performed simulations starting from two different initial conditions: a uniform random distribution of pathway components and a pre-polarized Cdc42 cluster. Similar to the polarity circuit alone (Fig 1B), the combined system exhibited one of three behaviors (Figs 3B and S4) depending on the molecular abundance of the components: an unpolarized regime with little Cdc42 clustering (S5A Fig), a polarized regime with stable and static clusters (S5B Fig), and an intermediate regime where clusters can form or disappear (Fig 3C). However, the unpolarized regime was expanded and the transition to the polarized regime required a larger amount of the Bem1-GEF complex (compare Figs 3B and 1B). In between, there was a substantially larger transient polarity regime. Interestingly, in this regime the behavior of the system appeared to be qualitatively different from the model with only the core polarity circuit. Rather than a single well-formed polarity site appearing and dispersing (Fig 1D), the addition of Far1-GEF seemed to promote multiple transient Cdc42 clusters that dynamically relocate in a manner reminiscent of indecisive polarity clustering in yeast cells (Fig 3C and S2 Movie). To quantify this observation, we employed a cluster detection algorithm based on Voronoi tessellation [48] and tracked the number of detected clusters over time (S6A Fig). From these time series we computed the fraction of time each model spent in states with 0, 1 and 2+ clusters and transition probabilities between states (S6B Fig). We then compared our simulation results directly with a similar analysis of the data presented in Clark-Cotton et al. 2021 [19] (see Methods for details). Our analysis revealed that coexisting Cdc42 clusters in the combined model were more frequent and exhibited similar behavior to polarity sites of indecisive yeast cells (S6 Fig). We also computed distributions for the dwell times in states with 0 or 1 clusters for the two models (S6C Fig). These distributions demonstrate that transitions between cluster states occur more frequently in the combined model than in the core polarity circuit alone. (We are not able to compute similar distributions from the experimental data because in the experiments, data were taken every 2 minutes, which is insufficient to resolve all transitions.) Note that indecisive behavior is also distinct from the unpolarized state in that it exhibits higher values of K with larger fluctuations and multiple coexisting clusters, which are absent in the unpolarized state. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Combined circuit allows relocation and stabilization of the polarity site. A) Reaction scheme for the combined polarity circuit. The combined model consists of dual positive feedback architecture (inset). B) Two-parameter bifurcation diagram for molecular abundances of Cdc42 and Bem1-GEF (Unpolarized regime–green, transient polarity–blue, polarized–red). C)–E): Simulations results for the transient polarity regime. Plots for K consist of 10 realizations. The distributions for active Cdc42 come from the black time series using the time points indicated by tick marks on the time axis. All simulations were performed using 3000 Cdc42, 170 Bem1-GEF, 30 Far1-GEF and 2500 receptor molecules, with different initial conditions as indicated. C) Multiple transient polarity sites form using uniform random distributions as initial conditions. In a subset of the simulations, a single polarity site is eventually formed. D) An established polarity site rapidly dissipates following the addition of randomly distributed receptors and Far1-GEF molecules. E) A polarity site is rapidly established and stably maintained for simulations started with a cluster of active receptors and uniformly distributed Cdc42 molecules. https://doi.org/10.1371/journal.pcbi.1011523.g003 When we initiated the simulations with a pre-polarized Cdc42 cluster but uniform receptors, the cluster immediately dissipated (Fig 3D and S3 Movie). Thus, even though the receptor-Far1 circuit has a positive feedback architecture, the slow timescale of receptor trafficking allows distributed receptors to overcome Bem1-GEF-maintained polarity. Note that in many of the simulations the polarity site did eventually reform (Fig 3D). In these cases the receptor distribution also became polarized. We suggest the following mechanism to explain how indecisive behavior arises in the combined model. Far1-GEF molecules recruited to the membrane by active receptors can seed multiple small Cdc42 clusters, which are then amplified by Bem1-GEF-mediated positive feedback. Without Far1-GEF, clusters would compete to yield a single winning polarity site [16,17]. However, as this competition process begins, Far1-GEF nucleates new clusters, thwarting effective competition: although some Cdc42 cluster(s) may transiently become dominant, newly seeded clusters continue to compete such that no stable cluster can develop. For this mechanism to operate, there are several requirements that the system must satisfy. First, the distributed nature of active receptors is key to the continual seeding of new clusters: if receptors were instead clustered, then the combined Bem1-GEF and Far1-GEF positive feedbacks would act synergistically to maintain a Cdc42 cluster. This was indeed the case. When simulations were run with an initially focused receptor distribution, all simulations produced a well-polarized Cdc42 distribution (Fig 3E and S4 Movie). As initial receptor distributions were varied, more dispersed receptor distributions lowered the probability that the system would remain polarized (Fig 4A). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. The Far1 pathway promotes indecisive behavior by delaying or preventing coarsening of polarity sites. A) The fraction of time polarized as a function of the degree of dispersion in the initial receptor distribution. The fraction of time was measured as the ratio of K values ≥ 1.5 to the total number of K values in the interval 2000 to 4000 secs to avoid any initial transients. Each point represents a single simulation result from 30 total simulations. Initial receptor distributions were generated from 2D Gaussian distributions. The degree of dispersion represents the standard deviation of the distribution. B) Final K values (t = 4000 secs) as a function of receptor membrane residence time. 5 ≤ n ≤ 30 simulations for each data point. Standard deviations are shaded. C) Final K values as a function of Far1-GEF molecule number. 5 ≤ n ≤ 30 simulations for each data point. D) Time to stabilization of the polarity site (K ≥ 3) as a function of number of fixed Far1-GEF molecules in the membrane. Distributions of Far1-GEF molecules are shown along the x-axis. E) Time series of K values for the case of 15 fixed Far1-GEF molecules (left panel). Right panels show distributions for active Cdc42 molecules for the black time series at times indicated with ticks on the time axis. F) Results for a 3D particle-based simulation. Cdc42T molecules shown as blue dots on the cell surface. Rate constants converted from 2D case as described in the Methods and listed in Tables 1 and 2. All simulations performed using 3000 Cdc42, 170 Bem1-GEF, 2500 receptor and unless otherwise noted 30 Far1-GEF molecules. https://doi.org/10.1371/journal.pcbi.1011523.g004 Second, slow trafficking of pheromone receptors is key to promote indecisive behavior: if trafficking rates were to be increased, then receptor internalization would deprive Far1 of the potential for seeding competing clusters, while delivery of receptors to currently dominant clusters would stabilize them. Indeed, we found that accelerating receptor traffic (reducing receptor residence time) switched the behavior of the combined model, from indecisive relocation to stable polarity (Fig 4B). Third, increasing the abundance of Far1-GEF would increase the number of newly seeded Cdc42 clusters, driving the system towards a more uniform Cdc42 distribution. This was also observed (Fig 4C). Changing either receptor trafficking rates or Far1-GEF abundance revealed parameter regimes that produced unpolarized states, polarized states, or switching between the two (Fig 4B and 4C). To directly visualize the hypothesized competition between clusters nucleated by Far1-GEF, we distributed immobile Far1-GEF molecules at specific locations in the simulation domain, so that each fixed Far1-GEF molecule could initiate a cluster of Cdc42. As the number of Far1-GEF molecules increased, the competition time between seeded Cdc42 clusters also increased, delaying the establishment of a single “winner” polarity site (Fig 4D). Cdc42 clusters relocated between different Far1-GEF “seeds” in an indecisive manner, and even when one cluster became dominant, smaller clusters continued to coexist (Fig 4E and S5 Movie). To confirm that our results held in three dimensions, we performed 3D simulations of the combined polarity circuit. Most parameters for 2D and 3D simulations were kept the same (Tables 1 and 2) so that the steady-state amount of species were similar between the two types of simulations (S7 Fig). Polarity sites in the 3D simulations relocated erratically around the cell cortex, as seen with indecisive polarity clustering in yeast cells (Fig 4F and S6 Movie). In sum, these simulations indicate that a system combining the core Bem1-GEF circuit with the receptor-Far1 circuit can exhibit indecisive polarity behavior, with clusters of Cdc42 rapidly relocating. Pheromone gradients act to stabilize the polarity site Our simulations demonstrated the Far1 pathway can destabilize polarity to promote relocation in the presence of a uniform pheromone concentration. By contrast, experimental studies found that this pathway is required for orientation and stabilization of polarity sites towards a mating partner [27,49–51]. One way to reconcile these findings is that the Far1 pathway’s effects are dependent on the pheromone concentration profile that cells experience. A yeast cell emits ~ 1400 pheromone molecules per second [52]. With this number, the maximum pheromone concentration that the partner cell is exposed to is likely to be below 10 nM [19]. Surprisingly, yeast cells secreting only 20% as much pheromone (peak concentration around 1–2 nM) are still able to stabilize their mating partner’s polarity site [53]. This is well below the concentration of uniform pheromone required for stabilization of a polarity site [51,54]. These findings suggested it may be the pheromone gradient, rather than the absolute pheromone concentration, that stabilizes the position of the polarity site [53]. To test the plausibility of this hypothesis, we included pheromone molecules in our combined polarity circuit simulations (Fig 5A and Table 3). In this situation the receptors are activated upon pheromone binding, allowing us to test whether a pheromone gradient can change polarity site behavior. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Simulating pheromone gradients. A) Reaction scheme for pheromone receptor binding. B) A 3D pheromone gradient was simulated between an emitter and receiver cell as described in Clark-Cotton et al. [19]. C and D) Time-averaged pheromone concentrations for two different emission rates (C—650 molecules/sec with a uniform background pheromone concentration of 1.5 nM, and D—150 molecules/sec). Concentrations were measured in the “rings” shown below the x-axis. Error bars denote ± std. https://doi.org/10.1371/journal.pcbi.1011523.g005 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Parameters for the pheromone-induced receptor-Far1 pathway. https://doi.org/10.1371/journal.pcbi.1011523.t003 Although the actual pheromone gradients experienced by mating yeast cells have not been experimentally visualized, previous work simulated the pheromone gradient that would be perceived by a cell with an adjacent partner secreting pheromone from its polarity site [19] (Fig 5B), assuming experimentally measured pheromone emission rates [52]. To simulate cells that are initially searching for a partner, we initialized the simulations with a uniform pheromone concentration of 1.5 nM, which resulted in relocating Cdc42 clusters (S8 Fig). Then, to simulate a situation in which the cell orients polarity (and hence pheromone secretion) towards the partner cell, we imposed a pheromone gradient that varied between 1.5–5.8 nM (Fig 5C). In many but not all simulations, this gradient led to cessation of polarity site relocation within 50 min and development of a stable cluster of Cdc42 oriented towards the pheromone source (Fig 6A and S7 Movie). We next ran simulations using a gradient that ranged from 0–1.2 nM (Fig 5D). Remarkably, in this case all simulations produced a stable polarity site located in the region of high pheromone within 30 min (Fig 6B and S8 Movie). These results are consistent with experimental findings in which cells secreting only 20% of the amount of pheromone as wildtype cells are able to stabilize the polarity sites of mating partners [53]. Polarity stabilization occurred more rapidly in 0–1.2 nM than in 1.5–5.8 nM gradients (Fig 6C), consistent with the idea that the main obstacle to polarity stabilization is provided by competition with distinct polarity clusters seeded by the background levels of pheromone. In summary, our simulations of the combined polarity circuit demonstrate that while a uniform concentration of pheromone destabilizes polarity and promotes relocation, a local pheromone gradient can stabilize polarity. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Pheromone gradients stabilize the polarity site. Simulation results using the two gradients shown in Fig 5. For both cases, cells were exposed to 1.5 nM uniform pheromone for the first 5 min. The pheromone gradient was applied starting at 5 min. A) Results for the gradient shown in Fig 5C. Time series for K for 10 simulations (Top). Distributions for active Cdc42 corresponding to the time series shown in black (Bottom). B) Same as A using gradient shown in Fig 5D. C) Time for K to 50 for the two gradients. https://doi.org/10.1371/journal.pcbi.1011523.g006 Discussion Models of the yeast polarity circuit The first realistic model for the yeast polarity circuit was proposed by Goryachev and Pokhilko, who considered a deterministic model that consisted of a set of reaction-diffusion equations [12]. They used their model to demonstrate that Bem1-GEF mediated positive feedback coupled with diffusion rates that vary between the membrane and cytosol was sufficient to generate a stable polarity site through a Turing mechanism [12]. Several more abstract minimalistic models, containing as few as two chemical species with non-linear positive feedback, were also studied to gain a deeper understanding of how cells might use Rho-family GTPases to generate spatial patterns. Because of their mathematical tractability, these models produced important insights into the mechanisms underlying establishment, competition and coexistence of GTPase clusters [14,15,30,57–61]. While theoretical work has begun to establish conditions that support polarity establishment and coarsening, less attention has been given to identifying mechanisms that produce movement or relocation of the polarity site. Dynamic polarity site relocation is critical during the indecisive phase of yeast mating, when polarity clusters undergoes erratic assembly, disassembly, and seemingly random motion. Aspects of this behavior have been recapitulated by stochastic models that incorporate stochastic effects resulting from randomness in biochemical reactions. Particle-based simulations of a very simple two-species model were used to investigate the behavior of a system with linear positive feedback [30,31]. With linear positive feedback, clustering of Cdc42 only occurs in regimes with small molecule numbers, making the degree and location of clustering susceptible to molecular fluctuations. Hegemann et al. exploited the noise-driven relocation of polarity clusters in this model to explore how pheromone gradients might direct polarity to the correct site via the Far1 pathway [27]. However, experimental findings suggest that yeast polarization is robust to increased molecular abundances [16,62,63], indicative of more robust nonlinear positive feedback. Particle-based models with non-linear positive feedback could also display noise-driven relocation of polarity clusters, but (as with the linear feedback model) such behavior required fine-tuning and only occurred in a small region of parameter space [32]. Thus, it remained unclear whether simply incorporating molecular noise into a polarity circuit would suffice to explain the indecisive polarity behavior seen in mating yeast or if additional regulatory elements are required. Prior modeling studies produced dynamic polarity sites by including negative feedback in the polarity circuit. Ghose et al. showed that actin-mediated negative feedback via stochastic vesicle delivery promotes persistent movement of a single stable polarity site [50,64]. Khalili et al. used a reaction-diffusion equation model for mating in fission yeast to demonstrate how coupled positive and negative feedback regulation acting on Ras1 can generate a single polarity site that continually assembles and disassembles in different locations [65]. However, neither study reproduced multiple transient clusters of GTPase activity as observed in the indecisive phase of budding yeast mating. Shi et al. implemented a local excitation, global inhibition biased excitable network (LEGI-BEN) model to describe cells undergoing chemotaxis [66]. In their model the LEGI module converts an external chemical gradient into an internal gradient of signaling activity which biases where the excitable network is active. In turn, the excitable network is linked to a polarization module involving the cytoskeleton allowing migration to occur in the direction of the gradient. This model differs from ours in that our model does not require an excitable network to generate multiple clusters of Cdc42 activity. Rather, active receptors seed formation of new clusters insuring a single polarity site does not form. Complementary positive feedback loops ensure successful mating Successful mating requires that yeast cells direct their growth toward potential mating partners, which requires proper positioning of the polarity site. To achieve this task, the polarity system must satisfy several often competing requirements: 1) detection of pheromone gradients, 2) establishment of a single polarity site, 3) reorientation of the polarity site when misaligned with the gradient, and 4) stabilization of the polarity site when oriented toward a mating partner. Our modeling results suggest that these requirements are met through a series of three coupled positive feedback loops acting on different time scales (Fig 7). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Summary of system architecture. Schematics of positive feedback loops for A) the core polarity circuit, B) the intracellular receptor-Far1 circuit, C) the combined circuit and D) the trans-cellular receptor-Far1 circuit. u denotes the membrane-associated slowly-diffusing “active” form, while v denotes the cytosolic, rapidly diffusing “inactive” form of the relevant protein—Cdc42 is indicated by the subscript C, Bem1-GEF by the subscript G, and Receptor-Far1-GEF by the subscript R. https://doi.org/10.1371/journal.pcbi.1011523.g007 The first positive feedback loop forms the core polarity circuit (Fig 7A). In this circuit, positive feedback occurs because active Cdc42 molecules increase the activation rate of neighboring Cdc42 molecules by recruiting active GEF molecules to their location. Because diffusion in the cytosol is fast as compared to diffusion in the membrane, this positive feedback loop provides an effective mechanism for forming clusters of active Cdc42. Rapid diffusion in the cytosol also allows competition between coexisting clusters for polarity factors in the cytosol. Larger clusters generate stronger positive feedback and thus are able to out-compete smaller clusters, leading to a “winner take all” mechanism for establishing a single polarity site. While strong positive feedback is effective at generating a unique polarity site, it also exhibits undesirable features for gradient tracking. First, initiation of a polarity cluster is highly susceptible to fluctuations in the local concentrations of relevant molecules, so that polarity sites can easily form at sites that are misaligned with the pheromone gradient. Second, it is difficult to relocate a polarity site once it has been established: except in a very narrow slice of parameter space, this makes polarity sites resistant to molecular fluctuations (Fig 7A). The Far1 pathway by which the polarity circuit is coupled to pheromone sensing in yeast appears to overcome both of these drawbacks (Fig 7B). Active pheromone receptors recruit GEF molecules in complex with the multifunctional protein Far1, thereby activating Cdc42 near active receptors. Thus, active receptors continuously seed formation of new Cdc42 clusters, while blunting competition between coexisting clusters. This diminished competition generates dynamic clusters of Cdc42 that form and disassemble throughout the cell surface, providing a search mechanism for finding a local pheromone source. The combined polarity/Far1 circuit exhibits a much larger region of parameter space where molecular fluctuations can drive polarity site relocation (Fig 7C). Interestingly, receptor-mediated activation of Cdc42 forms a second positive feedback loop because active Cdc42 molecules recruit actin cables along which vesicles containing new receptors that are trafficked (Fig 7B). Due to long receptor lifetimes on the membrane [67,68], this feedback loop does not support polarity establishment under conditions of constant pheromone concentration. However, in the presence of a pheromone gradient, positive feedback by the receptor-Far1 circuit operates more strongly in the region of high pheromone concentration, because receptor activation is higher leading to more active Cdc42 in this region. In turn, higher Cdc42 activity leads to an increase in the local concentration of receptors, reinforcing Cdc42 activity and promoting the formation of a stable polarity site. Interestingly, this mechanism of gradient tracking is able to respond effectively to pheromone gradients of low peak concentration (1.2 nM), consistent with observations that cells in which pheromone production has been reduced by 80% are still able to successfully signal their location to mating partners [53]. A potential limitation of this mechanism for gradient tracking is that cells can be confused when the gradient is superimposed on a background of constant pheromone (e.g., the case in which the gradient ranges from 1.5–5.8 nM). As pheromone concentrations experienced by yeast in the wild are likely to be quite variable, we speculate that yeast have evolved additional mechanisms to overcome this limitation. Finally, a third positive feedback loop acts between cells of opposite mating type (Fig 7D). All haploid yeast cells emit mating-type-specific pheromones, which bind to receptors on the surface of cells of the opposite mating type. Signaling through these receptors both increases the rate of pheromone production within the receiving cell and, by locally activating Cdc42, increases the probability that pheromone is released in the direction of the opposite mating type. Our simulations confirm experimental findings that this localized release of pheromone is important for establishing gradients steep enough to be detected by the adjacent mating partner [19]. This positive feedback loop is likely to play a role during the indecisive phase of mating when both cells’ polarity sites are dynamic. Considering this two-cell scenario will be the subject of future investigations. Biological implications and future directions Our conclusions may be applicable to polarity regulation in other systems. Similar to S. cerevisiae, polarity sites in the distantly related fission yeast S. pombe also exhibit erratic assembly-disassembly behavior during mating, and are stabilized when they happen to align with a partner [69,70]. Interestingly, fission yeast lack the Far1 pathway, but possess a pathway with the same architecture, where pheromone-receptor binding leads to local recruitment of a Cdc42 GEF [71]. Thus, our findings may apply broadly to fungal mating systems [25]. Our work suggests that molecular noise is sufficient to promote relocation of polarity clusters in cells that combine the polarity circuit with the receptor-Far1 pathway. Far1 is a multi-functional protein that is required for cell cycle arrest. Therefore, deletion of this protein would not provide a test of our model. However, it is possible to construct point mutants that specifically target Far1’s various functions. Such mutants would provide a mechanism for testing our model. It is also possible that other phenomena may also promote mobility of polarity clusters. In particular, polarized vesicle trafficking can cause dilution and lateral displacement of polarity factors [55,64,72,73]. In the future, it may be useful to investigate whether a model combining vesicle traffic and the receptor-Far1 pathway can accurately reproduce polarity site behaviors in mating yeast. Our proposed mechanism of polarity site relocation has implications for a range of cell functions beyond mating yeast cells. For example, migratory cells often exhibit relocating polarity sites while moving in chemical gradients [1,74,75]. In axon development, multiple polarity sites (neurites) are formed initially, and one of them differentiates into an axon as it moves up chemical gradients [76]. Many of these functions require receptor endocytosis [77–79]. It will be interesting to investigate whether dynamic polarity might result from differences in the time scales of polarity factor diffusion and receptor trafficking. Comparing the principles of polarity in mating yeast to those in other eukaryotic cells promises to provide insight into many cell functions. Models of the yeast polarity circuit The first realistic model for the yeast polarity circuit was proposed by Goryachev and Pokhilko, who considered a deterministic model that consisted of a set of reaction-diffusion equations [12]. They used their model to demonstrate that Bem1-GEF mediated positive feedback coupled with diffusion rates that vary between the membrane and cytosol was sufficient to generate a stable polarity site through a Turing mechanism [12]. Several more abstract minimalistic models, containing as few as two chemical species with non-linear positive feedback, were also studied to gain a deeper understanding of how cells might use Rho-family GTPases to generate spatial patterns. Because of their mathematical tractability, these models produced important insights into the mechanisms underlying establishment, competition and coexistence of GTPase clusters [14,15,30,57–61]. While theoretical work has begun to establish conditions that support polarity establishment and coarsening, less attention has been given to identifying mechanisms that produce movement or relocation of the polarity site. Dynamic polarity site relocation is critical during the indecisive phase of yeast mating, when polarity clusters undergoes erratic assembly, disassembly, and seemingly random motion. Aspects of this behavior have been recapitulated by stochastic models that incorporate stochastic effects resulting from randomness in biochemical reactions. Particle-based simulations of a very simple two-species model were used to investigate the behavior of a system with linear positive feedback [30,31]. With linear positive feedback, clustering of Cdc42 only occurs in regimes with small molecule numbers, making the degree and location of clustering susceptible to molecular fluctuations. Hegemann et al. exploited the noise-driven relocation of polarity clusters in this model to explore how pheromone gradients might direct polarity to the correct site via the Far1 pathway [27]. However, experimental findings suggest that yeast polarization is robust to increased molecular abundances [16,62,63], indicative of more robust nonlinear positive feedback. Particle-based models with non-linear positive feedback could also display noise-driven relocation of polarity clusters, but (as with the linear feedback model) such behavior required fine-tuning and only occurred in a small region of parameter space [32]. Thus, it remained unclear whether simply incorporating molecular noise into a polarity circuit would suffice to explain the indecisive polarity behavior seen in mating yeast or if additional regulatory elements are required. Prior modeling studies produced dynamic polarity sites by including negative feedback in the polarity circuit. Ghose et al. showed that actin-mediated negative feedback via stochastic vesicle delivery promotes persistent movement of a single stable polarity site [50,64]. Khalili et al. used a reaction-diffusion equation model for mating in fission yeast to demonstrate how coupled positive and negative feedback regulation acting on Ras1 can generate a single polarity site that continually assembles and disassembles in different locations [65]. However, neither study reproduced multiple transient clusters of GTPase activity as observed in the indecisive phase of budding yeast mating. Shi et al. implemented a local excitation, global inhibition biased excitable network (LEGI-BEN) model to describe cells undergoing chemotaxis [66]. In their model the LEGI module converts an external chemical gradient into an internal gradient of signaling activity which biases where the excitable network is active. In turn, the excitable network is linked to a polarization module involving the cytoskeleton allowing migration to occur in the direction of the gradient. This model differs from ours in that our model does not require an excitable network to generate multiple clusters of Cdc42 activity. Rather, active receptors seed formation of new clusters insuring a single polarity site does not form. Complementary positive feedback loops ensure successful mating Successful mating requires that yeast cells direct their growth toward potential mating partners, which requires proper positioning of the polarity site. To achieve this task, the polarity system must satisfy several often competing requirements: 1) detection of pheromone gradients, 2) establishment of a single polarity site, 3) reorientation of the polarity site when misaligned with the gradient, and 4) stabilization of the polarity site when oriented toward a mating partner. Our modeling results suggest that these requirements are met through a series of three coupled positive feedback loops acting on different time scales (Fig 7). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Summary of system architecture. Schematics of positive feedback loops for A) the core polarity circuit, B) the intracellular receptor-Far1 circuit, C) the combined circuit and D) the trans-cellular receptor-Far1 circuit. u denotes the membrane-associated slowly-diffusing “active” form, while v denotes the cytosolic, rapidly diffusing “inactive” form of the relevant protein—Cdc42 is indicated by the subscript C, Bem1-GEF by the subscript G, and Receptor-Far1-GEF by the subscript R. https://doi.org/10.1371/journal.pcbi.1011523.g007 The first positive feedback loop forms the core polarity circuit (Fig 7A). In this circuit, positive feedback occurs because active Cdc42 molecules increase the activation rate of neighboring Cdc42 molecules by recruiting active GEF molecules to their location. Because diffusion in the cytosol is fast as compared to diffusion in the membrane, this positive feedback loop provides an effective mechanism for forming clusters of active Cdc42. Rapid diffusion in the cytosol also allows competition between coexisting clusters for polarity factors in the cytosol. Larger clusters generate stronger positive feedback and thus are able to out-compete smaller clusters, leading to a “winner take all” mechanism for establishing a single polarity site. While strong positive feedback is effective at generating a unique polarity site, it also exhibits undesirable features for gradient tracking. First, initiation of a polarity cluster is highly susceptible to fluctuations in the local concentrations of relevant molecules, so that polarity sites can easily form at sites that are misaligned with the pheromone gradient. Second, it is difficult to relocate a polarity site once it has been established: except in a very narrow slice of parameter space, this makes polarity sites resistant to molecular fluctuations (Fig 7A). The Far1 pathway by which the polarity circuit is coupled to pheromone sensing in yeast appears to overcome both of these drawbacks (Fig 7B). Active pheromone receptors recruit GEF molecules in complex with the multifunctional protein Far1, thereby activating Cdc42 near active receptors. Thus, active receptors continuously seed formation of new Cdc42 clusters, while blunting competition between coexisting clusters. This diminished competition generates dynamic clusters of Cdc42 that form and disassemble throughout the cell surface, providing a search mechanism for finding a local pheromone source. The combined polarity/Far1 circuit exhibits a much larger region of parameter space where molecular fluctuations can drive polarity site relocation (Fig 7C). Interestingly, receptor-mediated activation of Cdc42 forms a second positive feedback loop because active Cdc42 molecules recruit actin cables along which vesicles containing new receptors that are trafficked (Fig 7B). Due to long receptor lifetimes on the membrane [67,68], this feedback loop does not support polarity establishment under conditions of constant pheromone concentration. However, in the presence of a pheromone gradient, positive feedback by the receptor-Far1 circuit operates more strongly in the region of high pheromone concentration, because receptor activation is higher leading to more active Cdc42 in this region. In turn, higher Cdc42 activity leads to an increase in the local concentration of receptors, reinforcing Cdc42 activity and promoting the formation of a stable polarity site. Interestingly, this mechanism of gradient tracking is able to respond effectively to pheromone gradients of low peak concentration (1.2 nM), consistent with observations that cells in which pheromone production has been reduced by 80% are still able to successfully signal their location to mating partners [53]. A potential limitation of this mechanism for gradient tracking is that cells can be confused when the gradient is superimposed on a background of constant pheromone (e.g., the case in which the gradient ranges from 1.5–5.8 nM). As pheromone concentrations experienced by yeast in the wild are likely to be quite variable, we speculate that yeast have evolved additional mechanisms to overcome this limitation. Finally, a third positive feedback loop acts between cells of opposite mating type (Fig 7D). All haploid yeast cells emit mating-type-specific pheromones, which bind to receptors on the surface of cells of the opposite mating type. Signaling through these receptors both increases the rate of pheromone production within the receiving cell and, by locally activating Cdc42, increases the probability that pheromone is released in the direction of the opposite mating type. Our simulations confirm experimental findings that this localized release of pheromone is important for establishing gradients steep enough to be detected by the adjacent mating partner [19]. This positive feedback loop is likely to play a role during the indecisive phase of mating when both cells’ polarity sites are dynamic. Considering this two-cell scenario will be the subject of future investigations. Biological implications and future directions Our conclusions may be applicable to polarity regulation in other systems. Similar to S. cerevisiae, polarity sites in the distantly related fission yeast S. pombe also exhibit erratic assembly-disassembly behavior during mating, and are stabilized when they happen to align with a partner [69,70]. Interestingly, fission yeast lack the Far1 pathway, but possess a pathway with the same architecture, where pheromone-receptor binding leads to local recruitment of a Cdc42 GEF [71]. Thus, our findings may apply broadly to fungal mating systems [25]. Our work suggests that molecular noise is sufficient to promote relocation of polarity clusters in cells that combine the polarity circuit with the receptor-Far1 pathway. Far1 is a multi-functional protein that is required for cell cycle arrest. Therefore, deletion of this protein would not provide a test of our model. However, it is possible to construct point mutants that specifically target Far1’s various functions. Such mutants would provide a mechanism for testing our model. It is also possible that other phenomena may also promote mobility of polarity clusters. In particular, polarized vesicle trafficking can cause dilution and lateral displacement of polarity factors [55,64,72,73]. In the future, it may be useful to investigate whether a model combining vesicle traffic and the receptor-Far1 pathway can accurately reproduce polarity site behaviors in mating yeast. Our proposed mechanism of polarity site relocation has implications for a range of cell functions beyond mating yeast cells. For example, migratory cells often exhibit relocating polarity sites while moving in chemical gradients [1,74,75]. In axon development, multiple polarity sites (neurites) are formed initially, and one of them differentiates into an axon as it moves up chemical gradients [76]. Many of these functions require receptor endocytosis [77–79]. It will be interesting to investigate whether dynamic polarity might result from differences in the time scales of polarity factor diffusion and receptor trafficking. Comparing the principles of polarity in mating yeast to those in other eukaryotic cells promises to provide insight into many cell functions. Methods 2D Particle-based model All particle-based simulations were performed using Smoldyn (v2.67) with continuous space and discretized time intervals on a Linux-based computing system (Longleaf cluster at UNC Chapel Hill, 2.50 GHz and 2.30 GHz Intel Processors) [33,36]. Periodic boundary conditions were assumed in both spatial directions. Molecules were regarded as point particles with no volumes. Brownian motion of the molecules was simulated with the Euler-Maruyama method as follows. Let x(t) and y(t) represent the coordinates of a given molecule at time t, then molecule’s position at t + Δt is calculated as: where Zx and Zy are independent random numbers drawn from the standard normal distribution, and D is the diffusion coefficient. Coordinates of all species were stored every 10 seconds from a 4000-second simulation. Second-order reactions are governed by two parameters: the reaction radius ρ and reaction rate λ. When two reacting partners are within a distance ρ, then the probability that they react within a time step Δt is P = 1-exp(-λΔt). When Δt is sufficiently small, P ≈ λΔt. When bound molecules dissociate, they are placed at a distance which is 0.00001 μm beyond the binding radius ρ to avoid immediate reassociation. First-order reactions occur with the probability P = 1-exp(-kΔt), where k is the rate constant for the reaction. Parameter values used in the simulations are listed in Tables 1 and 2. 3D Particle-based model The yeast cell was approximated as a sphere. Membrane-bound species diffuse on the surface of the sphere. Cytosolic species diffuse inside the sphere. Reaction rates for the 3D model are listed in Tables 1–3. Initial choices for the rate constants in the 3D simulations were obtained by converting rate constants from the 2D simulations using the method of Ramirez et al. [32]. However, a few rate constants required additional tuning to ensure the steady-state concentrations of the 2D and 3D simulations were similar. Rate constants for reactions between membrane-bound species did not change between the 2D and 3D simulations. Rate constants for first-order reactions in which a cytosolic species associated with the membrane were scaled as follows: Vc is the volume of the sphere, and Am is the area of the membrane. The rate constants for second-order reactions that involved a cytosolic species and membrane-bound species were scaled as follows: V2D is the 2D reaction area, and V3D is the reaction volume for 3D simulations. The reason V3D is half the volume of a sphere is because one of the reactants is membrane-bound. Simulating receptor cycling Actin cables directed by Cdc42 bring pheromone receptors to the polarity site. We did not explicitly take into account actin cables in our simulations. Actin-dependent delivery of receptors was implicitly modeled as a bimolecular reaction. Each individual Cdc42-GTP is able to recruit a receptor from the cytoplasm to the membrane with a specified rate (λ9 in Table 2, and λ12 in Table 3) during each time step. Membrane receptors are internalized via endocytosis, which was modeled as a first-order reaction with specified rate constants (k10 in Table 2, and k14, k15 in Table 3). Simulating pheromone molecules 3D Pheromone gradients were simulated according to the method described previously [19]. A pair of mating cells were modeled as two spheres with a diameter of 5 μm. One cell emitted pheromone. We assumed that the number of released pheromone molecules followed a Poisson process, and all molecules were emitted from a single point source. Vesicle release events were simulated in Smoldyn using the command pointsource. Pheromone was removed at a spherical absorbing boundary 7 μm from the origin. Cell membranes were treated as reflecting boundaries. Quantifying polarity Polarity was measured using normalized versions of Ripley’s K-function [17,32,40,41,80]. The function K(r) measures the deviation of the current particle distribution from a uniform distribution based on the cumulative distribution of pairwise molecular distances P(r).: For particle distributions on a 2D plane: For particle distributions on a 3D sphere: where mi(r)Δr is the number of molecules at a distance d from molecule i such that r ≤ d ≤ r+Δr, N is the total number of molecules in the domain, and A is the area of the domain. K(r) = 0 if particles are distributed uniformly. For a non-uniform distribution, the K-function typically goes through a maximum value for a finite value of r. To quantify polarity, we used the maximum value of K. For 2D simulations, we defined a molecule distribution with K < 1.5 as unpolarized states; K ≥ 1.5 as polarized states; and K ≥ 3 as stable polarity (S1A Fig). For 3D simulations, we defined a distribution with K < 30 as unpolarized states; K ≥ 30 as polarized states; and K ≥ 50 as stable polarity (S1B Fig). Identifying polarity regimes To identify the three polarity regimes (unpolarized, transient polarity, and polarized), we recorded values of K every 10 seconds during the last 2000 seconds of each simulation. Simulations were run using two kinds of initial conditions: random and polarized. For random initial conditions, positions of the Cdc42 molecules were chosen from a uniform distribution, whereas for the polarized case position for the Cdc42 molecules were chosen according to a Gaussian in the center of the domain with a standard deviation of 0.2. For a given number of Cdc42 and Bem1-GEF molecules, we ran ten simulations for each type of initial condition. For each simulation, we calculated the difference Δ between the number of polarized states F(K ≥ 1.5) and number of unpolarized states F(K < 1.5): If the system spends half its time in a polarized state and half unpolarized, Δ equals 0. A large positive value of Δ indicates a polarized regime and a large negative value indicates an unpolarized regime. Let N equal the total number of K values recorded during simulation. To identify the transient polarity regime, we used the condition: with α = 0.85. If Δ/N > α, the regime was categorized as polarized and Δ/N < -μ unpolarized. For a given number of Cdc42 molecules, the relationship between Δ and the number of Bem1-GEF molecules, x, was assumed to follow the logistic function: This function was then fit to the simulation results to determine values of Bem1-GEF molecules at which the system transitioned between polarity regimes (S2 and S4 Figs). Cluster detection by Voronoi tessellation We utilized a previous method based on Voronoi tessellation to detect clusters [48,81]. A cluster was defined as a group of adjacent Voronoi polygons with sizes below a predetermined threshold. The threshold for polygon sizes was determined as follows. As a negative control, we generated 50 realizations of particle distributions from a uniform distribution. These distributions were used to generate a probability density for Voronoi polygon sizes. We also obtained the probability density for the polygon size distribution of Cdc42-GTP molecules generated from our simulations. The intersection of the probability densities from the negative control and our simulations defines the threshold of polygon sizes for cluster formation. A polygon was deleted from the Voronoi diagram if its size was above the threshold, meaning that the density of molecules in the polygon was too low to be considered as part of a cluster. The remaining polygons on the Voronoi diagram would be considered as components of potential clusters. Then, we grouped the neighboring polygons into clusters and applied filters for cluster area and number density. The threshold area of a cluster was set to be 0.785 μm2 as the diameter of a polarity site of yeast was estimated to be 1 μm. We set the threshold number of Cdc42 in a cluster according to a previous measurement [62]. Groups of polygons passed the filters were recorded as true clusters. Manual cluster detection The analysis of polarity cluster behavior in mating yeast cells was performed using videos reported by Clark-Cotton et al. [19]. These consist of time-lapse two-color confocal imaging of a mixture of opposite-mating-type S. cerevisiae strains containing green (DLY9069: MATa BEM1-GFP) and red (DLY12944: MATα BEM1-tdTomato) polarity probes. Yeast growth and imaging conditions were described in detail in Clark-Cotton et al. [19]. When cells of opposite mating type are mixed together to allow mating, they are initially growing vegetatively by budding. As the budding cycle finishes, the Bem1 probe localizes to the mother-bud neck for cytokinesis, after which polarity clusters exhibit indecisive behavior for some time. A subset of cells then align their polarity sites towards each other, entering the committed phase that culminates in cell-cell fusion. For this analysis, we considered only cells that had completed cytokinesis (Bem1 probe no longer at the bud neck) but had not yet aligned polarity. Thus, all cells are considered “indecisive”. Fluorescent signals from Z-stacks were projected onto a single plane to observe polarity clusters (intense groups of pixels in red or green channels). Images taken at 2-min intervals were visually scored to count the number of polarity clusters. A polarity cluster was defined as a grouping of >10 pixels with intensity clearly above background. Groups of intense pixels that were separated by clear zones of background intensity were counted as separate clusters. 2D Particle-based model All particle-based simulations were performed using Smoldyn (v2.67) with continuous space and discretized time intervals on a Linux-based computing system (Longleaf cluster at UNC Chapel Hill, 2.50 GHz and 2.30 GHz Intel Processors) [33,36]. Periodic boundary conditions were assumed in both spatial directions. Molecules were regarded as point particles with no volumes. Brownian motion of the molecules was simulated with the Euler-Maruyama method as follows. Let x(t) and y(t) represent the coordinates of a given molecule at time t, then molecule’s position at t + Δt is calculated as: where Zx and Zy are independent random numbers drawn from the standard normal distribution, and D is the diffusion coefficient. Coordinates of all species were stored every 10 seconds from a 4000-second simulation. Second-order reactions are governed by two parameters: the reaction radius ρ and reaction rate λ. When two reacting partners are within a distance ρ, then the probability that they react within a time step Δt is P = 1-exp(-λΔt). When Δt is sufficiently small, P ≈ λΔt. When bound molecules dissociate, they are placed at a distance which is 0.00001 μm beyond the binding radius ρ to avoid immediate reassociation. First-order reactions occur with the probability P = 1-exp(-kΔt), where k is the rate constant for the reaction. Parameter values used in the simulations are listed in Tables 1 and 2. 3D Particle-based model The yeast cell was approximated as a sphere. Membrane-bound species diffuse on the surface of the sphere. Cytosolic species diffuse inside the sphere. Reaction rates for the 3D model are listed in Tables 1–3. Initial choices for the rate constants in the 3D simulations were obtained by converting rate constants from the 2D simulations using the method of Ramirez et al. [32]. However, a few rate constants required additional tuning to ensure the steady-state concentrations of the 2D and 3D simulations were similar. Rate constants for reactions between membrane-bound species did not change between the 2D and 3D simulations. Rate constants for first-order reactions in which a cytosolic species associated with the membrane were scaled as follows: Vc is the volume of the sphere, and Am is the area of the membrane. The rate constants for second-order reactions that involved a cytosolic species and membrane-bound species were scaled as follows: V2D is the 2D reaction area, and V3D is the reaction volume for 3D simulations. The reason V3D is half the volume of a sphere is because one of the reactants is membrane-bound. Simulating receptor cycling Actin cables directed by Cdc42 bring pheromone receptors to the polarity site. We did not explicitly take into account actin cables in our simulations. Actin-dependent delivery of receptors was implicitly modeled as a bimolecular reaction. Each individual Cdc42-GTP is able to recruit a receptor from the cytoplasm to the membrane with a specified rate (λ9 in Table 2, and λ12 in Table 3) during each time step. Membrane receptors are internalized via endocytosis, which was modeled as a first-order reaction with specified rate constants (k10 in Table 2, and k14, k15 in Table 3). Simulating pheromone molecules 3D Pheromone gradients were simulated according to the method described previously [19]. A pair of mating cells were modeled as two spheres with a diameter of 5 μm. One cell emitted pheromone. We assumed that the number of released pheromone molecules followed a Poisson process, and all molecules were emitted from a single point source. Vesicle release events were simulated in Smoldyn using the command pointsource. Pheromone was removed at a spherical absorbing boundary 7 μm from the origin. Cell membranes were treated as reflecting boundaries. Quantifying polarity Polarity was measured using normalized versions of Ripley’s K-function [17,32,40,41,80]. The function K(r) measures the deviation of the current particle distribution from a uniform distribution based on the cumulative distribution of pairwise molecular distances P(r).: For particle distributions on a 2D plane: For particle distributions on a 3D sphere: where mi(r)Δr is the number of molecules at a distance d from molecule i such that r ≤ d ≤ r+Δr, N is the total number of molecules in the domain, and A is the area of the domain. K(r) = 0 if particles are distributed uniformly. For a non-uniform distribution, the K-function typically goes through a maximum value for a finite value of r. To quantify polarity, we used the maximum value of K. For 2D simulations, we defined a molecule distribution with K < 1.5 as unpolarized states; K ≥ 1.5 as polarized states; and K ≥ 3 as stable polarity (S1A Fig). For 3D simulations, we defined a distribution with K < 30 as unpolarized states; K ≥ 30 as polarized states; and K ≥ 50 as stable polarity (S1B Fig). Identifying polarity regimes To identify the three polarity regimes (unpolarized, transient polarity, and polarized), we recorded values of K every 10 seconds during the last 2000 seconds of each simulation. Simulations were run using two kinds of initial conditions: random and polarized. For random initial conditions, positions of the Cdc42 molecules were chosen from a uniform distribution, whereas for the polarized case position for the Cdc42 molecules were chosen according to a Gaussian in the center of the domain with a standard deviation of 0.2. For a given number of Cdc42 and Bem1-GEF molecules, we ran ten simulations for each type of initial condition. For each simulation, we calculated the difference Δ between the number of polarized states F(K ≥ 1.5) and number of unpolarized states F(K < 1.5): If the system spends half its time in a polarized state and half unpolarized, Δ equals 0. A large positive value of Δ indicates a polarized regime and a large negative value indicates an unpolarized regime. Let N equal the total number of K values recorded during simulation. To identify the transient polarity regime, we used the condition: with α = 0.85. If Δ/N > α, the regime was categorized as polarized and Δ/N < -μ unpolarized. For a given number of Cdc42 molecules, the relationship between Δ and the number of Bem1-GEF molecules, x, was assumed to follow the logistic function: This function was then fit to the simulation results to determine values of Bem1-GEF molecules at which the system transitioned between polarity regimes (S2 and S4 Figs). Cluster detection by Voronoi tessellation We utilized a previous method based on Voronoi tessellation to detect clusters [48,81]. A cluster was defined as a group of adjacent Voronoi polygons with sizes below a predetermined threshold. The threshold for polygon sizes was determined as follows. As a negative control, we generated 50 realizations of particle distributions from a uniform distribution. These distributions were used to generate a probability density for Voronoi polygon sizes. We also obtained the probability density for the polygon size distribution of Cdc42-GTP molecules generated from our simulations. The intersection of the probability densities from the negative control and our simulations defines the threshold of polygon sizes for cluster formation. A polygon was deleted from the Voronoi diagram if its size was above the threshold, meaning that the density of molecules in the polygon was too low to be considered as part of a cluster. The remaining polygons on the Voronoi diagram would be considered as components of potential clusters. Then, we grouped the neighboring polygons into clusters and applied filters for cluster area and number density. The threshold area of a cluster was set to be 0.785 μm2 as the diameter of a polarity site of yeast was estimated to be 1 μm. We set the threshold number of Cdc42 in a cluster according to a previous measurement [62]. Groups of polygons passed the filters were recorded as true clusters. Manual cluster detection The analysis of polarity cluster behavior in mating yeast cells was performed using videos reported by Clark-Cotton et al. [19]. These consist of time-lapse two-color confocal imaging of a mixture of opposite-mating-type S. cerevisiae strains containing green (DLY9069: MATa BEM1-GFP) and red (DLY12944: MATα BEM1-tdTomato) polarity probes. Yeast growth and imaging conditions were described in detail in Clark-Cotton et al. [19]. When cells of opposite mating type are mixed together to allow mating, they are initially growing vegetatively by budding. As the budding cycle finishes, the Bem1 probe localizes to the mother-bud neck for cytokinesis, after which polarity clusters exhibit indecisive behavior for some time. A subset of cells then align their polarity sites towards each other, entering the committed phase that culminates in cell-cell fusion. For this analysis, we considered only cells that had completed cytokinesis (Bem1 probe no longer at the bud neck) but had not yet aligned polarity. Thus, all cells are considered “indecisive”. Fluorescent signals from Z-stacks were projected onto a single plane to observe polarity clusters (intense groups of pixels in red or green channels). Images taken at 2-min intervals were visually scored to count the number of polarity clusters. A polarity cluster was defined as a grouping of >10 pixels with intensity clearly above background. Groups of intense pixels that were separated by clear zones of background intensity were counted as separate clusters. Supporting information S1 Fig. Examples of the Ripley’s K-function. Cluster distributions corresponding to the listed K values. A) 2D examples for a square domain of length 8.8623 μm. B) 3D examples for a sphere of radius 2.5 μm. https://doi.org/10.1371/journal.pcbi.1011523.s001 (PDF) S2 Fig. Polarity regimes for the core polarity circuit. A) Difference in the number of polarized and unpolarized states as a function of Bem1-GEF abundance. Data points (o) represent simulation results for the total number of Cdc42 molecules indicated in the plot title. Curves represent fits to the data using a logistic function (unpolarized regime—green, polarized regime—red, and transient regime—blue). B and C) Distributions for K in the transient regime using a total Cdc42 abundance of 3000. Stars indicate the Bem1-GEF abundances used to produce the distributions (100 lower panels and 106 upper panels). https://doi.org/10.1371/journal.pcbi.1011523.s002 (PDF) S3 Fig. Simulation results for unpolarized and polarized regimes of the core polarity circuit. A) Time series for K for 10 simulations in the unpolarized regime (left panel). Distributions for active Cdc42 taken form black time series using time points indicated with ticks on the time axis (right panels). Simulations were performed using uniform random distributions of molecules as initial conditions with 3000 Cdc42 and 70 Bem1-GEF molecules. B) Same as A except with 3000 Cdc42 and 120 Bem1-GEF molecules. https://doi.org/10.1371/journal.pcbi.1011523.s003 (PDF) S4 Fig. Polarity regimes for the combined polarity circuit. A) Difference in the number of polarized and unpolarized states as a function of Bem1-GEF abundance. Data points (o) represent simulation results for the total number of Cdc42 molecules indicated in the plot title. Curves represent fits to the data using a logistic function (unpolarized regime—green, polarized regime—red, and transient regime—blue). B and C) Distributions for K in the transient regime using a total Cdc42 abundance of 3500. Stars indicate the Bem1-GEF abundances (135 lower panels and 165 upper panels). All simulations were performed using 30 Far1-GEF and 2500 receptor molecules. https://doi.org/10.1371/journal.pcbi.1011523.s004 (PDF) S5 Fig. Simulation results for unpolarized and polarized regimes of the combined polarity circuit. A) Time series for K for 10 simulations in the unpolarized regime (left panel). Distributions for active Cdc42 taken form black time series using time points indicated with ticks on the time axis (right panels). Simulations were performed using uniform random distributions of molecules as initial conditions with 3000 Cdc42, 80 Bem1-GEF, 30 Far1-GEF and 2500 receptor molecules. B) Same as A except with 280 Bem1-GEF. https://doi.org/10.1371/journal.pcbi.1011523.s005 (PDF) S6 Fig. Number of clusters as a function of time. A) Five examples of time series for the number of clusters, B) the distribution of cluster counts, the transition probabilities to a subsequent cluster count, and C) the dwell time of clusters for simulations of the core polarity circuit with 3000 Cdc42 and 102 Bem1-GEF molecules and simulations of the combined polarity circuit with 3000 Cdc42 and 170 Bem1-GEF molecules. Note that the images were taken at 120-second intervals in experiments, so we were not able to compare dwell times of clusters between simulations and experiments. The total time duration in A) are 4000 secs with intervals of 120 secs. The red line denotes the time at which cells became committed. https://doi.org/10.1371/journal.pcbi.1011523.s006 (PDF) S7 Fig. Comparison of 2D and 3D models. Simulations were performed with 3000 Cdc42, 170 Bem1-GEF, 30 Far1-GEF, and 2500 receptor molecules. Simulations started with uniformly distributed molecules. Lines represent molecule number averages and the shaded areas represent ± std for 30 simulations (3D –blue, 2D –red). All simulations were run for 66 min. https://doi.org/10.1371/journal.pcbi.1011523.s007 (PDF) S8 Fig. Indecisive polarity behavior with uniform pheromone concentration. Simulations were performed using 3000 Cdc42, 170 Bem1-GEF, 2500 receptor, 30 Far1-GEF and 1.5 nM uniform concentration of pheromone molecules. https://doi.org/10.1371/journal.pcbi.1011523.s008 (PDF) S1 Movie. Spatiotemporal distribution of Cdc42-GTP in a transient polarity regime of the core polarity circuit model simulation. Blue dots represent individual molecules of active Cdc42. Corresponds to Fig 1D. https://doi.org/10.1371/journal.pcbi.1011523.s009 (MP4) S2 Movie. Spatiotemporal distribution of Cdc42-GTP in a transient polarity regime of the combined polarity circuit model simulation. Corresponds to Fig 3C. https://doi.org/10.1371/journal.pcbi.1011523.s010 (MP4) S3 Movie. A cluster of Cdc42-GTP dissipates after introducing the Far1 pathway. Receptors and Far1-GEF molecules were introduced at 400 seconds. Corresponds to Fig 3D. https://doi.org/10.1371/journal.pcbi.1011523.s011 (MP4) S4 Movie. A cluster of Cdc42-GTP is established and maintained if receptors are initially polarized. Corresponds to Fig 3E. https://doi.org/10.1371/journal.pcbi.1011523.s012 (MP4) S5 Movie. Cdc42-GTP clusters relocate between fixed Far1-GEF molecules. 15 fixed Far1-GEF molecules directly activate Cdc42 without receptors. Corresponds to Fig 4E. https://doi.org/10.1371/journal.pcbi.1011523.s013 (MP4) S6 Movie. Spatiotemporal distribution of Cdc42-GTP of the 3D combined polarity circuit model simulation. Corresponds to Fig 4F. https://doi.org/10.1371/journal.pcbi.1011523.s014 (MP4) S7 Movie. A 1.5–5.8 nM pheromone gradient can promote stable formation of Cdc42-GTP clusters. 1.5 nM uniform pheromone was applied for the first 5 min. Starting from 5 min, a gradient was imposed. Corresponds to Fig 6A. https://doi.org/10.1371/journal.pcbi.1011523.s015 (MP4) S8 Movie. A 0–1.2 nM pheromone gradient can efficiently stabilize Cdc42-GTP clusters. 1.5 nM uniform pheromone was applied for the first 5 min. Starting from 5 min, a gradient was imposed. Corresponds to Fig 6B. https://doi.org/10.1371/journal.pcbi.1011523.s016 (MP4) Acknowledgments We thank Steven S. Andrews for developing and updating the computer program Smoldyn which is used for simulations in the project. We also thank all members of the Elston lab for helpful discussions on the project.

Weber, Leah L.;Zhang, Chuanyi;Ochoa, Idoia;El-Kebir, Mohammed

doi: 10.1371/journal.pcbi.1011544pmid: 37819942

Introduction Cancer results from an evolutionary process that yields a heterogeneous tumor composed of multiple subpopulations of cells, or clones, with distinct sets of somatic mutations [1] (Fig 1a). These mutations include single-nucleotide variants (SNVs) that alter a single base and copy-number aberrations (CNAs) that amplify or delete large genomic regions. Over the last decade, new developments in single-cell DNA sequencing (scDNA-seq) methods have helped uncover a wealth of insights regarding intra-tumor heterogeneity and cancer evolution [2–5]. In particular, the ongoing development and application of high-throughput, ultra-low coverage scDNA-seq technologies (<1×), such as direct library preparation (DLP+) [6] and acoustic cell tagmentation (ACT) [7], have paved the way for enriching our understanding of the role CNAs play in cancer progression and tumor evolution [6–8]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Phertilizer infers a clonal tree T, clonal genotypes Y and a cell clustering ϕ given ultra-low coverage single-cell sequencing data. (a) A tumor consists of clones with distinct genotypes. (b) Ultra-low coverage scDNA-seq produces total read counts and variant read counts for n cells and m SNV loci, and low dimension embedding of binned read counts . (c) Given maximum copy number c and sequencing error probability α, Phertilizer infers a clonal tree T, clonal genotypes Y and cell clustering ϕ with maximum posterior probability . https://doi.org/10.1371/journal.pcbi.1011544.g001 The advantage these ultra-low coverage scDNA-seq technologies have over other high-throughput scDNA-seq methods (>1×), like Mission Bio Tapestry [9], is the uniformity of coverage. This uniformity implies that the observed read counts for a genomic region is proportional to copy number, making it ideal for the analysis of subclonal CNAs that occur in only a small subset of tumor cells. However, this uniformity comes at the cost of sequencing depth, making it very difficult to identify and characterize the evolution of SNVs from ultra-low coverage scDNA-seq. Critically, to comprehensively study the evolution of a tumor from the same set of cells, both CNAs and SNVs should ideally be characterized by a single tumor phylogeny that depicts their coevolution. While this remains a long-term goal for the field, a first step in this direction is to increase our understanding of SNV evolution from ultra-low coverage scDNA-seq data by incorporating reliable copy number information into the inference of SNV tumor phylogenies. Although phylogeny inference methods from bulk sequencing and cell clustering from single-cell RNA sequencing are expanding to incorporate both SNV and CNA features, such as TUSV-ext [10] and CASIC [11], current methods for tumor phylogeny and/or clone inference from single-cell sequencing naturally tend to focus on the features (SNV or CNA events) for which the data is ideally suited [12–26]. One exception is BiTSC2 [24], which infers a phylogeny containing both SNV and CNA events. However, the method is not designed for high-throughput ultra-coverage data, with efficacy demonstrated on datasets containing at most 500 cells and coverages as low as 3×. Another exception in the medium to high coverage scDNA-seq regime (>10×) is SCARLET [27], which refines a given copy number tree using SNV read counts under a CNA loss supported evolutionary model. While SCARLET accounts for sequencing errors and missing data, it was not designed to handle the extreme sparsity of ultra-low coverage scDNA-seq. SBMClone [28] took the first step of using ultra-low coverage sequencing data to infer SNV clones via stochastic block modeling. Despite good performance on simulated data, especially with higher coverage (>0.2×), SBMClone was unable to identify clear structure in a 10x Genomics breast cancer dataset [4] without ad hoc use of additional copy number clone information. Moreover, it is non-trivial to convert the inferred parameters of SBMClone’s stochastic block model to clonal genotypes, which may impact downstream analyses. Similarly, given a set of candidate SNV loci, SECEDO [29] first calls SNVs using a Bayesian filtering approach and then subsequently clusters cells using the called SNVs. While both of these clustering methods capitalize on the ever-increasing throughput of ultra-low coverage scDNA-seq methods, neither method constrains the output by a tree and CNA features are used only in an ad hoc manner or for orthogonal validation. As a result, both methods imply that CNA and SNV data features should be segregated and analyzed in separate bioinformatics pipelines. The other emerging trend from the analysis of ultra-low coverage scDNA-seq is pseudobulk analysis [6]. This approach begins by identifying copy number clones using existing methods, followed by pooling cells that belong to the same CNA clones into pseudobulk samples, which are then independently analyzed to identify SNVs. Finally, phylogeny inference is performed with the copy number clones as the leaves of the tree. By doing so, this method does not allow for further refinement of these clones based on SNV evolution. Here we introduce Phertilizer, the first method to infer an SNV clonal tree from ultra-low single-cell DNA sequencing of tumors. To overcome SNV coverage sparsity in this type of data, we leverage the strong copy number signal inherent in the data to guide clonal tree inference. By analogy to the planting and growing of trees, Phertilizer seeks to grow a clonal tree with maximum posterior probability by recursively inferring elementary clonal trees as building blocks. Our simulations demonstrate that Phertilizer accurately infers phylogenies and cell clusters when the number of cells matches current practice. In particular, Phertilizer outperforms a current method [28] for simultaneously clustering SNVs and cells as well as another commonly used ad hoc approach [6]. On real data, we find that Phertilizer effectively utilizes the copy-number signal inherent in the data to uncover clonal structure, yielding high-fidelity clonal genotypes. Materials and methods Problem statement Our goal is to infer an SNV phylogeny, guided by copy number aberrations, from ultra-low coverage sequencing data consisting of n cells and m identified single-nucleotide variants (SNVs). More precisely, we are given variant reads A = [aiq] and total reads D = [diq], where aiq and diq are the variant and total read counts for SNV locus q ∈ [m] in cell i ∈ [n], respectively (Fig 1b). While the number of cells is large with the latest generation of ultra-low coverage single-cell DNA sequencing technology (n ≈ 1000 cells), the coverage, or the average number of reads that span a single locus is uniform but low (0.01× to 0.5×). For example with a coverage of 0.01×, we would on average observe aiq = diq = 0 reads for 99 out of every 100 loci q for each cell i. This sparsity renders phylogeny inference using SNVs extremely challenging. We propose to overcome this challenge using the following three key ideas. First, similarly to current methods, we leverage the clonal structure present in tumors, i.e., cells typically cluster into a small number of clones. Thus, we seek to group the n observed cells into k clones (k ≪ n) via a cell clustering and corresponding clonal genotypes defined as follows: Definition 1 Function ϕ: [k] → 2[n] is a cell clustering provided its image encodes a partition of cells [n] into k (disjoint and non-empty) parts. Definition 2 Matrix Y ∈ {0, 1}k×m encodes clonal genotypes where yjq = 1 indicates that SNV q is present in clone j and yjq = 0 indicates that SNV q is absent in clone j. Second, because ultra-low coverage scDNA-seq is sequenced uniformly, we may leverage the copy number signal inherent in the data to improve cell clustering performance [11] and guide tree inference. More specifically, we expect that all cells in a clone have identical copy number profiles. As we do not observe copy number directly, we will use observed reads counts , where b is the number of genomic bins, as a proxy for copy number. From read counts R, we derive distances that reflect copy-number similarity between pairs of cells on a low-dimensional embedding of binned read counts of R (i.e., ℓ ≪ b)— see Section A.1 in S1 Appendix. Third, similarly to methods such as SCITE [12], SciCloneFit [15] and SPhyR [14], which operate on medium-to-high coverage scDNA-seq data, we consider that the observed cells are generated as the result of a tree-like evolutionary process that constrains the order of SNV clusters. In particular, we use the infinite sites model [30] defined as follows. Definition 3. A tree T with nodes {v1, …, vk} rooted at node v1 is a clonal tree for clonal genotypes Y = [y1, …, yk]⊤ provided (i) each node vj is labeled by clonal genotype yj and (ii) each SNV q is gained exactly once and subsequently never lost. That is, there exists no directed edge (vj′, vj) where the SNV is lost, i.e., yjq = 1 and yj′q = 0. Moreover, either the root node contains the SNV, i.e., y1q = 1, or there exists exactly one directed edge (vj′, vj) where the SNV is introduced, i.e., yj′q = 0 and yjq = 1. To relate the data to our latent variables of interest (T, Y, ϕ), we introduce a generative model in Section A.2 in S1 Appendix that describes the generation of variant read counts A and the binned read count embedding . This model (Fig A in S1 Appendix) requires two hyperparameters c and α, where is the upper bound on the total number of chromosomal copies at any locus in the genome and α ∈ [0, 1] is the probability of misreading a single nucleotide during sequencing. Importantly, while Definitions 1 to 3 explicitly indicate the number k of clones, the number k of clones is not a hyperparameter and will be part of the inference. Specifically, our generative model enables us to approximate the posterior probability for a clonal tree T with any number k of nodes and associated clonal genotypes Y and cell clustering ϕ (derivation in Section A.2 in S1 Appendix). However, due to limitations of the sequencing technology the number of clones that are detectable from the data may be fewer than the number of nodes in clonal tree T. The ability to detect clone j from the observed data is a function of the sequencing coverage, the number of cells in clone j and the number of SNVs newly introduced in clone j. To prevent overfitting of the data, the pre-specified detection threshold controls the minimum amount of observed data in support of each inferred clone. See Section A.3 in S1 Appendix for a formal definition and additional details. This leads to the following problem. Problem 1 (Clonal Tree Inference (CTI)) Given variant reads , total reads , binned read count embedding , maximum copy number , sequencing error probability α ∈ [0, 1] and detection threshold , find a clonal tree T with detectable clonal genotypes Y and cell clustering ϕ with maximum posterior probability . Phertilizer To solve the CTI problem, Phertilizer maintains a set of candidate trees throughout three phases: (i) initialization, (ii) growing, and (iii) ranking each tree in by its posterior probability. First, in the initialization phase, the set is initialized with a single tree containing only a root node v1. All n cells are assigned to the node’s cell cluster ϕ(v1) and all genotypes are initialized to for each SNV q. Second, in the growing phase (Section A.4 in S1 Appendix), Phertilizer recursively constructs the candidate set of clonal trees and the respective clonal genotypes and cell clusterings by performing three different elementary tree operations (Linear, Branching and Identity) on each leaf node vj of each candidate tree (Fig 2). Specifically, each operation takes as input (T, Y, ϕ) and yields a new clonal tree T′ with updated genotypes Y′ and cell clustering ϕ′ by extending leaf vj of T (Fig 2). The key idea is that each elementary tree operation breaks down the CTI problem into smaller subproblems. Intuitively, a Linear operation (Fig 2a) replaces a leaf node with a two node linear subtree, while a Branching operation (Fig 2b) replaces the leaf node with a three node binary subtree. The former represents stepwise acquisition of SNVs with evidence of intermediary clones present in the observed data, while the latter indicates evidence of divergence from a common ancestor [31]. The Identity (Fig 2c) operation does not modify the tree and is useful during the growing process where all candidate trees are enumerated. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Phertilizer solves the CTI problem by enumerating clonal trees using three elementary operations in a recursive fashion. (a) Each operation yields a new clonal tree T′ by extending a leaf vj of the previous clonal tree T and reassigning its SNVs Δ(yj) and cells ϕ(j). The resulting clonal genotypes Y′ and cell clustering ϕ′ are constrained as depicted: (b) Linear, (c) Branching, and (d) Identity. https://doi.org/10.1371/journal.pcbi.1011544.g002 These operations are defined more formally in Section A.4 in S1 Appendix. While the specifics vary slightly, both Linear and Branching are solved using a coordinate descent approach. That is, we fix clonal genotypes Y′ and solve for the cell clustering ϕ′ and alternate. Drawing parallels between image segmentation, where combining both pixel and pixel location features results in better clustering [32], we incorporate the binned read count embedding and variant read counts A into a single feature [11]. We then use this feature as input to the normalized cut algorithm [32] (worst case running time O(n3)) to obtain a cell clustering with two clusters. The advantage of combining SNV and CNA signal into a single feature is that cell clustering is improved when one or both of these signals are weak, subsequently improving the SNV partition which we solve for next. Given a fixed cell clustering ϕ′, we use our generative model to update clonal genotypes Y′ by assigning each SNV to the node in extended tree T′ with maximum posterior probability. This is done in time O(nm). We terminate this process upon convergence or when a maximum number of iterations is reached. This results in the running time of one elementary tree operation as O(n3+ nm). The resulting clonal tree T′ is then appended to the candidate set provided all its clone are detectable for a specified detection threshold t and meet additional regularization criteria (Section A.4 in S1 Appendix). Third, once no new clonal trees are added to the candidate set , we return the clonal tree T, clonal genotypes Y and cell clustering ϕ with maximum posterior probability after post-processing (Section A.4 in S1 Appendix). Importantly, the top down approach by Phertilizer requires that no assumptions be made a priori regarding the number of nodes k in the inferred clonal tree. Phertilizer is implemented in Python 3, open source (BSD-3-Clause), and available at https://github.com/elkebir-group/phertilizer. Problem statement Our goal is to infer an SNV phylogeny, guided by copy number aberrations, from ultra-low coverage sequencing data consisting of n cells and m identified single-nucleotide variants (SNVs). More precisely, we are given variant reads A = [aiq] and total reads D = [diq], where aiq and diq are the variant and total read counts for SNV locus q ∈ [m] in cell i ∈ [n], respectively (Fig 1b). While the number of cells is large with the latest generation of ultra-low coverage single-cell DNA sequencing technology (n ≈ 1000 cells), the coverage, or the average number of reads that span a single locus is uniform but low (0.01× to 0.5×). For example with a coverage of 0.01×, we would on average observe aiq = diq = 0 reads for 99 out of every 100 loci q for each cell i. This sparsity renders phylogeny inference using SNVs extremely challenging. We propose to overcome this challenge using the following three key ideas. First, similarly to current methods, we leverage the clonal structure present in tumors, i.e., cells typically cluster into a small number of clones. Thus, we seek to group the n observed cells into k clones (k ≪ n) via a cell clustering and corresponding clonal genotypes defined as follows: Definition 1 Function ϕ: [k] → 2[n] is a cell clustering provided its image encodes a partition of cells [n] into k (disjoint and non-empty) parts. Definition 2 Matrix Y ∈ {0, 1}k×m encodes clonal genotypes where yjq = 1 indicates that SNV q is present in clone j and yjq = 0 indicates that SNV q is absent in clone j. Second, because ultra-low coverage scDNA-seq is sequenced uniformly, we may leverage the copy number signal inherent in the data to improve cell clustering performance [11] and guide tree inference. More specifically, we expect that all cells in a clone have identical copy number profiles. As we do not observe copy number directly, we will use observed reads counts , where b is the number of genomic bins, as a proxy for copy number. From read counts R, we derive distances that reflect copy-number similarity between pairs of cells on a low-dimensional embedding of binned read counts of R (i.e., ℓ ≪ b)— see Section A.1 in S1 Appendix. Third, similarly to methods such as SCITE [12], SciCloneFit [15] and SPhyR [14], which operate on medium-to-high coverage scDNA-seq data, we consider that the observed cells are generated as the result of a tree-like evolutionary process that constrains the order of SNV clusters. In particular, we use the infinite sites model [30] defined as follows. Definition 3. A tree T with nodes {v1, …, vk} rooted at node v1 is a clonal tree for clonal genotypes Y = [y1, …, yk]⊤ provided (i) each node vj is labeled by clonal genotype yj and (ii) each SNV q is gained exactly once and subsequently never lost. That is, there exists no directed edge (vj′, vj) where the SNV is lost, i.e., yjq = 1 and yj′q = 0. Moreover, either the root node contains the SNV, i.e., y1q = 1, or there exists exactly one directed edge (vj′, vj) where the SNV is introduced, i.e., yj′q = 0 and yjq = 1. To relate the data to our latent variables of interest (T, Y, ϕ), we introduce a generative model in Section A.2 in S1 Appendix that describes the generation of variant read counts A and the binned read count embedding . This model (Fig A in S1 Appendix) requires two hyperparameters c and α, where is the upper bound on the total number of chromosomal copies at any locus in the genome and α ∈ [0, 1] is the probability of misreading a single nucleotide during sequencing. Importantly, while Definitions 1 to 3 explicitly indicate the number k of clones, the number k of clones is not a hyperparameter and will be part of the inference. Specifically, our generative model enables us to approximate the posterior probability for a clonal tree T with any number k of nodes and associated clonal genotypes Y and cell clustering ϕ (derivation in Section A.2 in S1 Appendix). However, due to limitations of the sequencing technology the number of clones that are detectable from the data may be fewer than the number of nodes in clonal tree T. The ability to detect clone j from the observed data is a function of the sequencing coverage, the number of cells in clone j and the number of SNVs newly introduced in clone j. To prevent overfitting of the data, the pre-specified detection threshold controls the minimum amount of observed data in support of each inferred clone. See Section A.3 in S1 Appendix for a formal definition and additional details. This leads to the following problem. Problem 1 (Clonal Tree Inference (CTI)) Given variant reads , total reads , binned read count embedding , maximum copy number , sequencing error probability α ∈ [0, 1] and detection threshold , find a clonal tree T with detectable clonal genotypes Y and cell clustering ϕ with maximum posterior probability . Phertilizer To solve the CTI problem, Phertilizer maintains a set of candidate trees throughout three phases: (i) initialization, (ii) growing, and (iii) ranking each tree in by its posterior probability. First, in the initialization phase, the set is initialized with a single tree containing only a root node v1. All n cells are assigned to the node’s cell cluster ϕ(v1) and all genotypes are initialized to for each SNV q. Second, in the growing phase (Section A.4 in S1 Appendix), Phertilizer recursively constructs the candidate set of clonal trees and the respective clonal genotypes and cell clusterings by performing three different elementary tree operations (Linear, Branching and Identity) on each leaf node vj of each candidate tree (Fig 2). Specifically, each operation takes as input (T, Y, ϕ) and yields a new clonal tree T′ with updated genotypes Y′ and cell clustering ϕ′ by extending leaf vj of T (Fig 2). The key idea is that each elementary tree operation breaks down the CTI problem into smaller subproblems. Intuitively, a Linear operation (Fig 2a) replaces a leaf node with a two node linear subtree, while a Branching operation (Fig 2b) replaces the leaf node with a three node binary subtree. The former represents stepwise acquisition of SNVs with evidence of intermediary clones present in the observed data, while the latter indicates evidence of divergence from a common ancestor [31]. The Identity (Fig 2c) operation does not modify the tree and is useful during the growing process where all candidate trees are enumerated. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Phertilizer solves the CTI problem by enumerating clonal trees using three elementary operations in a recursive fashion. (a) Each operation yields a new clonal tree T′ by extending a leaf vj of the previous clonal tree T and reassigning its SNVs Δ(yj) and cells ϕ(j). The resulting clonal genotypes Y′ and cell clustering ϕ′ are constrained as depicted: (b) Linear, (c) Branching, and (d) Identity. https://doi.org/10.1371/journal.pcbi.1011544.g002 These operations are defined more formally in Section A.4 in S1 Appendix. While the specifics vary slightly, both Linear and Branching are solved using a coordinate descent approach. That is, we fix clonal genotypes Y′ and solve for the cell clustering ϕ′ and alternate. Drawing parallels between image segmentation, where combining both pixel and pixel location features results in better clustering [32], we incorporate the binned read count embedding and variant read counts A into a single feature [11]. We then use this feature as input to the normalized cut algorithm [32] (worst case running time O(n3)) to obtain a cell clustering with two clusters. The advantage of combining SNV and CNA signal into a single feature is that cell clustering is improved when one or both of these signals are weak, subsequently improving the SNV partition which we solve for next. Given a fixed cell clustering ϕ′, we use our generative model to update clonal genotypes Y′ by assigning each SNV to the node in extended tree T′ with maximum posterior probability. This is done in time O(nm). We terminate this process upon convergence or when a maximum number of iterations is reached. This results in the running time of one elementary tree operation as O(n3+ nm). The resulting clonal tree T′ is then appended to the candidate set provided all its clone are detectable for a specified detection threshold t and meet additional regularization criteria (Section A.4 in S1 Appendix). Third, once no new clonal trees are added to the candidate set , we return the clonal tree T, clonal genotypes Y and cell clustering ϕ with maximum posterior probability after post-processing (Section A.4 in S1 Appendix). Importantly, the top down approach by Phertilizer requires that no assumptions be made a priori regarding the number of nodes k in the inferred clonal tree. Phertilizer is implemented in Python 3, open source (BSD-3-Clause), and available at https://github.com/elkebir-group/phertilizer. Results Simulation study Overview. To assess the performance of Phertilizer and compare it with previously proposed methods, we performed a simulation study with known ground truth clonal trees, evaluating the following four questions: (i) How accurate are the inferred clonal trees? (ii) How well is each method able to identify clusters of cells with similar clonal genotypes? (iii) How accurate are the inferred clonal genotypes? (iv) How sensitive is each method to violations of the infinite sites assumption? We designed our simulation study to match the characteristics of current datasets generated from ultra-low coverage scDNA-seq. To achieve this, we generated simulation instances with varying number k ∈ {5, 9} of nodes, number n ∈ {1000, 2000} of sequenced cells and number m ∈ {5000, 10000, 15000} of SNVs with a mean sequencing coverage g ∈ {0.01×, 0.05×, 0.1×}. We replicated each of these combinations 10 times for a total of 360 instances. See Section B.1 in S1 Appendix for details on the simulation instances. We assessed the quality of an inferred solution (T, ϕ, Y) against a ground-truth tree T*, cell clustering ϕ* and clonal genotypes Y* using ancestral pair recall (APR), incomparable pair recall (IPR), and clustered pair recall (CPR) metrics for cells and SNVs [33], as well as genotype similarity. In addition, we computed a single accuracy value (domain: [0, 1]) composed of the weighted average of APR, IPR and CPR—where the weights are proportional to the number of pairs in each class—such that an SNV and cell accuracy of 1 imply that the inferred solution perfectly matches ground truth. The genotype similarity equals 1 minus the normalized Hamming distance between ground truth genotypes and inferred genotypes of cells. We refer to Fig 3a Section B.1.4 in S1 Appendix for examples and more formal definitions. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Phertilizer outperforms Baseline and SBMClone on simulated data. We show aggregated results for n ∈ {1000, 2000} cells, k ∈ {5, 9} clones and coverage g = 0.05×. (a) Example of SNV accuracy, ancestral pair recall (APR), clustered pair recall (CPR), and incomparable pair recall (IPR) metrics. See Section B.1.4 in S1 Appendix for the corresponding example for cell metrics and genotype similarity. (b) Phertilizer outperforms Baseline +SCITE and SBMClone +SCITE in APR and IPR for both SNVs and cells. Although competing methods rank higher in CPR, overall Phertilizer performs the best considering the accuracy. (c) Phertilizer more accurately recovers clonal genotypes than competing methods. https://doi.org/10.1371/journal.pcbi.1011544.g003 We benchmarked against SBMClone [28] because it is the only existing clustering/genotyping method that co-clusters cells and SNVs. We also benchmarked against a commonly adopted ad hoc practice which we refer to as Baseline [6, 7]. In Baseline, cells are first clustered into clones from the read count embedding . Clonal genotypes are obtained by pooling the reads of all cells assigned to clone j and setting yjq = 1 when the ratio of alternate to total reads at each locus q exceeds a threshold of 0.05 (details provided in Section B.1.1 in S1 Appendix). Since there exists no standalone method that performs cell clustering, genotyping and tree inference for ultra-low coverage scDNA-seq, we paired SBMClone and Baseline with SCITE [12]. We attempted to include SCARLET [27] and BiTSC2 [24] in our benchmarking as these scDNA-seq methods incorporate copy number aberrations in their models. However both failed to run on our simulated data. Specifically, SCARLET, which was designed for medium-to-high coverage data, was unable to appropriately handle missing entries in the total read count matrix D. BiTSC2, an MCMC method, was unable to scale to the size of our input datasets—the largest instances considered in the BiTSC2 paper [24] had n = 500 cells and m = 200 SNVs, significantly smaller than our smallest simulation instances with n = 1000 cells and m = 5000 SNVs. For brevity, we focus our discussion on a coverage g = 0.05× and show aggregated results for n ∈ {1000, 2000} cells and k ∈ {5, 9} clones. We report the median for all performance metrics and include the interquartile range (IQR), i.e., the difference between the 75th and 25th percentiles of the data, when appropriate. We note deviations in trends where relevant, and refer to Section B.1.5 in S1 Appendix for remaining results. Results. We first evaluated the accuracy of SNV placement on the inferred tree by assessing APR, CPR, IPR and accuracy for SNVs (Fig 3b, Fig I in S1 Appendix). Overall, Phertilizer achieved the highest SNV accuracy (median: 0.90) in terms of SNV placement of all three methods (Baseline+SCITE median: 0.81, SBMClone+SCITE median: 0.54). For SNV APR, Phertilizer (median: 0.92) outperformed both Baseline+SCITE (median: 0.71) and SBMClone+SCITE (median: 0.38). This implies that our Linear operation reliably partitions SNVs accurately and that our Branching operation performs well at identifying SNVs that should be placed at the parent node. Moving on to SNV IPR, Phertilizer (median: 1.0, IQR: 0.11) outperformed SBMClone+SCITE (median: 0.48, IQR: 0.5) with lower variability than Baseline+SCITE (median: 1.0, IQR: 0.18). Thus, in addition to correctly identifying the SNVs in the parent node, the Branching operation also successfully partitions the SNVs in the children nodes. However, for SNV CPR, Phertilizer performed worse compared to SBMClone+SCITE (median: 0.98) and Baseline+SCITE (median: 0.97) but still maintained good performance (median 0.84). It is important to note that an SNV CPR of 1 is achievable by clustering all SNVs into a single cluster. Thus, the cost of underfitting the data, or grouping the SNVs into a few very large clusters, will be reflected in decreased APR and IPR. Indeed, we observed this to be the case for both Baseline+SCITE, which performed relatively poorly on APR, and SBMClone+SCITE which was the worst performing on both APR and IPR. Next, we evaluated the accuracy of the cell clustering and placement on the inferred tree. We observed similar trends across accuracy, APR, IPR and CPR cell performance metrics to those identified for their SNV counterparts (Fig 3b, Fig I in S1 Appendix). Phertilizer achieved the highest overall cell accuracy (median: 0.82) in comparison to SBMClone+SCITE (median: 0.60) and Baseline+SCITE (median: 0.56). Likewise, Phertilizer (median: 0.84) outperformed all other methods on cell APR (Baseline+SCITE median: 0.0, SBMClone+SCITE median: 0.32). On cell IPR, both Phertilizer (median: 1.0, IQR: 0.2) and Baseline+SCITE (median: 1.0, IQR: 0.13) significantly outperformed SBMClone+SCITE (median: 0.67, IQR: 0.68) but Baseline+SCITE had slightly lower variability (IQR) than Phertilizer. Similarly to SNV CPR, Baseline+SCITE (median: 1.0) and SBMClone+SCITE (median: 1.0) outperformed Phertilizer on cell CPR (median: 0.87), but the corresponding decreased performance in cell APR and IPR was indicative of inferring too few cell clusters. In addition to providing more supporting evidence for the validity of our elementary operations, these cell placement performance metrics also highlight the advantage of Phertilizer utilizing both copy number and SNV signal for tree inference. In contrast, Baseline+SCITE prioritizes copy number signal and is unable to further refine a cluster of cells with distinct clonal genotypes but the same copy number profile. Conversely, SBMClone+SCITE ignores copy number signal and struggles to infer clones with sparse SNV signal. Finally, we assessed the genotype similarity and included SBMClone and Baseline as this is obtained prior to tree inference. Since genotype similarity compares the inferred genotype or each simulated cell against its ground truth genotype, it captures the interplay between cell clustering and clonal genotyping. Given that Phertilizer achieved the highest performance on the clonal tree inference and cell clustering metrics, we would expect Phertilizer to have the highest genotype similarity. Indeed, Fig 3c demonstrates that to be true since Phertilizer was the only method to have a median genotype similarity above 0.95. Baseline was the next highest (median: 0.88), closely followed by Baseline+SCITE (median: 0.85), while SBMClone+SCITE (median: 0.55) and SBMClone (median: 0.53) had the worst performance. When evaluating these metrics at the lowest coverage g = 0.01×, Phertilizer maintained top performance on both cell placement and genotype similarity but Baseline+SCITE was competitive with Phertilizer on SNV placement (Fig H in S1 Appendix). For the highest coverage g = 0.1×, Phertilizer achieved the highest accuracy on SNV placement and cell placement and had a median genotype similarity of 0.97 while the next closest competitor (Baseline) had a median similarity of 0.89 (Fig J in S1 Appendix). In terms of running time for the case of n = 1000 cells, m = 15000 SNVs, and a coverage of g = 0.01×, the median running time of Phertilizer was 460 s, 45.9 s for Baseline+SCITE and 101 s for SBMClone+SCITE (Fig M in S1 Appendix). To perform sensitivity analysis, we generated two additional sets of simulations. The first had the same parameters as above but excluded CNAs, such that every locus was heterozygous diploid. We also excluded Baseline+SCITE from comparison as only a single clone was inferred after cell clustering. We found Phertilizer still outperformed SBMClone+SCITE but that for coverage g = 0.01× our performance were slightly worse than simulations with CNAs (Fig K in S1 Appendix), implying CNA features aid inference when sequencing coverage is extremely sparse. For the second, simulations were generated under a Dollo [34] evolutionary model with k = 9, m = 15000, coverage g ∈ {0.01×, 0.05×, 0.01×}. We found that Phertilizer still outperformed Baseline+SCITE and SBMClone+SCITE, having maintained high scores on all performance metrics (Fig L in S1 Appendix) with the exception of cell APR for the lowest coverage. Hyperparameter selection. Phertilizer requires a number of hyperparameters for inference. To assess sensitivity to each of these hyperparameters, we performed an additional simulation study varying the base sequencing error probability α ∈ {0.001*, 0.01}, the maximum number of copies c ∈ {3, 5*, 9}, the detectability threshold t ∈ {3*, 5, 7, 11} in addition to runtime parameters such as the number of restarts ({5, 15*, 30, 60}, maximum number of iterations ({25, 50*, 100, 200}) per elementary tree operation and the quality check upper bound qc ∈{0.025*, 0.05} for cells not harboring the specified set of SNVs—see Section A.4.6 in S1 Appendix for details on the quality check. Hyperparameter choices with * indicate the default Phertilizer values used in the above simulation study. In all simulated instances, we fixed the number of cells at n = 1000, SNVs at m = 15000 and sequencing coverage at g = 0.05×— see Fig N and Fig O in S1 Appendix for these results. Overall, we found that overestimating the base sequencing error α led to a small drop in performance in terms of genotype similarity and SNV and cell accuracy. For the remaining comparisons, we fixed α = 0.001 and observed that increasing the maximum copies c to 9 slightly improved overall performance. This is likely attributed to the greater number of allele-specific copy number states that are considered for c = 9 than c = 5. We also found that performance remained similar for the detectability threshold until increased to t = 11. Due to the low-sequencing coverage, having such a high detectability threshold yields poorly resolved trees with fewer clones than ground truth. However, variation in the quality check threshold did not lead to significant differences in performance. Lastly, we found that 15 restarts with 50 maximum iterations per elementary tree operations was sufficient to maximize performance. However, it may be prudent to increase these values on real data at the cost of increasing runtime. Although this sensitivity analysis provides guidance for the setting of hyperparameter values, we recommend to perform a grid search over hyperparameters, such as α and c, when these values are difficult to estimate in practice. The posterior probability approximated by Phertilizer can be utilized to discriminate between output clonal trees with different parameter settings. In summary, we conclude that given the high level of accuracy obtained by Phertilizer on these performance metrics, not only are the elementary operations successful in isolation but also the posterior probability is useful in discriminating between candidate clonal trees. Additionally, we find that utilizing copy number signal, whenever available, is necessary when working with ultra-low coverage scDNA-seq data but not sufficient for accurate clonal tree reconstruction and/or SNV genotyping. High-grade serous ovarian cancer patient Utilizing a grid search over input parameters(Section B.2.1 in S1 Appendix), we ran Phertilizer on n = 890 DLP+ sequenced cells from three clonally-related cancer cells lines sourced from the same high-grade serous ovarian cancer patient [6]. We used the variant and total read counts A, D for m = 14, 068 SNVs and derived binned read counts R for b = 6, 207 bins (bin width of 500 KB) from data reported by Laks et al. [6]. The average sequencing coverage for these data was 0.25×. Utilizing an approach similar to the Baseline method described above, Laks et al. [6] identified 9 copy number clones (labeled A-I) via dimensionality reduction and density-based clustering and reconciled them in a phylogeny with the copy number clones as leaves (Fig 4a). We annotated inferred trees with mutations in cancer-related genes in the cBioPortal [35, 36] and Cancer Gene Census (CGC) [37] from COSMIC v97 (see Section B.2.2 in S1 Appendix). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Phertilizer improves upon the SNV phylogeny previously inferred by Laks et al. [6]. (a) The clonal tree inferred by Laks et al. [6] with edges labeled by SNV gains and cell numbers shown below leaf nodes. (b) The clonal tree inferred by Phertilizer with edges labeled by SNV gains and cell numbers shown below leaf nodes. Cancer-related genes are labeled next to the SNVs in (a) and (b) with ‘*’ indicating a stop-gain variant. (c) Mapping between the Laks et al. [6] cell placement and Phertilizer cell placement. (d-e) Cell mutational burden (CMB) comparison between cells within (blue) and outside (red) of each clade in the inferred Laks et al. [6] and Phertilizer clonal tree. https://doi.org/10.1371/journal.pcbi.1011544.g004 As shown in Fig 4b, Phertilizer inferred a clonal tree with 13 nodes and 8 clones. We found that Phertilizer’s tree closely aligned with the Laks et al. [6] tree, with both approaches correctly identifying three major clades corresponding to the three distinct cell lines of origin (Fig 4c). Additionally, driver genes TP53, SUGCT, and MYH9 are identified as clonal in both the Laks et.al [6] inferred tree (Fig 4a) and the Phertilizer clonal tree (Fig 4b). Similarly, subclonal SNVs in CHD2, ARID1A, ZHX1, HTR1D, and INSL4 were placed in the corresponding clades of both trees. To further assess the quality of each inferred clade, we developed a performance metric called cell mutational burden (CMB) defined as CMB(i, M) = ∑q∈M 1{aiq > 0}/∑q∈M 1{diq > 0}. In words, CMB(i, M) is the fraction of mapped SNV loci M with mapped variant reads in cell i. For a specified clade j or subtree rooted at node vj, SNVs Mj are the SNVs gained at node vj. For a cell i placed within clade j, we expect CMB(i, Mj) to be high, although the value will depend on copy number. By contrast, for cells placed outside of clade j, we expect CMB(i, Mj) to be low. More details on CMB are provided in Section B.2.3 in S1 Appendix. Fig 4d and 4e depicts the comparison of the distributions of CMB for all clades for cells placed within and outside that clade for Laks et al. [6] and Phertilizer, respectively. For cells placed outside of a specified clade, the reported tree by Laks et al. [6] as well as the tree inferred by Phertilizer have a median CMB of 0 for all clades. This is indicative of high specificity of both methods, i.e., SNVs are not assigned to a clade if there are observations of that SNV outside that clade. However, for cells placed within the clades, we observed greater variability for the Laks et al. [6] inferred clades than Phertilizer. This variability was most pronounced for the leaf nodes, especially C, D, G, F, H and I, where most have a small number of SNVs gained. We further analyzed clusters G and H, where the 25th percentile of the CMB for cells within the clade equals 0, and clusters C and D that had large variability (IQR of 0.74 and 0.51, respectively). The location of clusters G and H in the embedding space suggests overfitting during density-based clustering, with an arbitrary split of a larger cohesive cluster containing G and H (Fig P in S1 Appendix). In comparison, Phertilizer uses both copy number and SNV signal, resulting in the Laks et al. [6]G and H cells being clustered together to node 4 in the inferred Phertilizer clonal tree. Comparing CMB distributions for cells within clades G and H (Fig 4d) with Phertilizer’s inferred clade 4 (Fig 4e), we observed a higher 25th percentile (0.31) for node 4 than for G (0.00) and H (0.00). This resulted in large separation between the CMB distributions of cells within and outside of clade 4 for Phertilizer but not for Laks et al. [6]G and H clades, implying better SNV placement in Phertilizer’s clonal tree. The last major difference of note between the two inferred trees is the clustering of the cells in Laks et al. [6] nodes C and D versus Phertilizer’s nodes 8 and 9. Similarly to nodes G and H, we did not observe a clear separation of C and D in the embedding space (Fig P in S1 Appendix), making it difficult to define clusters of these cells without more rigorous copy number profiling and clustering. However, as we saw in the simulation study, Phertilizer was able to detect further SNV evolution within a set of cells having the same copy number profile. Cells in these clusters would not be split in half in the embedding space but instead should be scattered randomly throughout a single cluster in the embedding space. In addition to having observed cells in nodes 8 and 9 randomly scattered in a cluster in the embedding space (Fig P in S1 Appendix), we also observed a clear separation between the CMB distributions for cells placed within and outside of clades 8 and 9 (Fig 4e). For these clades, we also observed low variability with IQRs of 0.08 and 0.13, respectively, whereas clusters C and D have very high variability in the inferred Laks et al. [6] clonal tree. Overall, both inferred clonal trees are very similar but since Phertilizer simultaneously uses both CNA and SNV information, we obtained a slight improvement in terms of SNV phylogeny inference. Additionally, we note that the small number of SNVs gained at most of the leaf nodes is a direct result of the bottom up approach taken by Laks et al. [6], which performed pseudobulk SNV calling individually on each cell cluster. When SNVs are present in multiple cell clusters but at low prevalence in each cluster, they may not pass filtering of current somatic SNV callers for a cell cluster. This gives the appearance that SNVs are unique to a single clone, when in actuality they are present in multiple clones but have not been called. When sequencing coverage is closer to the 0.01× as opposed to the 0.25× that we have with these data, correctly genotyping a cell cluster becomes more challenging. In contrast, the top-down approach of Phertilizer is better suited to detect SNVs present in multiple cell clusters at low prevalence. Eight triple negative breast tumors We applied Phertilizer to eight triple negative breast tumors sequenced via ACT [7], labeled TN1 to TN8. After dimensionality reduction of normalized and GC-bias corrected binned read counts R, Minussi et al. [7] identified for each tumor two sets of cell clusters with varying granularity, denoted as superclones and subclones. To obtain the input set of SNVs for each patient, we performed SNV calling of a pseudobulk sample of pooled sequenced cells using MuTect2 in tumor-only mode [38]. See Section B.2.5 in S1 Appendix for further details on data processing. Table A in S1 Appendix displays the breakdown of each tumor in terms of the number n of cells, the number m of SNVs and average coverage g, and depicts the number of inferred clones by Phertilizer, Minussi et al. [7] superclones and subclones, as well as the number of inferred clones by SBMClone. Note that these data have a markedly lower coverage (ranging from 0.017× to 0.039×) than the DLP+ data (0.25×). We additionally ran Baseline+SCITE with the cell clusters fixed to the Minussi et al. [7] subclones but all instances except TN3 and TN5 timed out after 10 hours. However, the CMB distribution for the inferred trees for TN3 and TN5 (Fig S in S1 Appendix) provides no evidence in support of these trees. SBMClone only inferred a single clone for all patients, while Phertilizer inferred a tree with more than one clone for 4 out of the 8 tumors (TN1: 6, TN2: 6, TN4: 4 and TN8: 2). These four tumors have the highest average coverage of the eight patients. We will focus our discussion on the clonal trees inferred by Phertilizer for tumors TN1 (Fig 5) and TN2 (Fig T in S1 Appendix). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Phertilizer infers clonal tree for breast cancer tumor TN1. (a) The tree inferred by Phertilizer with numbers of SNVs labeled beside the edges, and numbers of cells labeled beneath the leaves. Cancer-related genes are labeled next to the SNVs (‘*’: stop-gain variant). (b) A mapping between Phertilizer’s cell clusters and the Minussi et al. [7] superclones. (c) The cell mutational burden (CMB) comparison between cells within (blue) and outside of (red) each clade in the inferred clonal tree. https://doi.org/10.1371/journal.pcbi.1011544.g005 For tumor TN1, Phertilizer inferred a branching tree with 11 nodes and 6 clones (Fig 5a). We also identified a subclonal missense SNV in driver gene DICER1 associated with tumorigenesis and poor prognosis [39, 40]. We noted good concordance between the Minussi et al. [7] superclones and the Phertilizer cell clusters, with the exception of 35 cells that appear as outliers in superclone 1 (Fig 5b, Fig R in S1 Appendix). This suggests these cells might fit better in superclone 2 based on SNV signal. In addition, we identified 8745 of the 13934 SNVs as truncal. This large truncal distance and branching structure were in alignment with the truncal distance in the clonal lineage tree inferred by Minussi et al. [7] using bulk whole exome sequencing. We used CMB to assess the performance of SNV and cell placement (Fig 5c). For clades 5 through 10, we observed that the median CMB for cells outside of the clade is 0. Clades 9 and 10 were particularly interesting because the embedding space depicts the occurrence of SNV evolution within Minussi et. al.’s [7] superclone 4 (Fig R in S1 Appendix). For clades 2 through 4, we noted the median CMB for cells outside of the clade was around 0.035 for each of these clades, while clade 1 is the highest at 0.077. Upon further investigation of these 358 SNVs, we found that the median number of mapped reads was 5 when aggregating all cells, making these SNVs especially challenging to place. This drop in performance on the median CMB for cells outside of a clade when compared to the ovarian cancer patient analyzed above is expected due to the drop in sequencing coverage from 0.25× to 0.031×. However, we still observed a large separation between CMB distributions for the cells within the clade and cells outside the clade for all clades. For tumor TN2, we inferred a branching clonal tree with 11 nodes and 6 clones (Fig T in S1 Appendix). Two of Phertilizer’s cell clusters directly agreed with Minussi et al. [7] superclones. However, Phertilizer split the remaining two superclones into four cell clusters (3, 4, 5, 6) using SNV information. We observed low median CMB (0) for cells outside of the clade and a distinct separation between the cells within and outside of the clade distributions, providing evidence for this cell clustering and SNV placement. For tumor TN4, we identified an 8-node tree with five cell clusters, with trends similar to tumors TN1 and TN2 in terms of cell clustering concordance and CMB (Fig U in S1 Appendix). Finally, for tumor TN8 at the lowest coverage 0.021× of these four tumors, Phertilizer only inferred a 3-node branching tree with two cell clusters (Fig V in S1 Appendix). Simulation study Overview. To assess the performance of Phertilizer and compare it with previously proposed methods, we performed a simulation study with known ground truth clonal trees, evaluating the following four questions: (i) How accurate are the inferred clonal trees? (ii) How well is each method able to identify clusters of cells with similar clonal genotypes? (iii) How accurate are the inferred clonal genotypes? (iv) How sensitive is each method to violations of the infinite sites assumption? We designed our simulation study to match the characteristics of current datasets generated from ultra-low coverage scDNA-seq. To achieve this, we generated simulation instances with varying number k ∈ {5, 9} of nodes, number n ∈ {1000, 2000} of sequenced cells and number m ∈ {5000, 10000, 15000} of SNVs with a mean sequencing coverage g ∈ {0.01×, 0.05×, 0.1×}. We replicated each of these combinations 10 times for a total of 360 instances. See Section B.1 in S1 Appendix for details on the simulation instances. We assessed the quality of an inferred solution (T, ϕ, Y) against a ground-truth tree T*, cell clustering ϕ* and clonal genotypes Y* using ancestral pair recall (APR), incomparable pair recall (IPR), and clustered pair recall (CPR) metrics for cells and SNVs [33], as well as genotype similarity. In addition, we computed a single accuracy value (domain: [0, 1]) composed of the weighted average of APR, IPR and CPR—where the weights are proportional to the number of pairs in each class—such that an SNV and cell accuracy of 1 imply that the inferred solution perfectly matches ground truth. The genotype similarity equals 1 minus the normalized Hamming distance between ground truth genotypes and inferred genotypes of cells. We refer to Fig 3a Section B.1.4 in S1 Appendix for examples and more formal definitions. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Phertilizer outperforms Baseline and SBMClone on simulated data. We show aggregated results for n ∈ {1000, 2000} cells, k ∈ {5, 9} clones and coverage g = 0.05×. (a) Example of SNV accuracy, ancestral pair recall (APR), clustered pair recall (CPR), and incomparable pair recall (IPR) metrics. See Section B.1.4 in S1 Appendix for the corresponding example for cell metrics and genotype similarity. (b) Phertilizer outperforms Baseline +SCITE and SBMClone +SCITE in APR and IPR for both SNVs and cells. Although competing methods rank higher in CPR, overall Phertilizer performs the best considering the accuracy. (c) Phertilizer more accurately recovers clonal genotypes than competing methods. https://doi.org/10.1371/journal.pcbi.1011544.g003 We benchmarked against SBMClone [28] because it is the only existing clustering/genotyping method that co-clusters cells and SNVs. We also benchmarked against a commonly adopted ad hoc practice which we refer to as Baseline [6, 7]. In Baseline, cells are first clustered into clones from the read count embedding . Clonal genotypes are obtained by pooling the reads of all cells assigned to clone j and setting yjq = 1 when the ratio of alternate to total reads at each locus q exceeds a threshold of 0.05 (details provided in Section B.1.1 in S1 Appendix). Since there exists no standalone method that performs cell clustering, genotyping and tree inference for ultra-low coverage scDNA-seq, we paired SBMClone and Baseline with SCITE [12]. We attempted to include SCARLET [27] and BiTSC2 [24] in our benchmarking as these scDNA-seq methods incorporate copy number aberrations in their models. However both failed to run on our simulated data. Specifically, SCARLET, which was designed for medium-to-high coverage data, was unable to appropriately handle missing entries in the total read count matrix D. BiTSC2, an MCMC method, was unable to scale to the size of our input datasets—the largest instances considered in the BiTSC2 paper [24] had n = 500 cells and m = 200 SNVs, significantly smaller than our smallest simulation instances with n = 1000 cells and m = 5000 SNVs. For brevity, we focus our discussion on a coverage g = 0.05× and show aggregated results for n ∈ {1000, 2000} cells and k ∈ {5, 9} clones. We report the median for all performance metrics and include the interquartile range (IQR), i.e., the difference between the 75th and 25th percentiles of the data, when appropriate. We note deviations in trends where relevant, and refer to Section B.1.5 in S1 Appendix for remaining results. Results. We first evaluated the accuracy of SNV placement on the inferred tree by assessing APR, CPR, IPR and accuracy for SNVs (Fig 3b, Fig I in S1 Appendix). Overall, Phertilizer achieved the highest SNV accuracy (median: 0.90) in terms of SNV placement of all three methods (Baseline+SCITE median: 0.81, SBMClone+SCITE median: 0.54). For SNV APR, Phertilizer (median: 0.92) outperformed both Baseline+SCITE (median: 0.71) and SBMClone+SCITE (median: 0.38). This implies that our Linear operation reliably partitions SNVs accurately and that our Branching operation performs well at identifying SNVs that should be placed at the parent node. Moving on to SNV IPR, Phertilizer (median: 1.0, IQR: 0.11) outperformed SBMClone+SCITE (median: 0.48, IQR: 0.5) with lower variability than Baseline+SCITE (median: 1.0, IQR: 0.18). Thus, in addition to correctly identifying the SNVs in the parent node, the Branching operation also successfully partitions the SNVs in the children nodes. However, for SNV CPR, Phertilizer performed worse compared to SBMClone+SCITE (median: 0.98) and Baseline+SCITE (median: 0.97) but still maintained good performance (median 0.84). It is important to note that an SNV CPR of 1 is achievable by clustering all SNVs into a single cluster. Thus, the cost of underfitting the data, or grouping the SNVs into a few very large clusters, will be reflected in decreased APR and IPR. Indeed, we observed this to be the case for both Baseline+SCITE, which performed relatively poorly on APR, and SBMClone+SCITE which was the worst performing on both APR and IPR. Next, we evaluated the accuracy of the cell clustering and placement on the inferred tree. We observed similar trends across accuracy, APR, IPR and CPR cell performance metrics to those identified for their SNV counterparts (Fig 3b, Fig I in S1 Appendix). Phertilizer achieved the highest overall cell accuracy (median: 0.82) in comparison to SBMClone+SCITE (median: 0.60) and Baseline+SCITE (median: 0.56). Likewise, Phertilizer (median: 0.84) outperformed all other methods on cell APR (Baseline+SCITE median: 0.0, SBMClone+SCITE median: 0.32). On cell IPR, both Phertilizer (median: 1.0, IQR: 0.2) and Baseline+SCITE (median: 1.0, IQR: 0.13) significantly outperformed SBMClone+SCITE (median: 0.67, IQR: 0.68) but Baseline+SCITE had slightly lower variability (IQR) than Phertilizer. Similarly to SNV CPR, Baseline+SCITE (median: 1.0) and SBMClone+SCITE (median: 1.0) outperformed Phertilizer on cell CPR (median: 0.87), but the corresponding decreased performance in cell APR and IPR was indicative of inferring too few cell clusters. In addition to providing more supporting evidence for the validity of our elementary operations, these cell placement performance metrics also highlight the advantage of Phertilizer utilizing both copy number and SNV signal for tree inference. In contrast, Baseline+SCITE prioritizes copy number signal and is unable to further refine a cluster of cells with distinct clonal genotypes but the same copy number profile. Conversely, SBMClone+SCITE ignores copy number signal and struggles to infer clones with sparse SNV signal. Finally, we assessed the genotype similarity and included SBMClone and Baseline as this is obtained prior to tree inference. Since genotype similarity compares the inferred genotype or each simulated cell against its ground truth genotype, it captures the interplay between cell clustering and clonal genotyping. Given that Phertilizer achieved the highest performance on the clonal tree inference and cell clustering metrics, we would expect Phertilizer to have the highest genotype similarity. Indeed, Fig 3c demonstrates that to be true since Phertilizer was the only method to have a median genotype similarity above 0.95. Baseline was the next highest (median: 0.88), closely followed by Baseline+SCITE (median: 0.85), while SBMClone+SCITE (median: 0.55) and SBMClone (median: 0.53) had the worst performance. When evaluating these metrics at the lowest coverage g = 0.01×, Phertilizer maintained top performance on both cell placement and genotype similarity but Baseline+SCITE was competitive with Phertilizer on SNV placement (Fig H in S1 Appendix). For the highest coverage g = 0.1×, Phertilizer achieved the highest accuracy on SNV placement and cell placement and had a median genotype similarity of 0.97 while the next closest competitor (Baseline) had a median similarity of 0.89 (Fig J in S1 Appendix). In terms of running time for the case of n = 1000 cells, m = 15000 SNVs, and a coverage of g = 0.01×, the median running time of Phertilizer was 460 s, 45.9 s for Baseline+SCITE and 101 s for SBMClone+SCITE (Fig M in S1 Appendix). To perform sensitivity analysis, we generated two additional sets of simulations. The first had the same parameters as above but excluded CNAs, such that every locus was heterozygous diploid. We also excluded Baseline+SCITE from comparison as only a single clone was inferred after cell clustering. We found Phertilizer still outperformed SBMClone+SCITE but that for coverage g = 0.01× our performance were slightly worse than simulations with CNAs (Fig K in S1 Appendix), implying CNA features aid inference when sequencing coverage is extremely sparse. For the second, simulations were generated under a Dollo [34] evolutionary model with k = 9, m = 15000, coverage g ∈ {0.01×, 0.05×, 0.01×}. We found that Phertilizer still outperformed Baseline+SCITE and SBMClone+SCITE, having maintained high scores on all performance metrics (Fig L in S1 Appendix) with the exception of cell APR for the lowest coverage. Hyperparameter selection. Phertilizer requires a number of hyperparameters for inference. To assess sensitivity to each of these hyperparameters, we performed an additional simulation study varying the base sequencing error probability α ∈ {0.001*, 0.01}, the maximum number of copies c ∈ {3, 5*, 9}, the detectability threshold t ∈ {3*, 5, 7, 11} in addition to runtime parameters such as the number of restarts ({5, 15*, 30, 60}, maximum number of iterations ({25, 50*, 100, 200}) per elementary tree operation and the quality check upper bound qc ∈{0.025*, 0.05} for cells not harboring the specified set of SNVs—see Section A.4.6 in S1 Appendix for details on the quality check. Hyperparameter choices with * indicate the default Phertilizer values used in the above simulation study. In all simulated instances, we fixed the number of cells at n = 1000, SNVs at m = 15000 and sequencing coverage at g = 0.05×— see Fig N and Fig O in S1 Appendix for these results. Overall, we found that overestimating the base sequencing error α led to a small drop in performance in terms of genotype similarity and SNV and cell accuracy. For the remaining comparisons, we fixed α = 0.001 and observed that increasing the maximum copies c to 9 slightly improved overall performance. This is likely attributed to the greater number of allele-specific copy number states that are considered for c = 9 than c = 5. We also found that performance remained similar for the detectability threshold until increased to t = 11. Due to the low-sequencing coverage, having such a high detectability threshold yields poorly resolved trees with fewer clones than ground truth. However, variation in the quality check threshold did not lead to significant differences in performance. Lastly, we found that 15 restarts with 50 maximum iterations per elementary tree operations was sufficient to maximize performance. However, it may be prudent to increase these values on real data at the cost of increasing runtime. Although this sensitivity analysis provides guidance for the setting of hyperparameter values, we recommend to perform a grid search over hyperparameters, such as α and c, when these values are difficult to estimate in practice. The posterior probability approximated by Phertilizer can be utilized to discriminate between output clonal trees with different parameter settings. In summary, we conclude that given the high level of accuracy obtained by Phertilizer on these performance metrics, not only are the elementary operations successful in isolation but also the posterior probability is useful in discriminating between candidate clonal trees. Additionally, we find that utilizing copy number signal, whenever available, is necessary when working with ultra-low coverage scDNA-seq data but not sufficient for accurate clonal tree reconstruction and/or SNV genotyping. Overview. To assess the performance of Phertilizer and compare it with previously proposed methods, we performed a simulation study with known ground truth clonal trees, evaluating the following four questions: (i) How accurate are the inferred clonal trees? (ii) How well is each method able to identify clusters of cells with similar clonal genotypes? (iii) How accurate are the inferred clonal genotypes? (iv) How sensitive is each method to violations of the infinite sites assumption? We designed our simulation study to match the characteristics of current datasets generated from ultra-low coverage scDNA-seq. To achieve this, we generated simulation instances with varying number k ∈ {5, 9} of nodes, number n ∈ {1000, 2000} of sequenced cells and number m ∈ {5000, 10000, 15000} of SNVs with a mean sequencing coverage g ∈ {0.01×, 0.05×, 0.1×}. We replicated each of these combinations 10 times for a total of 360 instances. See Section B.1 in S1 Appendix for details on the simulation instances. We assessed the quality of an inferred solution (T, ϕ, Y) against a ground-truth tree T*, cell clustering ϕ* and clonal genotypes Y* using ancestral pair recall (APR), incomparable pair recall (IPR), and clustered pair recall (CPR) metrics for cells and SNVs [33], as well as genotype similarity. In addition, we computed a single accuracy value (domain: [0, 1]) composed of the weighted average of APR, IPR and CPR—where the weights are proportional to the number of pairs in each class—such that an SNV and cell accuracy of 1 imply that the inferred solution perfectly matches ground truth. The genotype similarity equals 1 minus the normalized Hamming distance between ground truth genotypes and inferred genotypes of cells. We refer to Fig 3a Section B.1.4 in S1 Appendix for examples and more formal definitions. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Phertilizer outperforms Baseline and SBMClone on simulated data. We show aggregated results for n ∈ {1000, 2000} cells, k ∈ {5, 9} clones and coverage g = 0.05×. (a) Example of SNV accuracy, ancestral pair recall (APR), clustered pair recall (CPR), and incomparable pair recall (IPR) metrics. See Section B.1.4 in S1 Appendix for the corresponding example for cell metrics and genotype similarity. (b) Phertilizer outperforms Baseline +SCITE and SBMClone +SCITE in APR and IPR for both SNVs and cells. Although competing methods rank higher in CPR, overall Phertilizer performs the best considering the accuracy. (c) Phertilizer more accurately recovers clonal genotypes than competing methods. https://doi.org/10.1371/journal.pcbi.1011544.g003 We benchmarked against SBMClone [28] because it is the only existing clustering/genotyping method that co-clusters cells and SNVs. We also benchmarked against a commonly adopted ad hoc practice which we refer to as Baseline [6, 7]. In Baseline, cells are first clustered into clones from the read count embedding . Clonal genotypes are obtained by pooling the reads of all cells assigned to clone j and setting yjq = 1 when the ratio of alternate to total reads at each locus q exceeds a threshold of 0.05 (details provided in Section B.1.1 in S1 Appendix). Since there exists no standalone method that performs cell clustering, genotyping and tree inference for ultra-low coverage scDNA-seq, we paired SBMClone and Baseline with SCITE [12]. We attempted to include SCARLET [27] and BiTSC2 [24] in our benchmarking as these scDNA-seq methods incorporate copy number aberrations in their models. However both failed to run on our simulated data. Specifically, SCARLET, which was designed for medium-to-high coverage data, was unable to appropriately handle missing entries in the total read count matrix D. BiTSC2, an MCMC method, was unable to scale to the size of our input datasets—the largest instances considered in the BiTSC2 paper [24] had n = 500 cells and m = 200 SNVs, significantly smaller than our smallest simulation instances with n = 1000 cells and m = 5000 SNVs. For brevity, we focus our discussion on a coverage g = 0.05× and show aggregated results for n ∈ {1000, 2000} cells and k ∈ {5, 9} clones. We report the median for all performance metrics and include the interquartile range (IQR), i.e., the difference between the 75th and 25th percentiles of the data, when appropriate. We note deviations in trends where relevant, and refer to Section B.1.5 in S1 Appendix for remaining results. Results. We first evaluated the accuracy of SNV placement on the inferred tree by assessing APR, CPR, IPR and accuracy for SNVs (Fig 3b, Fig I in S1 Appendix). Overall, Phertilizer achieved the highest SNV accuracy (median: 0.90) in terms of SNV placement of all three methods (Baseline+SCITE median: 0.81, SBMClone+SCITE median: 0.54). For SNV APR, Phertilizer (median: 0.92) outperformed both Baseline+SCITE (median: 0.71) and SBMClone+SCITE (median: 0.38). This implies that our Linear operation reliably partitions SNVs accurately and that our Branching operation performs well at identifying SNVs that should be placed at the parent node. Moving on to SNV IPR, Phertilizer (median: 1.0, IQR: 0.11) outperformed SBMClone+SCITE (median: 0.48, IQR: 0.5) with lower variability than Baseline+SCITE (median: 1.0, IQR: 0.18). Thus, in addition to correctly identifying the SNVs in the parent node, the Branching operation also successfully partitions the SNVs in the children nodes. However, for SNV CPR, Phertilizer performed worse compared to SBMClone+SCITE (median: 0.98) and Baseline+SCITE (median: 0.97) but still maintained good performance (median 0.84). It is important to note that an SNV CPR of 1 is achievable by clustering all SNVs into a single cluster. Thus, the cost of underfitting the data, or grouping the SNVs into a few very large clusters, will be reflected in decreased APR and IPR. Indeed, we observed this to be the case for both Baseline+SCITE, which performed relatively poorly on APR, and SBMClone+SCITE which was the worst performing on both APR and IPR. Next, we evaluated the accuracy of the cell clustering and placement on the inferred tree. We observed similar trends across accuracy, APR, IPR and CPR cell performance metrics to those identified for their SNV counterparts (Fig 3b, Fig I in S1 Appendix). Phertilizer achieved the highest overall cell accuracy (median: 0.82) in comparison to SBMClone+SCITE (median: 0.60) and Baseline+SCITE (median: 0.56). Likewise, Phertilizer (median: 0.84) outperformed all other methods on cell APR (Baseline+SCITE median: 0.0, SBMClone+SCITE median: 0.32). On cell IPR, both Phertilizer (median: 1.0, IQR: 0.2) and Baseline+SCITE (median: 1.0, IQR: 0.13) significantly outperformed SBMClone+SCITE (median: 0.67, IQR: 0.68) but Baseline+SCITE had slightly lower variability (IQR) than Phertilizer. Similarly to SNV CPR, Baseline+SCITE (median: 1.0) and SBMClone+SCITE (median: 1.0) outperformed Phertilizer on cell CPR (median: 0.87), but the corresponding decreased performance in cell APR and IPR was indicative of inferring too few cell clusters. In addition to providing more supporting evidence for the validity of our elementary operations, these cell placement performance metrics also highlight the advantage of Phertilizer utilizing both copy number and SNV signal for tree inference. In contrast, Baseline+SCITE prioritizes copy number signal and is unable to further refine a cluster of cells with distinct clonal genotypes but the same copy number profile. Conversely, SBMClone+SCITE ignores copy number signal and struggles to infer clones with sparse SNV signal. Finally, we assessed the genotype similarity and included SBMClone and Baseline as this is obtained prior to tree inference. Since genotype similarity compares the inferred genotype or each simulated cell against its ground truth genotype, it captures the interplay between cell clustering and clonal genotyping. Given that Phertilizer achieved the highest performance on the clonal tree inference and cell clustering metrics, we would expect Phertilizer to have the highest genotype similarity. Indeed, Fig 3c demonstrates that to be true since Phertilizer was the only method to have a median genotype similarity above 0.95. Baseline was the next highest (median: 0.88), closely followed by Baseline+SCITE (median: 0.85), while SBMClone+SCITE (median: 0.55) and SBMClone (median: 0.53) had the worst performance. When evaluating these metrics at the lowest coverage g = 0.01×, Phertilizer maintained top performance on both cell placement and genotype similarity but Baseline+SCITE was competitive with Phertilizer on SNV placement (Fig H in S1 Appendix). For the highest coverage g = 0.1×, Phertilizer achieved the highest accuracy on SNV placement and cell placement and had a median genotype similarity of 0.97 while the next closest competitor (Baseline) had a median similarity of 0.89 (Fig J in S1 Appendix). In terms of running time for the case of n = 1000 cells, m = 15000 SNVs, and a coverage of g = 0.01×, the median running time of Phertilizer was 460 s, 45.9 s for Baseline+SCITE and 101 s for SBMClone+SCITE (Fig M in S1 Appendix). To perform sensitivity analysis, we generated two additional sets of simulations. The first had the same parameters as above but excluded CNAs, such that every locus was heterozygous diploid. We also excluded Baseline+SCITE from comparison as only a single clone was inferred after cell clustering. We found Phertilizer still outperformed SBMClone+SCITE but that for coverage g = 0.01× our performance were slightly worse than simulations with CNAs (Fig K in S1 Appendix), implying CNA features aid inference when sequencing coverage is extremely sparse. For the second, simulations were generated under a Dollo [34] evolutionary model with k = 9, m = 15000, coverage g ∈ {0.01×, 0.05×, 0.01×}. We found that Phertilizer still outperformed Baseline+SCITE and SBMClone+SCITE, having maintained high scores on all performance metrics (Fig L in S1 Appendix) with the exception of cell APR for the lowest coverage. Hyperparameter selection. Phertilizer requires a number of hyperparameters for inference. To assess sensitivity to each of these hyperparameters, we performed an additional simulation study varying the base sequencing error probability α ∈ {0.001*, 0.01}, the maximum number of copies c ∈ {3, 5*, 9}, the detectability threshold t ∈ {3*, 5, 7, 11} in addition to runtime parameters such as the number of restarts ({5, 15*, 30, 60}, maximum number of iterations ({25, 50*, 100, 200}) per elementary tree operation and the quality check upper bound qc ∈{0.025*, 0.05} for cells not harboring the specified set of SNVs—see Section A.4.6 in S1 Appendix for details on the quality check. Hyperparameter choices with * indicate the default Phertilizer values used in the above simulation study. In all simulated instances, we fixed the number of cells at n = 1000, SNVs at m = 15000 and sequencing coverage at g = 0.05×— see Fig N and Fig O in S1 Appendix for these results. Overall, we found that overestimating the base sequencing error α led to a small drop in performance in terms of genotype similarity and SNV and cell accuracy. For the remaining comparisons, we fixed α = 0.001 and observed that increasing the maximum copies c to 9 slightly improved overall performance. This is likely attributed to the greater number of allele-specific copy number states that are considered for c = 9 than c = 5. We also found that performance remained similar for the detectability threshold until increased to t = 11. Due to the low-sequencing coverage, having such a high detectability threshold yields poorly resolved trees with fewer clones than ground truth. However, variation in the quality check threshold did not lead to significant differences in performance. Lastly, we found that 15 restarts with 50 maximum iterations per elementary tree operations was sufficient to maximize performance. However, it may be prudent to increase these values on real data at the cost of increasing runtime. Although this sensitivity analysis provides guidance for the setting of hyperparameter values, we recommend to perform a grid search over hyperparameters, such as α and c, when these values are difficult to estimate in practice. The posterior probability approximated by Phertilizer can be utilized to discriminate between output clonal trees with different parameter settings. In summary, we conclude that given the high level of accuracy obtained by Phertilizer on these performance metrics, not only are the elementary operations successful in isolation but also the posterior probability is useful in discriminating between candidate clonal trees. Additionally, we find that utilizing copy number signal, whenever available, is necessary when working with ultra-low coverage scDNA-seq data but not sufficient for accurate clonal tree reconstruction and/or SNV genotyping. High-grade serous ovarian cancer patient Utilizing a grid search over input parameters(Section B.2.1 in S1 Appendix), we ran Phertilizer on n = 890 DLP+ sequenced cells from three clonally-related cancer cells lines sourced from the same high-grade serous ovarian cancer patient [6]. We used the variant and total read counts A, D for m = 14, 068 SNVs and derived binned read counts R for b = 6, 207 bins (bin width of 500 KB) from data reported by Laks et al. [6]. The average sequencing coverage for these data was 0.25×. Utilizing an approach similar to the Baseline method described above, Laks et al. [6] identified 9 copy number clones (labeled A-I) via dimensionality reduction and density-based clustering and reconciled them in a phylogeny with the copy number clones as leaves (Fig 4a). We annotated inferred trees with mutations in cancer-related genes in the cBioPortal [35, 36] and Cancer Gene Census (CGC) [37] from COSMIC v97 (see Section B.2.2 in S1 Appendix). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Phertilizer improves upon the SNV phylogeny previously inferred by Laks et al. [6]. (a) The clonal tree inferred by Laks et al. [6] with edges labeled by SNV gains and cell numbers shown below leaf nodes. (b) The clonal tree inferred by Phertilizer with edges labeled by SNV gains and cell numbers shown below leaf nodes. Cancer-related genes are labeled next to the SNVs in (a) and (b) with ‘*’ indicating a stop-gain variant. (c) Mapping between the Laks et al. [6] cell placement and Phertilizer cell placement. (d-e) Cell mutational burden (CMB) comparison between cells within (blue) and outside (red) of each clade in the inferred Laks et al. [6] and Phertilizer clonal tree. https://doi.org/10.1371/journal.pcbi.1011544.g004 As shown in Fig 4b, Phertilizer inferred a clonal tree with 13 nodes and 8 clones. We found that Phertilizer’s tree closely aligned with the Laks et al. [6] tree, with both approaches correctly identifying three major clades corresponding to the three distinct cell lines of origin (Fig 4c). Additionally, driver genes TP53, SUGCT, and MYH9 are identified as clonal in both the Laks et.al [6] inferred tree (Fig 4a) and the Phertilizer clonal tree (Fig 4b). Similarly, subclonal SNVs in CHD2, ARID1A, ZHX1, HTR1D, and INSL4 were placed in the corresponding clades of both trees. To further assess the quality of each inferred clade, we developed a performance metric called cell mutational burden (CMB) defined as CMB(i, M) = ∑q∈M 1{aiq > 0}/∑q∈M 1{diq > 0}. In words, CMB(i, M) is the fraction of mapped SNV loci M with mapped variant reads in cell i. For a specified clade j or subtree rooted at node vj, SNVs Mj are the SNVs gained at node vj. For a cell i placed within clade j, we expect CMB(i, Mj) to be high, although the value will depend on copy number. By contrast, for cells placed outside of clade j, we expect CMB(i, Mj) to be low. More details on CMB are provided in Section B.2.3 in S1 Appendix. Fig 4d and 4e depicts the comparison of the distributions of CMB for all clades for cells placed within and outside that clade for Laks et al. [6] and Phertilizer, respectively. For cells placed outside of a specified clade, the reported tree by Laks et al. [6] as well as the tree inferred by Phertilizer have a median CMB of 0 for all clades. This is indicative of high specificity of both methods, i.e., SNVs are not assigned to a clade if there are observations of that SNV outside that clade. However, for cells placed within the clades, we observed greater variability for the Laks et al. [6] inferred clades than Phertilizer. This variability was most pronounced for the leaf nodes, especially C, D, G, F, H and I, where most have a small number of SNVs gained. We further analyzed clusters G and H, where the 25th percentile of the CMB for cells within the clade equals 0, and clusters C and D that had large variability (IQR of 0.74 and 0.51, respectively). The location of clusters G and H in the embedding space suggests overfitting during density-based clustering, with an arbitrary split of a larger cohesive cluster containing G and H (Fig P in S1 Appendix). In comparison, Phertilizer uses both copy number and SNV signal, resulting in the Laks et al. [6]G and H cells being clustered together to node 4 in the inferred Phertilizer clonal tree. Comparing CMB distributions for cells within clades G and H (Fig 4d) with Phertilizer’s inferred clade 4 (Fig 4e), we observed a higher 25th percentile (0.31) for node 4 than for G (0.00) and H (0.00). This resulted in large separation between the CMB distributions of cells within and outside of clade 4 for Phertilizer but not for Laks et al. [6]G and H clades, implying better SNV placement in Phertilizer’s clonal tree. The last major difference of note between the two inferred trees is the clustering of the cells in Laks et al. [6] nodes C and D versus Phertilizer’s nodes 8 and 9. Similarly to nodes G and H, we did not observe a clear separation of C and D in the embedding space (Fig P in S1 Appendix), making it difficult to define clusters of these cells without more rigorous copy number profiling and clustering. However, as we saw in the simulation study, Phertilizer was able to detect further SNV evolution within a set of cells having the same copy number profile. Cells in these clusters would not be split in half in the embedding space but instead should be scattered randomly throughout a single cluster in the embedding space. In addition to having observed cells in nodes 8 and 9 randomly scattered in a cluster in the embedding space (Fig P in S1 Appendix), we also observed a clear separation between the CMB distributions for cells placed within and outside of clades 8 and 9 (Fig 4e). For these clades, we also observed low variability with IQRs of 0.08 and 0.13, respectively, whereas clusters C and D have very high variability in the inferred Laks et al. [6] clonal tree. Overall, both inferred clonal trees are very similar but since Phertilizer simultaneously uses both CNA and SNV information, we obtained a slight improvement in terms of SNV phylogeny inference. Additionally, we note that the small number of SNVs gained at most of the leaf nodes is a direct result of the bottom up approach taken by Laks et al. [6], which performed pseudobulk SNV calling individually on each cell cluster. When SNVs are present in multiple cell clusters but at low prevalence in each cluster, they may not pass filtering of current somatic SNV callers for a cell cluster. This gives the appearance that SNVs are unique to a single clone, when in actuality they are present in multiple clones but have not been called. When sequencing coverage is closer to the 0.01× as opposed to the 0.25× that we have with these data, correctly genotyping a cell cluster becomes more challenging. In contrast, the top-down approach of Phertilizer is better suited to detect SNVs present in multiple cell clusters at low prevalence. Eight triple negative breast tumors We applied Phertilizer to eight triple negative breast tumors sequenced via ACT [7], labeled TN1 to TN8. After dimensionality reduction of normalized and GC-bias corrected binned read counts R, Minussi et al. [7] identified for each tumor two sets of cell clusters with varying granularity, denoted as superclones and subclones. To obtain the input set of SNVs for each patient, we performed SNV calling of a pseudobulk sample of pooled sequenced cells using MuTect2 in tumor-only mode [38]. See Section B.2.5 in S1 Appendix for further details on data processing. Table A in S1 Appendix displays the breakdown of each tumor in terms of the number n of cells, the number m of SNVs and average coverage g, and depicts the number of inferred clones by Phertilizer, Minussi et al. [7] superclones and subclones, as well as the number of inferred clones by SBMClone. Note that these data have a markedly lower coverage (ranging from 0.017× to 0.039×) than the DLP+ data (0.25×). We additionally ran Baseline+SCITE with the cell clusters fixed to the Minussi et al. [7] subclones but all instances except TN3 and TN5 timed out after 10 hours. However, the CMB distribution for the inferred trees for TN3 and TN5 (Fig S in S1 Appendix) provides no evidence in support of these trees. SBMClone only inferred a single clone for all patients, while Phertilizer inferred a tree with more than one clone for 4 out of the 8 tumors (TN1: 6, TN2: 6, TN4: 4 and TN8: 2). These four tumors have the highest average coverage of the eight patients. We will focus our discussion on the clonal trees inferred by Phertilizer for tumors TN1 (Fig 5) and TN2 (Fig T in S1 Appendix). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Phertilizer infers clonal tree for breast cancer tumor TN1. (a) The tree inferred by Phertilizer with numbers of SNVs labeled beside the edges, and numbers of cells labeled beneath the leaves. Cancer-related genes are labeled next to the SNVs (‘*’: stop-gain variant). (b) A mapping between Phertilizer’s cell clusters and the Minussi et al. [7] superclones. (c) The cell mutational burden (CMB) comparison between cells within (blue) and outside of (red) each clade in the inferred clonal tree. https://doi.org/10.1371/journal.pcbi.1011544.g005 For tumor TN1, Phertilizer inferred a branching tree with 11 nodes and 6 clones (Fig 5a). We also identified a subclonal missense SNV in driver gene DICER1 associated with tumorigenesis and poor prognosis [39, 40]. We noted good concordance between the Minussi et al. [7] superclones and the Phertilizer cell clusters, with the exception of 35 cells that appear as outliers in superclone 1 (Fig 5b, Fig R in S1 Appendix). This suggests these cells might fit better in superclone 2 based on SNV signal. In addition, we identified 8745 of the 13934 SNVs as truncal. This large truncal distance and branching structure were in alignment with the truncal distance in the clonal lineage tree inferred by Minussi et al. [7] using bulk whole exome sequencing. We used CMB to assess the performance of SNV and cell placement (Fig 5c). For clades 5 through 10, we observed that the median CMB for cells outside of the clade is 0. Clades 9 and 10 were particularly interesting because the embedding space depicts the occurrence of SNV evolution within Minussi et. al.’s [7] superclone 4 (Fig R in S1 Appendix). For clades 2 through 4, we noted the median CMB for cells outside of the clade was around 0.035 for each of these clades, while clade 1 is the highest at 0.077. Upon further investigation of these 358 SNVs, we found that the median number of mapped reads was 5 when aggregating all cells, making these SNVs especially challenging to place. This drop in performance on the median CMB for cells outside of a clade when compared to the ovarian cancer patient analyzed above is expected due to the drop in sequencing coverage from 0.25× to 0.031×. However, we still observed a large separation between CMB distributions for the cells within the clade and cells outside the clade for all clades. For tumor TN2, we inferred a branching clonal tree with 11 nodes and 6 clones (Fig T in S1 Appendix). Two of Phertilizer’s cell clusters directly agreed with Minussi et al. [7] superclones. However, Phertilizer split the remaining two superclones into four cell clusters (3, 4, 5, 6) using SNV information. We observed low median CMB (0) for cells outside of the clade and a distinct separation between the cells within and outside of the clade distributions, providing evidence for this cell clustering and SNV placement. For tumor TN4, we identified an 8-node tree with five cell clusters, with trends similar to tumors TN1 and TN2 in terms of cell clustering concordance and CMB (Fig U in S1 Appendix). Finally, for tumor TN8 at the lowest coverage 0.021× of these four tumors, Phertilizer only inferred a 3-node branching tree with two cell clusters (Fig V in S1 Appendix). Discussion Ultra-low coverage scDNA-seq has greatly enhanced our ability to study tumor evolution from a copy number perspective [7, 17]. Capitalizing on the strong copy number signal inherent in this data, we proposed a new method, Phertilizer, that grows an SNV phylogeny in a recursive fashion using elementary tree operations. We demonstrated the effectiveness of our approach relative to existing clustering methods on both simulated and real data. Importantly, we found that for the current number of cells (800 − 2000) used in practice, Phertilizer performs markedly better than these methods, yielding more accurate clonal trees, cell clusters, and clonal genotypes. As the first method to reconstruct the evolutionary history of SNVs from ultra-low coverage scDNA-seq, Phertilizer helps advance the study of tumor evolution and makes progress towards the goal of joint SNV and CNA phylogeny inference at single-cell resolution. There are several additional limitations and directions for future research. First, as sequencing coverage drops below 0.02× as in the ACT data, Phertilizer does not infer clonal trees with more than one clone. Although inference is impacted by numerous factors, like copy number profile, it does perhaps suggest a coverage of approximately 0.02× as the limit of detection for Phertilizer. Second, accurate variant calling also remains an open problem for ultra-low scDNA-seq data, making it challenging to identify input subclonal variants. Beyond scDNA-seq data, accurate SNV variant calling from scRNA-seq and ATAC-seq datasets is also challenging [41, 42]. But new methods, such as Monopogen [43], SComatic [44], VarCA [45], scAllele [46], reference-based methods [47] and the use of patient derived cell lines [48] are rapidly improving our ability to accurately conduct single-cell somatic mutational profiling from diverse technologies beyond scDNA-seq. By accounting for different error profiles in variant calling, Phertilizer could be extended to directly model the evolution of somatic variants from scRNA-seq and ATAC-seq datasets. Third, beyond SNVs and CNAs we plan to support structural variants and integrate other omics modalities such as methylation and transcription [49]. Fourth, the infinite sites model used in this work is often violated due to copy-number deletions. While we demonstrated robustness to such violations, a future direction is to use the Dollo evolutionary model [14, 34]. Finally, our model lacks an explicit placement of CNA events on the tree. Tree reconciliation methods, such as PACTION [50], can now be applied to integrate an SNV clonal tree generated by Phertilizer and a CNA tree to obtain a joint tree. Supporting information S1 Appendix. Supplementary materials. https://doi.org/10.1371/journal.pcbi.1011544.s001 (PDF) Acknowledgments We thank the Navin Lab and Darlan Minussi for their assistance with the ACT data. Additionally, we thank the Shah Lab, including Daniel Lai, Robert Reinert, and Andrew McPherson, for their assistance with the DLP+ data.

Triandafillou, Catherine G.;Pan, Rosalind Wenshan;Dinner, Aaron R.;Drummond, D. Allan

doi: 10.1371/journal.pcbi.1011565pmid: 37844070

Introduction In the overarching quest to understand how genotype shapes phenotype, the question of how protein sequence encodes protein function has proved a rich and enduring challenge. An early and still pervasive conceptual framework in which stable sequence-encoded protein structures confer biological function has been met by a newer (yet by now firmly established) companion approach born from the recognition that many functions—binding, selective recruitment, formation of large-scale structures, and more—can be achieved by sequences which do not adopt a stable conformation (intrinsically disordered regions, or IDRs). Although neither approach is exclusive of the other, and indeed they anchor a continuum [1], for historical and methodological reasons many analyses adopt one approach or the other based on various heuristics [2–7]. Such heuristics have had an outsized impact on how sequence-function maps are explored. In one early and influential study of IDRs, Uversky and colleagues discovered that plotting mean net charge against mean hydropathy (hydrophobicity) permits a dividing line to be drawn separating folded from disordered proteins [2,8]. In these analyses, highly charged, weakly hydrophobic sequences have a strong tendency to be disordered. More recently developed heuristics go beyond composition: simulation studies suggest that the degree of mixing of opposite charges within a highly charged, nearly net-neutral (polyampholyte) sequence is a predictor for the biophysical properties of such polypeptides, specifically whether they form expanded or compact structures in solution [3,9]. These studies assume that the sequences in question do not take on well-defined structures, largely based on the observation that many disordered proteins are polyampholytes (~75% of known IDRs have a fraction of charged residues (FCR) > 0.35 [10]). These findings, although based on a few hand-picked sequences from different organisms and proteins, have broadly informed the analysis of many other sequences [11–13]. Other analyses, while still converging on the general finding that disorder is associated most strongly with a bias toward charged residues and away from hydrophobic residues, have emphasized extreme compositional biases themselves as strong predictors of disorder [4,5]. The most common quantitative description of sequence compositional bias is the Shannon entropy, often referred to as complexity [6,14]. Complexity here has a statistical, not biological, interpretation; “simple” sequences such as homopolymers or sequences composed of only a few types of amino acids have low sequence entropy and thus low complexity. Low-complexity regions (LCRs) have in the past decade experienced a surge of attention, driven by the observation that they are associated with mesoscale organization in cells: clusters, granules, hydrogels [15–17], membraneless organelles, and a host of related structures now referred to as biomolecular condensates [18]. Charged LCRs in particular play a crucial role in biomolecular condensation in highly influential model systems, mediating complex coacervation [7] and phase separation [19–22]. Particular sequence features such as enrichment with positively charged residues like arginine and with conformationally flexible glycine, most memorably in the RGG motif [23], appear often in RNA binding proteins that are known to condense; interactions with cationic residues can be modulated by negatively charged regions, leading to the proposal of a molecular grammar for such interactions [24]. While further work is needed to understand when and how these interactions drive condensation, it is also important to develop a better understanding of their biophysical properties, and especially how the regions behave in isolation. Together, these lines of inquiry both reflect and create conditions in which highly charged, low-hydrophobicity LCRs may be studied nearly exclusively through the lens of disorder [3,10,25]. Because of the historical roots of the disorder presumption—particularly that many of the paradigm-shaping observations were made as databases of sequences and structures were in their infancy—the presumption itself has persisted with few challenges. A confluence of trends and events has laid the groundwork for a productive reexamination of these assumptions. First, the maturation of structural and sequence databases has prompted increasingly critical looks at our understanding of LCRs [26] and IDRs [27]. In parallel, specific examples have accumulated of well-defined structure in sequences which would, by existing heuristics, be overwhelmingly predicted to be disordered: alpha-helices in myosin [28] and caldesmon [29], and a coiled-coil region in the mRNA export protein GLE1 [30]. Indeed, there is a long-established connection between charge patterning and helix formation [10–13,31,32]. Finally, new methods now permit more reliable and farther-reaching assessment of the disorder presumption for highly charged regions, notably high-quality structure prediction [33,34]. So motivated, we return to the root issues: to what extent are naturally occurring highly charged protein regions structured versus disordered? What is the empirical relationship between the fraction and patterning of charged residues and the biophysical properties of a region? Are highly charged regions conserved over evolutionary time, as we would expect for biologically important properties? And how easily can one distinguish between structured and disordered regions on the basis of simple sequence heuristics? To answer these questions, we systematically identify highly charged regions proteome-wide in related eukaryotes (budding yeasts) and characterize their sequence properties, predicted structure, and evolution. In contrast to previous studies [3,9,10,35], our work examines the entire proteome, permitting us to quantify the frequency and, with proteome-scale homology, evolutionary conservation of charged regions at the genomic scale. We find that naturally occurring polyampholytes are highly prevalent and, despite being low-complexity with often-poor sequence conservation, these regions are often predicted to form or contain alpha-helices. We confirm these predictions through comparison with experimentally derived structures. These results demonstrate that certain LCRs, even those enriched for charged amino acids thought to be important for intermolecular interactions driving phase separation, may adopt a well-defined structure in the right sequence or physicochemical context. More broadly, we show that it is important to consider structural properties explicitly when evaluating the other properties of an LCR. Results Highly charged regions of low sequence complexity are prevalent in the yeast proteome We first established criteria for regions to be “highly charged” and used them to identify regions in the S. cerevisiae proteome, our departure point owing to its extensive experimental and evolutionary characterization. Examining amino acid usage (Fig 1A), we found that the charged residues (glutamate, aspartate, lysine, and arginine) together constitute 23% of the total amino acids. Unlike all other categories of amino acids, the frequency of each charged amino acid (as determined from the frequency of observed codons) deviates strongly from expectation based on the underlying nucleotide frequency (Fig 1A, light gray points), evidence for evolutionary selection. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Regions high in charge are prevalent in the yeast proteome and are low complexity. a Top: the average frequency of amino acids in the yeast proteome. Light gray trace is the expected frequency based on the average nucleotide frequency and the codons which code for each amino acid. Bottom: The difference between the dark and light traces in the top panel. b Cartoon of the algorithm which detects highly charged regions. c Summary of the per-region fraction of charged residues (FCR) and normalized net charge for identified regions. d Complexity of the charged region calculated according to the compositional entropy defined in [14]. Gray and black distributions are normalized to the yeast proteome entropy, the purple distribution is normalized to the entropy of a charged-enriched hypothetical proteome. Reference sequences are known low complexity domains from Sup35 (a typical prion-like low-complexity protein) and Pab1 (atypical hydrophobic-rich intrinsically disordered region), and a folded sequence (actin). e Normalized hydropathy of the charged regions (purple) and length-matched randomly drawn regions (gray). Reference sequences are the same as those in d. https://doi.org/10.1371/journal.pcbi.1011565.g001 To isolate highly charged regions, we took a sliding-window approach, moving a 12-amino-acid window across all protein sequences and selecting regions with a fraction of charged residues (FCR) above 0.4, with some tolerance for transient deviations (see Fig 1B and Methods). After trimming uncharged ends off these segments, the resulting highly charged regions have a median length of 50 and a FCR ≥ 0.43 (Figs 1C and S1C), more stringent than, for example, a published definition of a strong polyampholyte (FCR > 0.30) [36]. We identified 1,047 regions in 800 proteins; about 14% of protein-coding genes encode at least one highly charged region. The FCR in these regions is just over two standard deviations above that for randomly chosen regions in the proteome (S1A Fig); the regions also have substantially higher charge density than the proteins which contain them (S1B Fig). We examined the distribution of both FCR and normalized net charge across all regions; it is more common for a region to have a net charge close to zero, although there is a significant number of net negatively charged regions (Fig 1C). Examples of neutral regions and those that carry a net charge can be found in Table 1. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Example highly charged regions from the yeast proteome. https://doi.org/10.1371/journal.pcbi.1011565.t001 We expected that the highly charged regions would have lower complexity and hydrophobicity than random regions, because they must be enriched for a small subset of (charged) amino acids, and we confirmed that this is the case (Fig 1D and 1E gray and black traces). Because the highly charged regions could be low complexity merely due to the selection bias imposed by looking for enrichment for a few amino acids, we calculated the complexity normalized by the entropy of a synthetic, mostly charged proteome (50% charged amino acids and all other amino acids equally represented, see Methods for details, Fig 1D purple trace). Even with this correction, the regions are far less complex than randomly drawn regions (P < 10−6, Wilcoxon rank sum test), suggesting a further bias in amino acid usage in these regions beyond that explained by their enrichment for charge. We also examined the distribution of the proteins containing these regions within the cell. We found that they were enriched in the nucleus, and especially in the nucleolus (S1D Fig), consistent with recent findings that across several species nucleolar proteins are enriched for charge-rich low-complexity sequences [37]. In summary, we find that highly charged regions which exceed even stringent definitions of polyampholytes are common in the yeast proteome, are on average less hydrophobic and less complex than average sequences, and are enriched in specific nuclear compartments. Secondary structure is pervasive in highly charged regions Given the historical and intuitive associations between low hydrophobicity/high charge density and disorder, we predicted that the vast majority of the regions we identified would not adopt a well-defined structure. We thus set out to determine what proportion of the regions we identified were IDRs using experimentally derived structures and recently available proteome-wide structure prediction (AlphaFold) [33]. Although the biophysical properties of disordered regions cannot be accurately determined using AlphaFold structures [38], disorder can be inferred in two ways. The first is to impute disorder to residues with a low AlphaFold confidence score [38]; the second is to ascribe disorder to regions with high-confidence coil (e.g., not helix or sheet) predictions. We employ both methods. To validate of these choices, we analyzed the predicted AlphaFold structure of protein regions from the DisProt database (which contains proteins that have been empirically measured to be disordered through a variety of experimental means including circular dichroism and NMR) and found that the vast majority of confidently predicted residues in these regions were scored as “coil” by DSSP (S2A Fig). Thus, by using AlphaFold we were able to assess both structure and disorder—with the same method—proteome-wide. Returning to the highly charged regions, we used the AlphaFold predictions to classify each residue as either disordered (low confidence, or high-confidence and scored as coil) or ordered (high-confidence and scored as helix or sheet; see Methods for a complete description of scoring cutoffs). While a significant number of the highly charged regions were almost completely composed of residues classified as disordered, in many regions (40% of the total) more than half of the residues were predicted to be structured (Fig 2A). We examined the secondary structure classification for all confidently predicted residues (45% of the total) across the entire dataset. The highly charged regions were markedly enriched for alpha-helical secondary structure compared to disordered regions from DisProt and had a similar frequency to length-matched randomly drawn regions (Figs 2B and S2A). There were seven (out of 800) cases where proteins predicted to have high helical content were also found in the DisProt database. We investigated these more deeply and found that in only three cases was there overlap between the region we identified and the region annotated in DisProt. The full results of this investigation are found in S1 Table. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Secondary structure is pervasive in highly charged, low complexity regions. a The distribution of the fraction of a region predicted to be disordered with AlphaFold or present in DisProt. See Methods for secondary structure scoring. b The number of predicted helical, sheet, and disordered (which includes coil; see Methods) residues for the highly charged regions, randomly-selected regions, and experimentally verified disordered regions from DisProt. c (Left) The proportion of the top 200 most structured regions for which the protein that contains them is in the PDB. (Right) For those that are in the PDB, the proportion of regions that is a helix, sheet, or missing. d Uversky plot of the highly charged regions. The dividing line is from [2]. https://doi.org/10.1371/journal.pcbi.1011565.g002 We examined the subset of proteins which have empirically determined structures as a check on AlphaFold’s predictions, to guard against hallucination or other systematic error in highly charged regions. We searched the Protein Data Bank (PDB) for the 200 regions with the highest predicted fraction of structured residues (>92% predicted structured, 19% of the total regions). 27% of the proteins in this subset could be found in the PDB and, of those that were found, 42 (68%) had the highly charged region resolved. In all of these cases but two (3%), the region predicted to be a helix was experimentally determined to be a helix (Fig 2C). That is, where experimental evidence is available for these regions, odds are better than 20:1 that the helical prediction will be confirmed by experimental data. In roughly a third of cases the region is absent from the PDB, and because disordered structures frequently evade structural resolution, it is possible that these regions are disordered under some conditions. In particular, solvent conditions (e.g., pH) and sequence context could modulate the net charge on each amino acid, altering the propensity for structure. Indeed, recent simulations suggest just such a mechanism for a highly-charged LCR in Hero11 [39], and experimental studies have demonstrated that a transient helix forms in the acidic activation domain of Gcn4 upon interaction with a binding partner [40]. Regions may also be modified post-translationally, potentially altering net charge, charge patterning, and localization. Given that these local effects are challenging to predict [41], and in principle could both promote or inhibit structure formation, we consider our estimate a reasonable lower bound on the propensity for helix formation in these regions. Lending further credence to our estimate, recent work comparing AlphaFold predictions to NMR structures has demonstrated that AlphaFold can reasonably and with high confidence predict conditionally folded structures [42]. It is also possible that the helical region is stable within a disordered element, like a pipe on a jump rope [43], or that only a portion of the protein lacking the highly charged region was expressed and characterized. From our analysis, we conclude that there is no evidence to suggest structural predictions are inaccurate for these regions, and we confirm the presence of many highly charged helices in experimental data. The presence of substantial helical structure in these highly charged, low-hydrophobicity regions raised questions as to how these regions would be scored by the metrics which initially established connections between disorder and these sequence features. As a set, these regions contradict the argument that large amounts of uncompensated charge predicts disorder, captured in the popular charge/hydropathy or Uversky plot [2]. We created a Uversky plot of the highly charged regions, and all but three fell above the dividing line into the “natively unfolded” region (Fig 2D). Thus classic methods for determining whether a region is ordered or disordered have virtually no predictive power for regions of this composition—a surprising result given that these regions would appear perfectly suited for such a heuristic. More recently developed metrics have been used to assign biophysical properties to highly charged regions. In particular, connections have been made between the patterning of charges (κ) and the predicted radius of gyration (Rg, a measure of compaction) [3,9]. Rg is used to characterize ensembles of disordered conformations; the implicit argument appears to be that because most IDRs are polyampholytes, analysis of polyampholyte conformations can be carried out productively without considering structured conformations. Yet in specific cases, the very polyampholyte sequences being assigned to various disordered conformational states by computational analysis due to their charge patterning [3] are known to be helical—such as the (EEEKKK)n and (EEEEKKKK)n polymers [32,41,44]. Remarkably, this is more than a mere conceptual curiosity: our set of highly charged sequences in budding yeast contains the sequence KKKKEEEEKKKKEEEEKKKKEEEEKKKKEEEEKKKKEEEEKKKKEEEEKKKQEEEEKKKKEEEEKKKQ in the protein Mnn4, a region which is, with modifications, conserved in other fungal species (S2C Fig). Such sequences and their relatively well-studied biophysical behavior offer additional evidence of the importance of considering helical structure in biologically relevant highly charged regions. To emphasize this point, we calculated the FCR and κ values for all the regions in our dataset, and compared how the helical and disordered regions fell in that space. Although the helical regions on average had lower κ values than the disordered regions, the disordered regions spanned the entire κ space; thus it is important to establish the structural or predicted structural state of a sequence before interpreting how the κ value relates to the radius of gyration (S2B Fig). Many of the sequences we identify show the hallmarks of so-called single alpha helices (SAH), helices of length 25–200 residues frequently, though not exclusively, formed by (E4K4)n repeats [9,34,35]. Below, we establish the evolutionary conservation of these regions, suggesting their structure, as well as high charge density, are likely to confer a fitness benefit. An ancient translation initiation factor contains a conserved highly charged helix with sequence properties similar to an IDR To more deeply investigate the sequence and structural properties of highly charged regions, we focused on a specific example where a region predicted to be a helix by AlphaFold had a solved empirical structure for comparison. We chose the broadly conserved eukaryotic translation initiation factor eIF3A (Rpg1 in S. cerevisiae) in which we identified several highly charged regions. One such region was predicted to be almost entirely helical, which is confirmed in the cryo-electron microscopy (cryoEM) structure (Fig 3B) [45]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. eIF3A, an essential eukaryotic translation initiation factor, contains a conserved, highly charged helix that varies in length but not in secondary structure. a Alignment of the eIF3A highly charged region (orthologs from all distantly-related species with predicted proteomes in AlphaFold) with negatively charged residues colored red, positively charged residues colored blue, and gaps and all other amino acids in white. Although the highly charged nature of the region is conserved, the sequence itself is variable. b Representative image of the cryoEM structure of yeast eIF3A [42] with the highly charged region (resolved as a helix) shown in purple, and a reference helix shown in black. c Alignment of eIF3A (same as in a) colored by the secondary structure predicted by AlphaFold; S. cerevisiae sequence is highlighted in red. Note that the highly charged region is predicted to be helical in every species represented. d Despite strong secondary structure conservation, the length of the highly charged helix varies significantly more than a reference helix from the same protein. https://doi.org/10.1371/journal.pcbi.1011565.g003 When we created a sequence alignment of all the homologous proteins for which a structure had been predicted from AlphaFold, we found significant variation in both the length and the sequence of the region (Fig 3A). Such variability is typical in disordered low-complexity regions and seen as the accumulation of many insertions and deletions in a multiple sequence alignment, but we were surprised to see this variation because the yeast version of this sequence was structured. To determine whether the homologous sequences were likely to be structured as well, we used DSSP (implemented in the MDTraj package for Python [46]) to classify the secondary structure predicted by AlphaFold and projected the predictions into alignment space (Fig 3C). Despite the lack of conservation at the sequence level, the helical nature of the region was conserved across all the homologs. However, the length of the helix varied significantly: the coefficient of variation of the highly charged helix was 0.17, compared to 0.11 for a reference helix located elsewhere in the same protein (Fig 3D). This analysis demonstrates that a region with all the features typically associated with IDRs (high length variation as indicated by gaps in the alignment, poor sequence conservation, low complexity, low hydrophobicity) can be associated with a charged region that is in fact structured and retains this structure across evolutionary time. Helical highly charged regions can be predicted from amino acid composition Given the prevalence of structure in the highly charged regions that we detected, and the failure of existing composition-based heuristics to discriminate between the two categories (disordered and helical), one might guess that more sophisticated and general methods might be needed. An alternative possibility is that alternative simple heuristics exist, a possibility which would be demonstrated by producing such a heuristic. We therefore asked whether we could build a simple predictor of helical structure in highly charged regions from composition alone. We stress that our objective here is to demonstrate a principle, and to gain some insight into the factors that determine disorder versus structure, rather than to compete with the many examples of sophisticated software designed to broadly predict disorder [33,34,47,48]. Using the proteome-wide predictions of structure from AlphaFold, we created a dataset of regions which were predicted to be either completely disordered or completely helical (13,437 sequences from 63.6% of the proteome). On a Uversky plot, the helices and IDRs drawn from the yeast proteome, like the highly charged regions, overlap significantly (Fig 4A). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Helical regions can be predicted from composition. a Uversky plot of all regions used to train the LR model. The marginals of the distribution are shown on the plot border. b The coefficients of the logistic regression (LR) model which predicts whether a region is helical or disordered on the basis of amino acid composition. The model was trained on purely helical and disordered regions (predicted by AlphaFold) selected from the S. cerevisiae proteome. Amino acids with a positive coefficient are correlated with helices, those with a negative value are correlated with disordered regions. c The helix propensity from [46] plotted against the LR model coefficients. d Accuracy of both the LR model (top), the Uversky dividing line (middle, from [2]), and flDPnn [47] (bottom) on purely helical and disordered regions (held out data from the training set, n = 3360; left), randomly-drawn regions, which are predicted by AlphaFold to be majority (but not completely) helical or disordered (n = 3405; center), and the highly charged regions (n = 681; right). e Summarized accuracy for all categories in d. f Summarized accuracy of the LR model with only a subset of coefficients, g (Left) Accuracy of the LR model prediction of regions from other organisms. (Right) Timetree showing the evolutionary divergence of the organisms. https://doi.org/10.1371/journal.pcbi.1011565.g004 Using this dataset, we built a logistic regression model which classified regions as helical or disordered on the basis of their amino acid composition (see Methods for model details). The coefficients of this logistic regression (LR) model are shown in Fig 4B; as expected, residues known to affect helical character such as proline and glycine have a large regression coefficient, indicating that their presence is highly predictive. More generally, the coefficients from the model are inversely correlated with the individual amino acid helix propensity [49] (Fig 4C). We assessed the accuracy of the LR model in several ways. First, we calculated the rate of true and false positives and negatives (Fig 4D) for several classes of sequences. The LR model performed extremely well, correctly identifying both helical and disordered regions in the testing data (25% held out from the original dataset) with an accuracy of 92.5% (Fig 4E). We also assessed its performance on a new set of randomly selected regions which were predicted by AlphaFold to contain both helical and disordered character; the LR model predicted the dominant structural feature from composition alone 86.9% of the time (Fig 4E). Finally, we predicted the highly charged regions, where the LR model performed with an overall accuracy of 90.8% (Fig 4E). Most of this accuracy can be captured using only the top five coefficients of the model (Fig 4F). We also scored a dataset of PDB structures with secondary structure annotation (see Methods); the model performed with an accuracy of ~90% on these experimentally determined structures (S3C Fig). To see whether there were systematic differences in the relationship between amino acid composition and secondary structure when using real versus predicted structures, we also created a second LR model trained on purely helical or disordered sequences from the PDB-derived dataset and compared it to the original LR model (S3A and S3B Fig). The coefficients of the two models are highly correlated, with the interesting exceptions of the two helix-breakers P (which has the same order of importance in both models but a much larger magnitude coefficient in the PDB LR model) and G (which has a much higher relative importance in the PDB LR model). The PDB LR model performed with an accuracy exceeding 90%, a result fully independent of AlphaFold predictions and as such an important robustness check. To put our results in context and understand the breakdown of existing heuristics, we compared the accuracy of the LR model to the accuracy of the simple charge/hydropathy (Uversky) model, as well as a state-of-the-art deep-learning based disorder-predictor called flDPnn [47]. As expected, the Uversky model performed better than chance but worse than the LR model on the same sets of randomly drawn regions, but was completely non-predictive for highly charged regions (Fig 4E). This reinforces the idea that normalized net charge is not predictive of disorder (see marginal distributions in the right hand side of Fig 4C), at least for regions with this length distribution. In sum, virtually all the predictive power in the Uversky charge/hydropathy heuristic comes from hydropathy. Although the flDPnn predictions were significantly better than the Uversky prediction for the highly charged regions, they did not exceed the performance of the LR model. Finally, we were curious whether our model was specific to amino acid usage in yeast, or if it could be extended to other proteomes. Using three other proteomes for which structures have been predicted, Schizosaccharomyces pombe, Caenorhabditis elegans, and Homo sapiens, we performed the same procedure of random region selection, labeling using the AlphaFold predictions and confidences, and classification with the LR model trained on AlphaFold predictions of yeast regions. The prediction accuracy was nearly identical to or slightly higher than the S. cerevisiae accuracy (Fig 4G). This simple model based only on the composition of a region is sufficient to predict helical or disordered character in proteomes that diverged over a billion years ago. Highly charged regions are evolutionarily conserved The unique evolutionary signatures suggested by our analysis of eIF3A, coupled with the consistency in predictive power of the LR model across vast evolutionary distances, led us to broader questions about the conservation of the regions we had identified. To what extent do highly charged regions retain their sequence properties and structure as organisms evolve? To address this question, we turned to AYbRAH, a curated database of protein homologs and paralogs in 33 fungal species spanning 600 million years of evolution [50]. This dataset combines automated homology detection and manual curation to achieve high-confidence predictions of highly diverged orthologs. First, we quantified the sequence conservation of the regions in question by examining their alignments and calculating both the frequency of gaps and the sequence divergence (average position-wise entropy). Sequences with high insertion and deletion rates, a known feature of IDRs, will have higher alignment gap frequencies. Those with a high point mutation rate will have high divergence. A low value of both metrics indicates sequence conservation. We calculated these values for all the highly charged regions, and compared them to length-matched, randomly drawn regions from the rest of the same proteins. We found that the charged regions have significantly more gaps (P<0.001, Mann-Whitney U test) and are more divergent (P<0.001, Mann-Whitney U test) than the proteins in which they are found (Fig 5A). We compared these distributions to the same values calculated for experimentally verified IDRs from DisProt, and found that although the charged regions have similar gap frequency to IDRs, they are even more divergent at the sequence level. Thus simply viewing an alignment of these LCRs, without using a secondary structure prediction algorithm, one might conclude that they are disordered. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Highly charged regions are evolutionarily conserved. a The distribution of gap frequencies (left) and average position-wise entropy (right) summarizing multiple sequence alignments (MSAs) of the highly charged regions (purple), randomly drawn regions from the rest of the proteins that contain them (black), and IDRs from DisProt (gray). b Summary of the variance (in log odds space) of usage of categories of amino acids for all proteomes in the AYbRAH database. c FCR values, averaged across AYbRAH alignments, for the highly charged regions identified in the S. cerevisiae proteome and length-matched randomly-drawn regions and their associated AYbRAH MSA. d The average compositional conservation of regions enriched for all sets of four amino acids with the same total frequency as the charged amino acids, plotted as a CDF. Higher values indicate less drift, and lower values indicate more drift (regression to the proteome average). https://doi.org/10.1371/journal.pcbi.1011565.g005 Despite this apparent lack of conservation, we were curious whether any aspects beyond sequence of the highly charged regions were conserved. These regions were identified because of their unique sequence composition, so we tested the degree to which they retained this composition over evolutionary time. To first determine the expected compositional variation, we measured the variation in the total proportion of each amino acid across the species represented in the AYbRAH database (S4A Fig). We found that as a group, the charged amino acids had very little variation in proportion of usage (Fig 5B). Consistent with selection, a high fraction of charged residues was preserved across species and substantially differed from randomly drawn regions in the same species (Fig 5C). The distribution in Fig 5C contains some regions that on average fall below the threshold of 0.4 FCR that we established for the original search in the yeast proteome; this is not surprising given that all sequences are subject to drift, which pulls them towards the proteome average for any given trait unless selection intervenes. Therefore, we created a method to quantify drift in charged regions relative to other compositionally extreme regions. We first identified regions in the yeast proteome enriched for all groups of four amino acids with a combined frequency within +/–0.01 of the combined frequency of the charged amino acids (0.233). For each of the 209 datasets, we calculated the mean proportion of the amino acids in question for each identified region across the AYbRAH alignment (note that FCR is a special case of this property where the four amino acids in question are glutamate, aspartate, lysine, and arginine) If the composition (enrichment of the four amino acids in question) is conserved, we should expect that the mean enrichment score across regions and alignments should be close to the mean of the original enriched dataset (close to or higher than the 0.4 threshold). In contrast, if the property is not conserved, we should expect this enrichment score to be close to the proteome average. To compare directly between datasets, we scaled this enrichment score to a unit scale between an effective 0 (the proteome average), and 1 (the median of the enriched regions detected from the S. cerevisiae proteome). Sets of four with a conservation score close to 1 are highly conserved for that property, while those that are close to 0 have experienced high levels of drift, indicating that they are not conserved. We find that the set of four charged amino acids falls within the top 5% of these scores (Fig 5D). From this analysis we conclude that in the highly charged regions, the charge density is extremely well-conserved. Highly charged regions of low sequence complexity are prevalent in the yeast proteome We first established criteria for regions to be “highly charged” and used them to identify regions in the S. cerevisiae proteome, our departure point owing to its extensive experimental and evolutionary characterization. Examining amino acid usage (Fig 1A), we found that the charged residues (glutamate, aspartate, lysine, and arginine) together constitute 23% of the total amino acids. Unlike all other categories of amino acids, the frequency of each charged amino acid (as determined from the frequency of observed codons) deviates strongly from expectation based on the underlying nucleotide frequency (Fig 1A, light gray points), evidence for evolutionary selection. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Regions high in charge are prevalent in the yeast proteome and are low complexity. a Top: the average frequency of amino acids in the yeast proteome. Light gray trace is the expected frequency based on the average nucleotide frequency and the codons which code for each amino acid. Bottom: The difference between the dark and light traces in the top panel. b Cartoon of the algorithm which detects highly charged regions. c Summary of the per-region fraction of charged residues (FCR) and normalized net charge for identified regions. d Complexity of the charged region calculated according to the compositional entropy defined in [14]. Gray and black distributions are normalized to the yeast proteome entropy, the purple distribution is normalized to the entropy of a charged-enriched hypothetical proteome. Reference sequences are known low complexity domains from Sup35 (a typical prion-like low-complexity protein) and Pab1 (atypical hydrophobic-rich intrinsically disordered region), and a folded sequence (actin). e Normalized hydropathy of the charged regions (purple) and length-matched randomly drawn regions (gray). Reference sequences are the same as those in d. https://doi.org/10.1371/journal.pcbi.1011565.g001 To isolate highly charged regions, we took a sliding-window approach, moving a 12-amino-acid window across all protein sequences and selecting regions with a fraction of charged residues (FCR) above 0.4, with some tolerance for transient deviations (see Fig 1B and Methods). After trimming uncharged ends off these segments, the resulting highly charged regions have a median length of 50 and a FCR ≥ 0.43 (Figs 1C and S1C), more stringent than, for example, a published definition of a strong polyampholyte (FCR > 0.30) [36]. We identified 1,047 regions in 800 proteins; about 14% of protein-coding genes encode at least one highly charged region. The FCR in these regions is just over two standard deviations above that for randomly chosen regions in the proteome (S1A Fig); the regions also have substantially higher charge density than the proteins which contain them (S1B Fig). We examined the distribution of both FCR and normalized net charge across all regions; it is more common for a region to have a net charge close to zero, although there is a significant number of net negatively charged regions (Fig 1C). Examples of neutral regions and those that carry a net charge can be found in Table 1. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Example highly charged regions from the yeast proteome. https://doi.org/10.1371/journal.pcbi.1011565.t001 We expected that the highly charged regions would have lower complexity and hydrophobicity than random regions, because they must be enriched for a small subset of (charged) amino acids, and we confirmed that this is the case (Fig 1D and 1E gray and black traces). Because the highly charged regions could be low complexity merely due to the selection bias imposed by looking for enrichment for a few amino acids, we calculated the complexity normalized by the entropy of a synthetic, mostly charged proteome (50% charged amino acids and all other amino acids equally represented, see Methods for details, Fig 1D purple trace). Even with this correction, the regions are far less complex than randomly drawn regions (P < 10−6, Wilcoxon rank sum test), suggesting a further bias in amino acid usage in these regions beyond that explained by their enrichment for charge. We also examined the distribution of the proteins containing these regions within the cell. We found that they were enriched in the nucleus, and especially in the nucleolus (S1D Fig), consistent with recent findings that across several species nucleolar proteins are enriched for charge-rich low-complexity sequences [37]. In summary, we find that highly charged regions which exceed even stringent definitions of polyampholytes are common in the yeast proteome, are on average less hydrophobic and less complex than average sequences, and are enriched in specific nuclear compartments. Secondary structure is pervasive in highly charged regions Given the historical and intuitive associations between low hydrophobicity/high charge density and disorder, we predicted that the vast majority of the regions we identified would not adopt a well-defined structure. We thus set out to determine what proportion of the regions we identified were IDRs using experimentally derived structures and recently available proteome-wide structure prediction (AlphaFold) [33]. Although the biophysical properties of disordered regions cannot be accurately determined using AlphaFold structures [38], disorder can be inferred in two ways. The first is to impute disorder to residues with a low AlphaFold confidence score [38]; the second is to ascribe disorder to regions with high-confidence coil (e.g., not helix or sheet) predictions. We employ both methods. To validate of these choices, we analyzed the predicted AlphaFold structure of protein regions from the DisProt database (which contains proteins that have been empirically measured to be disordered through a variety of experimental means including circular dichroism and NMR) and found that the vast majority of confidently predicted residues in these regions were scored as “coil” by DSSP (S2A Fig). Thus, by using AlphaFold we were able to assess both structure and disorder—with the same method—proteome-wide. Returning to the highly charged regions, we used the AlphaFold predictions to classify each residue as either disordered (low confidence, or high-confidence and scored as coil) or ordered (high-confidence and scored as helix or sheet; see Methods for a complete description of scoring cutoffs). While a significant number of the highly charged regions were almost completely composed of residues classified as disordered, in many regions (40% of the total) more than half of the residues were predicted to be structured (Fig 2A). We examined the secondary structure classification for all confidently predicted residues (45% of the total) across the entire dataset. The highly charged regions were markedly enriched for alpha-helical secondary structure compared to disordered regions from DisProt and had a similar frequency to length-matched randomly drawn regions (Figs 2B and S2A). There were seven (out of 800) cases where proteins predicted to have high helical content were also found in the DisProt database. We investigated these more deeply and found that in only three cases was there overlap between the region we identified and the region annotated in DisProt. The full results of this investigation are found in S1 Table. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Secondary structure is pervasive in highly charged, low complexity regions. a The distribution of the fraction of a region predicted to be disordered with AlphaFold or present in DisProt. See Methods for secondary structure scoring. b The number of predicted helical, sheet, and disordered (which includes coil; see Methods) residues for the highly charged regions, randomly-selected regions, and experimentally verified disordered regions from DisProt. c (Left) The proportion of the top 200 most structured regions for which the protein that contains them is in the PDB. (Right) For those that are in the PDB, the proportion of regions that is a helix, sheet, or missing. d Uversky plot of the highly charged regions. The dividing line is from [2]. https://doi.org/10.1371/journal.pcbi.1011565.g002 We examined the subset of proteins which have empirically determined structures as a check on AlphaFold’s predictions, to guard against hallucination or other systematic error in highly charged regions. We searched the Protein Data Bank (PDB) for the 200 regions with the highest predicted fraction of structured residues (>92% predicted structured, 19% of the total regions). 27% of the proteins in this subset could be found in the PDB and, of those that were found, 42 (68%) had the highly charged region resolved. In all of these cases but two (3%), the region predicted to be a helix was experimentally determined to be a helix (Fig 2C). That is, where experimental evidence is available for these regions, odds are better than 20:1 that the helical prediction will be confirmed by experimental data. In roughly a third of cases the region is absent from the PDB, and because disordered structures frequently evade structural resolution, it is possible that these regions are disordered under some conditions. In particular, solvent conditions (e.g., pH) and sequence context could modulate the net charge on each amino acid, altering the propensity for structure. Indeed, recent simulations suggest just such a mechanism for a highly-charged LCR in Hero11 [39], and experimental studies have demonstrated that a transient helix forms in the acidic activation domain of Gcn4 upon interaction with a binding partner [40]. Regions may also be modified post-translationally, potentially altering net charge, charge patterning, and localization. Given that these local effects are challenging to predict [41], and in principle could both promote or inhibit structure formation, we consider our estimate a reasonable lower bound on the propensity for helix formation in these regions. Lending further credence to our estimate, recent work comparing AlphaFold predictions to NMR structures has demonstrated that AlphaFold can reasonably and with high confidence predict conditionally folded structures [42]. It is also possible that the helical region is stable within a disordered element, like a pipe on a jump rope [43], or that only a portion of the protein lacking the highly charged region was expressed and characterized. From our analysis, we conclude that there is no evidence to suggest structural predictions are inaccurate for these regions, and we confirm the presence of many highly charged helices in experimental data. The presence of substantial helical structure in these highly charged, low-hydrophobicity regions raised questions as to how these regions would be scored by the metrics which initially established connections between disorder and these sequence features. As a set, these regions contradict the argument that large amounts of uncompensated charge predicts disorder, captured in the popular charge/hydropathy or Uversky plot [2]. We created a Uversky plot of the highly charged regions, and all but three fell above the dividing line into the “natively unfolded” region (Fig 2D). Thus classic methods for determining whether a region is ordered or disordered have virtually no predictive power for regions of this composition—a surprising result given that these regions would appear perfectly suited for such a heuristic. More recently developed metrics have been used to assign biophysical properties to highly charged regions. In particular, connections have been made between the patterning of charges (κ) and the predicted radius of gyration (Rg, a measure of compaction) [3,9]. Rg is used to characterize ensembles of disordered conformations; the implicit argument appears to be that because most IDRs are polyampholytes, analysis of polyampholyte conformations can be carried out productively without considering structured conformations. Yet in specific cases, the very polyampholyte sequences being assigned to various disordered conformational states by computational analysis due to their charge patterning [3] are known to be helical—such as the (EEEKKK)n and (EEEEKKKK)n polymers [32,41,44]. Remarkably, this is more than a mere conceptual curiosity: our set of highly charged sequences in budding yeast contains the sequence KKKKEEEEKKKKEEEEKKKKEEEEKKKKEEEEKKKKEEEEKKKKEEEEKKKQEEEEKKKKEEEEKKKQ in the protein Mnn4, a region which is, with modifications, conserved in other fungal species (S2C Fig). Such sequences and their relatively well-studied biophysical behavior offer additional evidence of the importance of considering helical structure in biologically relevant highly charged regions. To emphasize this point, we calculated the FCR and κ values for all the regions in our dataset, and compared how the helical and disordered regions fell in that space. Although the helical regions on average had lower κ values than the disordered regions, the disordered regions spanned the entire κ space; thus it is important to establish the structural or predicted structural state of a sequence before interpreting how the κ value relates to the radius of gyration (S2B Fig). Many of the sequences we identify show the hallmarks of so-called single alpha helices (SAH), helices of length 25–200 residues frequently, though not exclusively, formed by (E4K4)n repeats [9,34,35]. Below, we establish the evolutionary conservation of these regions, suggesting their structure, as well as high charge density, are likely to confer a fitness benefit. An ancient translation initiation factor contains a conserved highly charged helix with sequence properties similar to an IDR To more deeply investigate the sequence and structural properties of highly charged regions, we focused on a specific example where a region predicted to be a helix by AlphaFold had a solved empirical structure for comparison. We chose the broadly conserved eukaryotic translation initiation factor eIF3A (Rpg1 in S. cerevisiae) in which we identified several highly charged regions. One such region was predicted to be almost entirely helical, which is confirmed in the cryo-electron microscopy (cryoEM) structure (Fig 3B) [45]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. eIF3A, an essential eukaryotic translation initiation factor, contains a conserved, highly charged helix that varies in length but not in secondary structure. a Alignment of the eIF3A highly charged region (orthologs from all distantly-related species with predicted proteomes in AlphaFold) with negatively charged residues colored red, positively charged residues colored blue, and gaps and all other amino acids in white. Although the highly charged nature of the region is conserved, the sequence itself is variable. b Representative image of the cryoEM structure of yeast eIF3A [42] with the highly charged region (resolved as a helix) shown in purple, and a reference helix shown in black. c Alignment of eIF3A (same as in a) colored by the secondary structure predicted by AlphaFold; S. cerevisiae sequence is highlighted in red. Note that the highly charged region is predicted to be helical in every species represented. d Despite strong secondary structure conservation, the length of the highly charged helix varies significantly more than a reference helix from the same protein. https://doi.org/10.1371/journal.pcbi.1011565.g003 When we created a sequence alignment of all the homologous proteins for which a structure had been predicted from AlphaFold, we found significant variation in both the length and the sequence of the region (Fig 3A). Such variability is typical in disordered low-complexity regions and seen as the accumulation of many insertions and deletions in a multiple sequence alignment, but we were surprised to see this variation because the yeast version of this sequence was structured. To determine whether the homologous sequences were likely to be structured as well, we used DSSP (implemented in the MDTraj package for Python [46]) to classify the secondary structure predicted by AlphaFold and projected the predictions into alignment space (Fig 3C). Despite the lack of conservation at the sequence level, the helical nature of the region was conserved across all the homologs. However, the length of the helix varied significantly: the coefficient of variation of the highly charged helix was 0.17, compared to 0.11 for a reference helix located elsewhere in the same protein (Fig 3D). This analysis demonstrates that a region with all the features typically associated with IDRs (high length variation as indicated by gaps in the alignment, poor sequence conservation, low complexity, low hydrophobicity) can be associated with a charged region that is in fact structured and retains this structure across evolutionary time. Helical highly charged regions can be predicted from amino acid composition Given the prevalence of structure in the highly charged regions that we detected, and the failure of existing composition-based heuristics to discriminate between the two categories (disordered and helical), one might guess that more sophisticated and general methods might be needed. An alternative possibility is that alternative simple heuristics exist, a possibility which would be demonstrated by producing such a heuristic. We therefore asked whether we could build a simple predictor of helical structure in highly charged regions from composition alone. We stress that our objective here is to demonstrate a principle, and to gain some insight into the factors that determine disorder versus structure, rather than to compete with the many examples of sophisticated software designed to broadly predict disorder [33,34,47,48]. Using the proteome-wide predictions of structure from AlphaFold, we created a dataset of regions which were predicted to be either completely disordered or completely helical (13,437 sequences from 63.6% of the proteome). On a Uversky plot, the helices and IDRs drawn from the yeast proteome, like the highly charged regions, overlap significantly (Fig 4A). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Helical regions can be predicted from composition. a Uversky plot of all regions used to train the LR model. The marginals of the distribution are shown on the plot border. b The coefficients of the logistic regression (LR) model which predicts whether a region is helical or disordered on the basis of amino acid composition. The model was trained on purely helical and disordered regions (predicted by AlphaFold) selected from the S. cerevisiae proteome. Amino acids with a positive coefficient are correlated with helices, those with a negative value are correlated with disordered regions. c The helix propensity from [46] plotted against the LR model coefficients. d Accuracy of both the LR model (top), the Uversky dividing line (middle, from [2]), and flDPnn [47] (bottom) on purely helical and disordered regions (held out data from the training set, n = 3360; left), randomly-drawn regions, which are predicted by AlphaFold to be majority (but not completely) helical or disordered (n = 3405; center), and the highly charged regions (n = 681; right). e Summarized accuracy for all categories in d. f Summarized accuracy of the LR model with only a subset of coefficients, g (Left) Accuracy of the LR model prediction of regions from other organisms. (Right) Timetree showing the evolutionary divergence of the organisms. https://doi.org/10.1371/journal.pcbi.1011565.g004 Using this dataset, we built a logistic regression model which classified regions as helical or disordered on the basis of their amino acid composition (see Methods for model details). The coefficients of this logistic regression (LR) model are shown in Fig 4B; as expected, residues known to affect helical character such as proline and glycine have a large regression coefficient, indicating that their presence is highly predictive. More generally, the coefficients from the model are inversely correlated with the individual amino acid helix propensity [49] (Fig 4C). We assessed the accuracy of the LR model in several ways. First, we calculated the rate of true and false positives and negatives (Fig 4D) for several classes of sequences. The LR model performed extremely well, correctly identifying both helical and disordered regions in the testing data (25% held out from the original dataset) with an accuracy of 92.5% (Fig 4E). We also assessed its performance on a new set of randomly selected regions which were predicted by AlphaFold to contain both helical and disordered character; the LR model predicted the dominant structural feature from composition alone 86.9% of the time (Fig 4E). Finally, we predicted the highly charged regions, where the LR model performed with an overall accuracy of 90.8% (Fig 4E). Most of this accuracy can be captured using only the top five coefficients of the model (Fig 4F). We also scored a dataset of PDB structures with secondary structure annotation (see Methods); the model performed with an accuracy of ~90% on these experimentally determined structures (S3C Fig). To see whether there were systematic differences in the relationship between amino acid composition and secondary structure when using real versus predicted structures, we also created a second LR model trained on purely helical or disordered sequences from the PDB-derived dataset and compared it to the original LR model (S3A and S3B Fig). The coefficients of the two models are highly correlated, with the interesting exceptions of the two helix-breakers P (which has the same order of importance in both models but a much larger magnitude coefficient in the PDB LR model) and G (which has a much higher relative importance in the PDB LR model). The PDB LR model performed with an accuracy exceeding 90%, a result fully independent of AlphaFold predictions and as such an important robustness check. To put our results in context and understand the breakdown of existing heuristics, we compared the accuracy of the LR model to the accuracy of the simple charge/hydropathy (Uversky) model, as well as a state-of-the-art deep-learning based disorder-predictor called flDPnn [47]. As expected, the Uversky model performed better than chance but worse than the LR model on the same sets of randomly drawn regions, but was completely non-predictive for highly charged regions (Fig 4E). This reinforces the idea that normalized net charge is not predictive of disorder (see marginal distributions in the right hand side of Fig 4C), at least for regions with this length distribution. In sum, virtually all the predictive power in the Uversky charge/hydropathy heuristic comes from hydropathy. Although the flDPnn predictions were significantly better than the Uversky prediction for the highly charged regions, they did not exceed the performance of the LR model. Finally, we were curious whether our model was specific to amino acid usage in yeast, or if it could be extended to other proteomes. Using three other proteomes for which structures have been predicted, Schizosaccharomyces pombe, Caenorhabditis elegans, and Homo sapiens, we performed the same procedure of random region selection, labeling using the AlphaFold predictions and confidences, and classification with the LR model trained on AlphaFold predictions of yeast regions. The prediction accuracy was nearly identical to or slightly higher than the S. cerevisiae accuracy (Fig 4G). This simple model based only on the composition of a region is sufficient to predict helical or disordered character in proteomes that diverged over a billion years ago. Highly charged regions are evolutionarily conserved The unique evolutionary signatures suggested by our analysis of eIF3A, coupled with the consistency in predictive power of the LR model across vast evolutionary distances, led us to broader questions about the conservation of the regions we had identified. To what extent do highly charged regions retain their sequence properties and structure as organisms evolve? To address this question, we turned to AYbRAH, a curated database of protein homologs and paralogs in 33 fungal species spanning 600 million years of evolution [50]. This dataset combines automated homology detection and manual curation to achieve high-confidence predictions of highly diverged orthologs. First, we quantified the sequence conservation of the regions in question by examining their alignments and calculating both the frequency of gaps and the sequence divergence (average position-wise entropy). Sequences with high insertion and deletion rates, a known feature of IDRs, will have higher alignment gap frequencies. Those with a high point mutation rate will have high divergence. A low value of both metrics indicates sequence conservation. We calculated these values for all the highly charged regions, and compared them to length-matched, randomly drawn regions from the rest of the same proteins. We found that the charged regions have significantly more gaps (P<0.001, Mann-Whitney U test) and are more divergent (P<0.001, Mann-Whitney U test) than the proteins in which they are found (Fig 5A). We compared these distributions to the same values calculated for experimentally verified IDRs from DisProt, and found that although the charged regions have similar gap frequency to IDRs, they are even more divergent at the sequence level. Thus simply viewing an alignment of these LCRs, without using a secondary structure prediction algorithm, one might conclude that they are disordered. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Highly charged regions are evolutionarily conserved. a The distribution of gap frequencies (left) and average position-wise entropy (right) summarizing multiple sequence alignments (MSAs) of the highly charged regions (purple), randomly drawn regions from the rest of the proteins that contain them (black), and IDRs from DisProt (gray). b Summary of the variance (in log odds space) of usage of categories of amino acids for all proteomes in the AYbRAH database. c FCR values, averaged across AYbRAH alignments, for the highly charged regions identified in the S. cerevisiae proteome and length-matched randomly-drawn regions and their associated AYbRAH MSA. d The average compositional conservation of regions enriched for all sets of four amino acids with the same total frequency as the charged amino acids, plotted as a CDF. Higher values indicate less drift, and lower values indicate more drift (regression to the proteome average). https://doi.org/10.1371/journal.pcbi.1011565.g005 Despite this apparent lack of conservation, we were curious whether any aspects beyond sequence of the highly charged regions were conserved. These regions were identified because of their unique sequence composition, so we tested the degree to which they retained this composition over evolutionary time. To first determine the expected compositional variation, we measured the variation in the total proportion of each amino acid across the species represented in the AYbRAH database (S4A Fig). We found that as a group, the charged amino acids had very little variation in proportion of usage (Fig 5B). Consistent with selection, a high fraction of charged residues was preserved across species and substantially differed from randomly drawn regions in the same species (Fig 5C). The distribution in Fig 5C contains some regions that on average fall below the threshold of 0.4 FCR that we established for the original search in the yeast proteome; this is not surprising given that all sequences are subject to drift, which pulls them towards the proteome average for any given trait unless selection intervenes. Therefore, we created a method to quantify drift in charged regions relative to other compositionally extreme regions. We first identified regions in the yeast proteome enriched for all groups of four amino acids with a combined frequency within +/–0.01 of the combined frequency of the charged amino acids (0.233). For each of the 209 datasets, we calculated the mean proportion of the amino acids in question for each identified region across the AYbRAH alignment (note that FCR is a special case of this property where the four amino acids in question are glutamate, aspartate, lysine, and arginine) If the composition (enrichment of the four amino acids in question) is conserved, we should expect that the mean enrichment score across regions and alignments should be close to the mean of the original enriched dataset (close to or higher than the 0.4 threshold). In contrast, if the property is not conserved, we should expect this enrichment score to be close to the proteome average. To compare directly between datasets, we scaled this enrichment score to a unit scale between an effective 0 (the proteome average), and 1 (the median of the enriched regions detected from the S. cerevisiae proteome). Sets of four with a conservation score close to 1 are highly conserved for that property, while those that are close to 0 have experienced high levels of drift, indicating that they are not conserved. We find that the set of four charged amino acids falls within the top 5% of these scores (Fig 5D). From this analysis we conclude that in the highly charged regions, the charge density is extremely well-conserved. Discussion To understand the biology of proteins and their subdomains, heuristics are almost inevitably used: comparison to other proteins to infer similarity by homology, motif identification to predict binding partners, and so on. In the case of highly charged protein regions, several heuristics appear to converge on the conclusion that such regions are overwhelmingly likely to be disordered. By virtue of their strongly biased sequence composition, they tend to fall into the class of low-complexity sequences associated with lack of stable structure [51]; they tolerate insertion/deletion events at higher rates than typical well-folded sequences; their high charge favors interactions with solvent, and low hydrophobicity suggests the absence of a solvent-protected hydrophobic core. Consistent with this, many analyses of such regions proceed as though structure can be mostly or completely ignored. Here, we have shown that naturally occurring highly charged regions are predicted to adopt helical structure to a degree which cannot be neglected—~40% in a proteome-wide analysis—and that these predictions are supported by existing experimental data for both structured and disordered sequences. Moreover, we show that all these heuristic signals of disorder are in fact compatible with fully structured polypeptides: extended charged helices which have no hydrophobic core, grow and shrink in length over evolutionary time, interact with solvent on all sides, and form from sequences of two or even one type of amino acid (Fig 6A). Together, our results indicate that understanding the biology of highly charged sequences requires integrating insights from both structural and disorder-based approaches. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Rethinking assumptions of disorder in highly charged regions. a The same set of predictors that have been associated with disordered regions turn out to also be compatible with a fully structured helical region. b Present leading-edge methods for disorder and structure prediction disagree completely and also fail to capture experimental reality. The highly charged region of Rcf1 is alternatively predicted to be near-completely disordered or completely helical; experimentally and biologically, it forms most of the membrane-spanning helices and exposed loops in this dimeric mitochondrial inner membrane protein. https://doi.org/10.1371/journal.pcbi.1011565.g006 A consequence of these results is that they upend multiple well-established heuristics for determining how to think about, and study, a sequence’s biological behaviors. The shortcut that charge and hydrophobicity can serve as accurate dimensions for separating structured from disordered sequences, powerfully demonstrated using limited data available at the time [2], does not work for highly charged, low-hydrophobicity sequences. Although many sophisticated methods for detecting disorder or single helices have been developed [33,34,47,48,52], the assumption that, because many disordered sequences are polyampholytes, other polyampholyte sequences can reasonably be treated as if disordered persists in modern work [3]. We emphasize that our results say nothing about the utility of further results built on the assumption of disorder, conditioned on its accuracy. And we further stress that considerable work may properly focus only on disordered sequences with no claims regarding the assessment of disorder. Nevertheless, as for the example of (E4K4)n polymers, it is straightforward to find examples in which sequences known to have well-defined structure are treated as if they did not, evidence for the undue influence of improper heuristics. Given these results, it might seem inevitable that sophisticated structure-prediction methods would be required to more accurately discern whether particular highly charged sequences adopt a helical conformation. However, we introduce a simple amino acid composition-based classifier—logistic regression with as few as five inputs—which can predict structure (or its absence) with accuracy above 90%. This model is trained on biologically occurring sequences, a tiny and profoundly biased subset of protein sequence space, such that we do not expect its performance to carry over to arbitrary sequences. Still, as a heuristic method implemented with a handful of numbers, it balances simplicity and accuracy (particularly over the charge/hydropathy heuristic) in a way which is practically useful in diagnosing structure for charged sequences. The notion that intrinsically disordered proteins or regions sometimes adopt structure is well-understood [35,53], particularly in the case of folding upon binding [54]. Because of this distinction between conformations in isolation versus when bound to a partner, structures in the PDB may tell only a portion of the story. Similarly, AlphaFold specifically predicts structures most likely to appear in the PDB [33,42], rather than, for example, conformations which are occupied most of the time in the biological context. To the extent that our results depend on these resources, they similarly remain inapplicable to questions about the broader conformational ensembles that highly charged sequences may sample. But how stable or frequently adopted might these structures be? From the perspective of evolutionary conservation, even a conformation which is occupied for a tiny fraction of the time may impose dominant constraints on a sequence, if this conformation contributes to organism fitness. To the extent that we wish to understand the relationship between sequence and biological function, this potential for rare conformations to dictate function may permit most conformations in the ensemble to be neglected—much as recognition of folding upon binding for a disordered region may properly focus attention on the bound state, even if it is fleeting. In the case of highly charged regions which must adopt helical conformations to carry out their functions, certain near-absolute constraints must be satisfied; no matter how unstable the helix, a proline kink in the backbone cannot be straightened, and so depletion of proline from these regions provides an additional signal. On the other hand, presence of a proline powerfully indicates that a straight helix cannot form and is therefore unlikely to be the functional conformation, no matter what other sequence signals exist and no matter how fleeting the helix state is proposed to be. Even the best methods for predicting structure and for predicting disorder can disagree and fail to capture experimental reality. Consider the highly charged region of the yeast protein Rcf1 (Fig 6B). A top-ranked modern disorder predictor, flDPnn [47,52], predicts this region to be entirely disordered. AlphaFold predicts it to be entirely helical with high confidence. Neither captures reality: experimentally, this region forms most of a dimeric five-pass transmembrane protein in which charged residues, exposed on stable helices, form dimer-stabilizing salt bridges through the mitochondrial inner membrane [55] (Fig 6B). To the extent that cases like this closing example persist [56], the challenges we identify here remain open. Moverover, discrepancies between predicted proteome-wide frequencies of disorder [57] and inference of the same measure from experiment [58] invite deeper investigation of specific cases, as we have done here. Broadly, while our results uncover previously overlooked structure in highly charged regions, the dual challenges of determining the biologically active configurations of these sequences, and of determining the statistical features of the conformational ensembles they occupy, remain open. Rather than looking at such sequences through the lens of disorder, it appears that both lenses—structure and disorder—will be needed to give the proper depth of focus. Methods Extraction of highly charged regions from the yeast proteome The S288C reference genome was obtained from the Saccharomyces Genome Database (SGD). For each gene in the reference genome, we first computed fractional charge as a moving average across its sequence using a window size of 12 residues and a triangular weight, where the highest weight was assigned to the middle region of each window. We then searched for highly charged regions in each sequence based on a fractional charge threshold of 0.4 and tolerance of 10 residues. Tolerance refers to the maximum number of residues that we allow to have moving average values below the fractional charge threshold before terminating the region. This tolerance allows for transient deviations from high charge and prevents fragmenting highly charged regions with small insertions of uncharged amino acids. We extracted regions that were longer than a given minimum region length of 30 residues, then trimmed any remaining uncharged residues from the N and C terminal ends of the sequences (these result from the triangular weighting scheme and the tolerance). Calculating sequence complexity Sequence complexity was calculated according to [6] using the following equation: where K2 is the unnormalized complexity, N is the number of possible residues (in this case the 20 natural amino acids), and ni is the number of each residue in the sequence, which has length L. This value is normalized to the “entropy of the language” (e.g., the yeast proteome), such that a sequence with compositional properties exactly equal to the average frequencies will have a complexity of 1. The entropy of the language is calculated using the equation where pi is the frequency of letter i in the reference. We used two different languages as references; the first is the yeast proteome (so each pi represents the average frequency of that amino acid in the proteome). We also used a modified reference enriched for charged residues; each of the amino acids lysine, arginine, aspartate, and glutamate had a frequency of 0.125; the remaining frequency (0.5) was distributed evenly among the other amino acids. Generation of null distribution for amino acid usage We counted the number of occurrences of adenine (A), thymine (T), cytosine (C), and guanine (G) in the DNA sequences of all open reading frames in S. cerevisiae. The expected frequency of each codon was computed as the product of the frequencies of all nucleotides that appear in that codon and the expected frequency of each amino acid was computed as the sum of the expected frequencies of all codons specifying that amino acid. Extraction of AlphaFold data We used proteome-wide structure predictions from AlphaFold to analyze the structure of the regions we identified with high proportions of charged residues. We downloaded the structures for all S. cerevisiae proteins from the AlphaFold website (https://alphafold.ebi.ac.uk/download#proteomes-section) [33]. We read the PDB files into Python and used DSSP implemented in MDTraj [46] to score secondary structure. We used custom Python functions to extract the confidence scores from the predicted structure file for each protein. Construction of a logistic regression (LR) model to predict secondary structure To classify the secondary structure of a region on the basis of composition, we first constructed a training dataset built from AlphaFold structures. We classified all residues in all structures as either helical (classified as helical by DSSP and with a pLDDT score above 70), disordered (classified as coil by DSSP and with a pLDDT score above 70 or any residue with a pLDDT score less than 50) [38], sheet (classified as sheet by DSSP and with a pLDDT score above 70), or other. The pLDDT cutoff of 70 marks was chosen because this value was used by the creators of AlphaFold to distinguish between “Confident” and “Low” model confidence. We did not explicitly test the dependence of our results on the value of this cutoff since it was defined and extensively validated by the creators of the model. To generate training and testing data for the LR model, we exhaustively searched for purely helical and disordered regions by identifying regions that were greater than 25 amino acids long and only contained either helical or disordered residues. We extracted 6882 helical regions and 6366 disordered regions from the S. cerevisiae proteome. We randomly selected 75% of these regions as training data and 25% of the regions as testing data and built a LR model using amino acid composition as the predictor, that is, each individual amino acid is assigned a weight in the regression. The LR model is built using the scikit-learn package in Python. We also constructed a LR model based on empirical structures from the PDB; secondary structure and disorder were annotated on a per-residue basis from the experimental 3D structure by the PDB and were obtained by request [59] (see their Methods for details). The final dataset contained 64,804 regions greater than 25 amino acids long and consisting solely of either helix or disorder; the regions were approximately equally split between helical and disordered. Otherwise, the procedure was identical to that used for the AlphaFold predictions. Classifying regions and computing model accuracy We used the per-residue secondary structure classifications described above to score the highly charged regions: regions with more than 60% helical or disordered residues were labeled as their dominant type, and all others were labeled as “intermediate.” These experimental classifications were taken to be ground truth in the accuracy scores for each method (Uversky, LR, and flDPnn). Out of all the regions, 34% were labeled as disordered and 31% were labeled as helical. We then used the LR model to classify these regions on the basis of their amino acid composition. The ground-truth labels were used to compute the false negative and positive rates, and the overall model accuracy. To directly compare to the LR dataset, we extracted all purely helical or purely disordered regions as well as random regions from the AlphaFold dataset. We used the same scoring conditions as that used for the highly charged regions (>60% helical or disordered), scored each region using the LR model, and computed model accuracy in the same way as described above. To compare between the accuracy of the LR model and the Uversky model, we also used the Uversky model to classify the three sets of protein regions as helical or disordered. In particular, we calculated the normalized net charge and normalized mean hydropathy for each of the regions. The regions that fall above and to the left of the dividing line are classified as disordered and the remaining regions are classified as helical. The accuracy of the Uversky model is computed in the same way as for the LR model. Finally, we used flDPnn to classify each of the highly charged and random regions as disordered or ordered. We averaged the predicted score for disorder for each of the residues in each given region and classified regions with a mean disorder score of 0.3 (the threshold chosen by flDPnn) or higher as disordered. The remaining non-disordered regions were assumed to be compatible with helices, the only order type in this dataset. The accuracy of flDPnn predictions was then computed in the same way as above. Evolutionary analysis: sequence properties We used custom Python scripts to extract multiple sequence alignments (MSAs) for all proteins in the AYbRAH alignment [50]. A region was considered “present” in a protein in the MSA if the region which aligned to the S. cerevisiae sequence that we identified as highly charged contained at least 30 amino acids (the same length minimum length as was required for a region detected by the algorithm). To compute alignment quality, the longest and shortest sequences in each alignment were removed, and any resulting columns containing only gaps (represented by the symbol “-” in the alignment) were removed. We then quantified the frequency of gaps as well as sequence column-wise entropy, which we refer to as “sequence divergence.” The mean frequency of gaps was computed as the number of “-” characters divided by the total number of characters across all sequences in an alignment. Sequence divergence was computed by summarizing the frequency of amino acids in each column as a one-dimensional probability distribution and calculating the Shannon entropy of that distribution. Evolutionary analysis: compositional drift To test compositional drift for regions enriched in selected amino acids, we identified all unique sets of four amino acids (excluding the charged amino acids) with the same combined frequency as the four charged amino acids. We modified the charged region algorithm to detect regions enriched for these sets of four. We used an enrichment threshold of 0.35 (35% of the region composed of the amino acids in question), minimum length of 30 amino acids, and a tolerance of 15; these parameters were selected to yield datasets that most closely matched the number of hits and length distribution of the original dataset (enrichment for E, D, R, and K). For each of these datasets, we calculated the conservation by averaging the enrichment score (percent composition of the specific amino acids) across the alignment in each region, taking the mean of that distribution, and scaling it between the S. cerevisiae proteome average (0) and the average enrichment of the hits detected by the algorithm (1). This allowed us to compare datasets directly: sets of amino acids with values close to 0 experienced enough drift that they approach the proteome average; those with values close to 1 stay far from the average and close to the (rescaled) enrichment threshold and thus are likely conserved. Analysis of the eIF3A charged helix We used MUSCLE version 3.8.31 [60] with default parameters to generate an alignment of all the eIF3A homologs for which a structure was available on AlphaFold as of April 2022 (35 species of model organisms including bacteria, yeast, mold, rice, soybean, mouse, and human). This dataset was used in Fig 3. A cryoEM structure of eIF3A in complex with the ribosome was used for structural analysis. The structure has PDB code 6ZCE (http://doi.org/10.2210/pdb6ZCE/pdb) [45]. Secondary structure was scored with DSSP as described above. The length of the charged helix was calculated by identifying the start of the highly charged region and then counting the amino acids until a run of more than three non-helix characters was encountered. A reference helix from earlier in the sequence was chosen as a comparison. Analysis of Rcf1 The 70-residue Rcf1 charged region (Fig 6B) was used for disorder predictions with flDPnn [47] at http://biomine.cs.vcu.edu/servers/flDPnn/ with default settings, and for structure prediction with AlphaFold through ColabFold [48]. Statistical tests All p-values were calculated with the Mann-Whitney U Test (Wilcoxon Rank Sum Test), either two-sided if no hypotheses were formed about the relationship between the two distributions or one-sided otherwise. The evaluation of these tests was done in Python and can be found (along with the data) in the Jupyter notebook for the relevant figure. Extraction of highly charged regions from the yeast proteome The S288C reference genome was obtained from the Saccharomyces Genome Database (SGD). For each gene in the reference genome, we first computed fractional charge as a moving average across its sequence using a window size of 12 residues and a triangular weight, where the highest weight was assigned to the middle region of each window. We then searched for highly charged regions in each sequence based on a fractional charge threshold of 0.4 and tolerance of 10 residues. Tolerance refers to the maximum number of residues that we allow to have moving average values below the fractional charge threshold before terminating the region. This tolerance allows for transient deviations from high charge and prevents fragmenting highly charged regions with small insertions of uncharged amino acids. We extracted regions that were longer than a given minimum region length of 30 residues, then trimmed any remaining uncharged residues from the N and C terminal ends of the sequences (these result from the triangular weighting scheme and the tolerance). Calculating sequence complexity Sequence complexity was calculated according to [6] using the following equation: where K2 is the unnormalized complexity, N is the number of possible residues (in this case the 20 natural amino acids), and ni is the number of each residue in the sequence, which has length L. This value is normalized to the “entropy of the language” (e.g., the yeast proteome), such that a sequence with compositional properties exactly equal to the average frequencies will have a complexity of 1. The entropy of the language is calculated using the equation where pi is the frequency of letter i in the reference. We used two different languages as references; the first is the yeast proteome (so each pi represents the average frequency of that amino acid in the proteome). We also used a modified reference enriched for charged residues; each of the amino acids lysine, arginine, aspartate, and glutamate had a frequency of 0.125; the remaining frequency (0.5) was distributed evenly among the other amino acids. Generation of null distribution for amino acid usage We counted the number of occurrences of adenine (A), thymine (T), cytosine (C), and guanine (G) in the DNA sequences of all open reading frames in S. cerevisiae. The expected frequency of each codon was computed as the product of the frequencies of all nucleotides that appear in that codon and the expected frequency of each amino acid was computed as the sum of the expected frequencies of all codons specifying that amino acid. Extraction of AlphaFold data We used proteome-wide structure predictions from AlphaFold to analyze the structure of the regions we identified with high proportions of charged residues. We downloaded the structures for all S. cerevisiae proteins from the AlphaFold website (https://alphafold.ebi.ac.uk/download#proteomes-section) [33]. We read the PDB files into Python and used DSSP implemented in MDTraj [46] to score secondary structure. We used custom Python functions to extract the confidence scores from the predicted structure file for each protein. Construction of a logistic regression (LR) model to predict secondary structure To classify the secondary structure of a region on the basis of composition, we first constructed a training dataset built from AlphaFold structures. We classified all residues in all structures as either helical (classified as helical by DSSP and with a pLDDT score above 70), disordered (classified as coil by DSSP and with a pLDDT score above 70 or any residue with a pLDDT score less than 50) [38], sheet (classified as sheet by DSSP and with a pLDDT score above 70), or other. The pLDDT cutoff of 70 marks was chosen because this value was used by the creators of AlphaFold to distinguish between “Confident” and “Low” model confidence. We did not explicitly test the dependence of our results on the value of this cutoff since it was defined and extensively validated by the creators of the model. To generate training and testing data for the LR model, we exhaustively searched for purely helical and disordered regions by identifying regions that were greater than 25 amino acids long and only contained either helical or disordered residues. We extracted 6882 helical regions and 6366 disordered regions from the S. cerevisiae proteome. We randomly selected 75% of these regions as training data and 25% of the regions as testing data and built a LR model using amino acid composition as the predictor, that is, each individual amino acid is assigned a weight in the regression. The LR model is built using the scikit-learn package in Python. We also constructed a LR model based on empirical structures from the PDB; secondary structure and disorder were annotated on a per-residue basis from the experimental 3D structure by the PDB and were obtained by request [59] (see their Methods for details). The final dataset contained 64,804 regions greater than 25 amino acids long and consisting solely of either helix or disorder; the regions were approximately equally split between helical and disordered. Otherwise, the procedure was identical to that used for the AlphaFold predictions. Classifying regions and computing model accuracy We used the per-residue secondary structure classifications described above to score the highly charged regions: regions with more than 60% helical or disordered residues were labeled as their dominant type, and all others were labeled as “intermediate.” These experimental classifications were taken to be ground truth in the accuracy scores for each method (Uversky, LR, and flDPnn). Out of all the regions, 34% were labeled as disordered and 31% were labeled as helical. We then used the LR model to classify these regions on the basis of their amino acid composition. The ground-truth labels were used to compute the false negative and positive rates, and the overall model accuracy. To directly compare to the LR dataset, we extracted all purely helical or purely disordered regions as well as random regions from the AlphaFold dataset. We used the same scoring conditions as that used for the highly charged regions (>60% helical or disordered), scored each region using the LR model, and computed model accuracy in the same way as described above. To compare between the accuracy of the LR model and the Uversky model, we also used the Uversky model to classify the three sets of protein regions as helical or disordered. In particular, we calculated the normalized net charge and normalized mean hydropathy for each of the regions. The regions that fall above and to the left of the dividing line are classified as disordered and the remaining regions are classified as helical. The accuracy of the Uversky model is computed in the same way as for the LR model. Finally, we used flDPnn to classify each of the highly charged and random regions as disordered or ordered. We averaged the predicted score for disorder for each of the residues in each given region and classified regions with a mean disorder score of 0.3 (the threshold chosen by flDPnn) or higher as disordered. The remaining non-disordered regions were assumed to be compatible with helices, the only order type in this dataset. The accuracy of flDPnn predictions was then computed in the same way as above. Evolutionary analysis: sequence properties We used custom Python scripts to extract multiple sequence alignments (MSAs) for all proteins in the AYbRAH alignment [50]. A region was considered “present” in a protein in the MSA if the region which aligned to the S. cerevisiae sequence that we identified as highly charged contained at least 30 amino acids (the same length minimum length as was required for a region detected by the algorithm). To compute alignment quality, the longest and shortest sequences in each alignment were removed, and any resulting columns containing only gaps (represented by the symbol “-” in the alignment) were removed. We then quantified the frequency of gaps as well as sequence column-wise entropy, which we refer to as “sequence divergence.” The mean frequency of gaps was computed as the number of “-” characters divided by the total number of characters across all sequences in an alignment. Sequence divergence was computed by summarizing the frequency of amino acids in each column as a one-dimensional probability distribution and calculating the Shannon entropy of that distribution. Evolutionary analysis: compositional drift To test compositional drift for regions enriched in selected amino acids, we identified all unique sets of four amino acids (excluding the charged amino acids) with the same combined frequency as the four charged amino acids. We modified the charged region algorithm to detect regions enriched for these sets of four. We used an enrichment threshold of 0.35 (35% of the region composed of the amino acids in question), minimum length of 30 amino acids, and a tolerance of 15; these parameters were selected to yield datasets that most closely matched the number of hits and length distribution of the original dataset (enrichment for E, D, R, and K). For each of these datasets, we calculated the conservation by averaging the enrichment score (percent composition of the specific amino acids) across the alignment in each region, taking the mean of that distribution, and scaling it between the S. cerevisiae proteome average (0) and the average enrichment of the hits detected by the algorithm (1). This allowed us to compare datasets directly: sets of amino acids with values close to 0 experienced enough drift that they approach the proteome average; those with values close to 1 stay far from the average and close to the (rescaled) enrichment threshold and thus are likely conserved. Analysis of the eIF3A charged helix We used MUSCLE version 3.8.31 [60] with default parameters to generate an alignment of all the eIF3A homologs for which a structure was available on AlphaFold as of April 2022 (35 species of model organisms including bacteria, yeast, mold, rice, soybean, mouse, and human). This dataset was used in Fig 3. A cryoEM structure of eIF3A in complex with the ribosome was used for structural analysis. The structure has PDB code 6ZCE (http://doi.org/10.2210/pdb6ZCE/pdb) [45]. Secondary structure was scored with DSSP as described above. The length of the charged helix was calculated by identifying the start of the highly charged region and then counting the amino acids until a run of more than three non-helix characters was encountered. A reference helix from earlier in the sequence was chosen as a comparison. Analysis of Rcf1 The 70-residue Rcf1 charged region (Fig 6B) was used for disorder predictions with flDPnn [47] at http://biomine.cs.vcu.edu/servers/flDPnn/ with default settings, and for structure prediction with AlphaFold through ColabFold [48]. Statistical tests All p-values were calculated with the Mann-Whitney U Test (Wilcoxon Rank Sum Test), either two-sided if no hypotheses were formed about the relationship between the two distributions or one-sided otherwise. The evaluation of these tests was done in Python and can be found (along with the data) in the Jupyter notebook for the relevant figure. Supporting information S1 Fig. Regions high in charge are prevalent in the yeast proteome and are low complexity. a The distribution of FCR values for randomly-drawn regions of different length in the yeast proteome (black, purple), and randomly generated regions using the average S. cerevisiae amino acid frequencies (gray). Vertical lines show two standard deviations above the mean. The minimum FCR of the highly charged regions after trimming is shown with a red dashed line. b The distribution of FCR values for the regions detected (purple), the proteins that contain them (green and gray), and the per-protein FCR for all yeast proteins (black). c Distribution of region lengths for highly charged regions. d Enrichment for proteins containing highly charged regions in different cell compartments. The proportion of the proteins in the category with at least one highly charged region is shown in black. The expected proportion (based on the number of unique genes detected by the algorithm and the total number of yeast proteins) is shown as a horizontal white line. https://doi.org/10.1371/journal.pcbi.1011565.s001 (TIFF) S2 Fig. Secondary structure is pervasive in highly charged, low complexity regions. a The frequency of (per-residue) secondary structure for all the confidently predicted (pLDDT > 70) residues from the AlphaFold structure for highly charged regions, length-matched random regions, and experimentally-verified disordered regions from DisProt. The majority of confidently prediction regions in disordered (DisProt) regions are coil. b The κ value of the region versus the proportion predicted helix for all regions that are more than 60% helix or disordered. c Alignment of MNN4 C-terminal region in fungal species in the AYbRAH database. https://doi.org/10.1371/journal.pcbi.1011565.s002 (TIFF) S3 Fig. Helical regions can be predicted from composition. a Coefficients from the LR model trained on helical and coil/disordered regions randomly drawn from the PDB. b Comparison between model coefficients in both the PDB and AlphaFold trained models. The importance of P and G varies the most between the two; P in magnitude and G in order of importance. c Accuracy of the AlphaFold and PDB trained models and the accuracy of the AlphaFold model when used to classify PDB structures. https://doi.org/10.1371/journal.pcbi.1011565.s003 (TIFF) S4 Fig. Highly charged regions are evolutionarily conserved. a Variation in frequency for each individual amino acid. Distribution is the frequency of usage in each proteome in the AYbRAH database. b Absolute coverage (number of species in the AYbRAH alignment which contain non-gap characters in the detected highly charged region relative to the maximum number of species possible) and relative coverage (number of species in the AYbRAH alignment which have non-gap characters in the highly-charged region relative to the number of species with a homolog of the protein) for each protein with a detected highly charged region. c Length variation (between AYbRAH homologs) in regions labeled as helix (teal) or disordered (gray) based on their predicted structure from AlphaFold and experimentally-verified yeast disordered regions from DisProt (black). https://doi.org/10.1371/journal.pcbi.1011565.s004 (TIFF) S1 Table. Overlap between predicted helical highly charged regions and proteins found in DisProt. The proteins listed were predicted to contain highly helical regions (summarized in Fig 2C) and also found in the Disprot database (a reference for experimentally verified disordered proteins and regions). https://doi.org/10.1371/journal.pcbi.1011565.s005 (XLSX) Acknowledgments The authors thank Alex Holehouse for helpful discussions, and Alexander Cope for providing the structure-annotated PDB data.

Boughter, Christopher T.;Meier-Schellersheim, Martin

doi: 10.1371/journal.pcbi.1011577pmid: 37862356

Introduction To control infection and disease, the adaptive immune system of higher organisms utilizes a complex collection of receptors and signaling pathways specifically tailored to each individual immunological challenge [1–4]. Over the past decade, researchers have increasingly leveraged these receptors, specifically antibodies and T cell receptors (TCRs), to generate novel therapeutics [5–11]. Generally, the success of natural immune responses or therapeutics are strongly dependent on the ability of these receptors to recognize and appropriately respond to pathogenic threats. However, recognition of pathogens is a dynamic challenge for the immune system as the generation of its receptors is dependent on the identity of the pathogens, and the pathogens themselves are frequently capable of generating compensatory mutations that, in turn, require adaptations in the immune responses. Both sides of this competition are subject to a balancing act; successful pathogens must mutate and generate variants that reduce detection by the host immune system yet maintain a sufficient level of biological fitness, whereas successful immune responses must recruit or generate receptors that bind with high affinity and specificity to a given pathogen yet ideally maintain sufficient breadth to adapt quickly to these pathogenic variants [12–14]. This biological back and forth is encapsulated by the amino acid sequences that determine the interaction strength between the molecular players involved in adaptive immune recognition. The costs for determining these amino acid sequences of immune receptors has been decreasing rapidly [15], thereby providing access to datasets exponentially increasing in size [16–20]. Likewise, current sequencing technologies allow us to follow the evolution of viruses and identify variants of concern in real-time across the globe [21]. Characterization of the peptides presented by MHC, also referred to as the immunopeptidome, relies on mass spectrometry-based identification. While this method severely limits the coverage of each experiment, single immunopeptidomic assays can yield thousands of identified pathogenic or self-peptides [22]. As these sequence databases continue to expand, methods for analyzing their large datasets must keep pace, helping researchers to identify key distinguishing features of the sequences identified in any given immunological niche. Excellent software exist for the analysis of TCR sequences [23–28], antibodies [25, 29–32], and peptides [33–35]. Conversely, the analyses of viral sequences are largely dependent on multi-sequence alignments, phylogenetic analysis, or custom pipelines from researchers in a specific viral sub-field. While each of these approaches are powerful tools in their respective fields, they make comparisons across immune repertoires difficult. Software that compares, for instance, peptide and TCR repertoires typically give a simple binary “yes” or “no” to questions of binding, removing the underlying biophysical context that determines these interactions. Further, a majority of the analyses are developed for a very specific task, such as prediction of peptide binding to a specific MHC allele or identification of the evolutionary trajectory of a given antibody sequence. General characterizations of a given immune repertoire are often done via an in-house analysis, focusing on simplified quantities such as net biophysical properties of sequences, as well as their lengths or conservation. To facilitate more thorough analyses and comparisons of amino acid sequences, we developed the AIMS (Automated Immune Molecule Separator) software to take into account their fundamental biophysical properties to characterize, differentiate, and identify clusters within immune repertoires. While the initial input and encoding of sequences into AIMS is different for each of the distinct molecular classes of immune repertoires, the downstream analysis is identical and allows for cross-receptor comparisons and the identification of patterns in the corresponding trends of interacting molecules. The application of AIMS for targeted investigations of specific biological systems has been previously described [36–38]. Here, we outline applications of the software to each immune repertoire class with a specific focus on the software’s integrated analytical capabilities for cross-repertoire analyses. Results Encoding amino acid sequences and their biophysical properties Although the ideal repertoire analysis would build off of complex structures either determined experimentally or predicted computationally, the former approach is inherently low-throughput while the latter is unreliable, even for the most advanced structural prediction software to date [28, 39]. The AIMS software, conversely, takes advantage of the structural conservation inherent to immune molecules, selecting out only the regions involved in the interaction interface. These conserved interacting regions, which are highly variable at the sequence level, are then aligned in matrix form using a pseudo-structural approach that varies across the available analysis modes for each molecular species. By incorporating general structural features, rather than explicit contact predictions, AIMS reduces the bias of the analysis by minimizing the reliance on assumptions of structural accuracy. Among TCR-peptide-MHC complexes, the interaction interface is strikingly similar, with crystal structures consistently finding nearly identical docking angles between the two [37, 40–44]. TCRs contact the peptide and the MHC α-helices via their six complementarity determining region (CDR) loops, which are in turn connected via stem regions to their well conserved framework regions. These stem regions adjacent to the CDR loops are never found within 5 Å of the antigen [26], and are easily identified by highly conserved amino acids, allowing for the exclusion of framework regions from the analysis. In a majority of structures, assuming the conserved stem regions are defined as endpoints, the central 4–5 residues of the CDR3 loops contact the central residues of the peptide [26] (Fig 1A and 1B). From these general structural rules, we can inform the encoding of TCR sequences into AIMS, imposing a “central” alignment scheme as the standard. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Example of AIMS encoding for the analysis of TCR-peptide interactions. (A) Rendering of a specific TCR-pMHC interaction (PDB ID: 1OGA) with TCRα shown in blue, TCRβ in orange, MHC in white. (B) Inset shows a zoom in on this TCR-peptide interface, with the MHC now translucent. Representative AIMS encoding of the single TCR CDR3β sequence (C, central-encoding) or the peptide sequence (D, bulge-encoding) in panel A. Below these single encodings are examples of full TCR repertoire (C) or immunopeptidome (D) AIMS-encoded matrices. Each amino acid in the structures, the single encoded sequences, and the matrices is represented by a unique color. https://doi.org/10.1371/journal.pcbi.1011577.g001 In the central alignment scheme, we align each sequence to the central residue of each CDR loop. Whereas most analysis tools segregate TCR sequences by length, thereby artificially segmenting the data, the central alignment scheme of AIMS allows for receptors of all lengths to be analyzed simultaneously while focusing on the key regions of the receptor. Due to the length differences of the TCR sequences in a given dataset, signals from the CDR stem regions will be averaged out, thus prioritizing signals from the center of the CDR loops. We can visualize an example of this encoding for a test dataset of paired TCRα and TCRβ sequences from the VDJ database [16] (Fig 1C). In this matrix, we can see that each amino acid is encoded as a unique number in the matrix 1 to 21, or a unique color in the figure, with padded zeros between CDR loops represented by white space in the figure. To control for potential artifacts introduced by this approach, analysis can be repeated with “left” or “right” alignment of the sequences, aligning to the N- or C-termini, respectively, of the given sequences (S1 Fig). The standard AIMS encoding for peptides is slightly different from this central TCR alignment scheme. For class I MHC, the flanking regions of bound peptides are frequently ‘buried’ as highly conserved anchor residues that bind to the MHC platform (Fig 1A and 1B). The majority of TCR contacts are made with the central regions of the peptides that bulge out of the MHC binding groove in the case of longer peptides [45]. However, exceptions to this paradigm may not be uncommon, with TCRs capable of contacting the often-buried peptide N-terminal residue [46] and the C-terminal residue potentially extending out of the MHC pocket [47]. Nonetheless, the length distribution of peptides presented by class I MHC is narrow [33], subverting some of the sequence length concerns present in the TCR analysis. As such, peptide encoding in AIMS adopts a “bulge” scheme. The bulge scheme aligns the N- and C-terminal residues to either edge of the matrix, along with a user-defined number of additional flanking residues. Zeros are padded between these flanking regions and the remaining residues are centrally aligned as in the case of the TCR sequences, again adopting the same numeric amino acid encoding scheme (Fig 1D). We can see clearly for this subset of HLA-A2 presented Influenza peptides taken from the Immune Epitope Database (IEDB) [48] the relative conservation at anchor position 2, compared to the variability at the center of the peptide sequences. Importantly, this bulge alignment can also be applied to TCR and antibody sequences, putting more focus on their conserved stem regions. AIMS is capable of analyzing other molecules with conserved structural features and localized interfacial heterogeneity, including antibodies [36], MHC and MHC-like molecules [37], and, more generally, any molecular subset that can be successfully aligned using existing multi-sequence alignment software [38] (S2 Fig). The generalized AIMS encoding scheme allows for any molecular biologist or bioinformatician to take advantage of the downstream biophysical characterization tools of AIMS for their application of interest. All downstream repertoire characterization follows from this initial encoding, and takes identical paths regardless of the immune repertoire under consideration (S3 Fig). In the following sections we will outline the distinct AIMS modules, applying them to data that best demonstrates the utility of the analyses we perform, rather than opting for a contiguous analysis to a single dataset. More extensive descriptions of AIMS input and output options are provided in the supporting information accompanying this manuscript. Unsupervised clustering of a TCR repertoire from an unsorted dataset To illustrate the implementation of dimensionality reduction and clustering modules in AIMS, we generate a new analysis from the paired-chain data derived from VDJdb [16]. These sequences are complete with metadata regarding their epitope specificity, MHC allele presentation, and the haplotype of the individual each receptor was isolated from, if it was naturally derived. As intuition and recent quantitative work have suggested [49], paired-chain TCR sequence data greatly increases the information content from a given repertoire sequencing experiment when compared to single chain sequencing. Using the dimensionality reduction and clustering modules, we can determine precisely how strongly the analysis changes upon inclusion of the CDR3α sequences (Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Comparison of the purity of receptor clustering using paired chain (left) or single chain CDR3B (right) data. Dimensionality reduction using UMAP, followed by density based OPTICS clustering subsects the data into biophysically similar paired chain (A) and single chain (B) receptors. The first ten of these clusters are then re-visualized in their AIMS-encoded matrix, with black lines marking different clusters (C, D). The antigen specificity of each of these clusters is then quantified by percentages (E, F), with the colors corresponding to specific peptides as shown in the key. https://doi.org/10.1371/journal.pcbi.1011577.g002 We first generate the sequence encoding and biophysical property matrices (see S1 Table for list of properties) for both the paired-chain dataset and the CDR3β-only dataset. From this biophysical property matrix, the sequences are projected onto a three-dimensional space using the uniform manifold approximation and projection (UMAP) dimensionality reduction algorithm [50], and subsequently clustered using the density-based OPTICS (Ordering Points To Identify the Clustering Structure) algorithm [51] (Fig 2A and 2B). It is important to note that, for improved clarity, the projections shown here remove four outlier sequences with a proline in CDR3α and CDR3β, a very rare amino acid in TCR CDR loops (see S4 Fig for projection with outliers). From the UMAP projection and OPTICS clustering of Fig 2A and 2B we can see that the number of distinct outlier populations is increased in the paired chain dataset, suggesting the identification of a higher number of biophysically distinct sequences. Visualizing just a subset of the clusters back as encoded matrices (Fig 2C and 2D) we see that this is likely due to the clustering of the paired chain data picking up unique motifs in both CDR3α and CDR3β, suggesting that these strong outliers can come from either chain. These clusters can then be analyzed for cluster membership compared to their original dataset identifiers, here based on the antigen recognized by each TCR (Fig 2E and 2F). Importantly, cluster purity can be measured relative to the metadata of choice, such as antigenic species source, allele of presenting MHC, organism haplotype, or virtually any other identifiable characteristic that can be encoded as metadata for the sample (S5 Fig). This visualization of antigen recognition for each cluster (Fig 2E and 2F) highlights subtle differences between the paired-chain and single-chain datasets. We see that despite a nearly equal number of clusters identified, the average cluster purity is higher for the paired chain data at 0.50 ± 0.28 when compared to the single chain data at 0.37 ± 0.24, although not significantly so. We notice however that the few pure or close to pure clusters are coming from the same two antigens, NLVPMVATV and LLWNGPMAV, largely because these antigens make up 43% and 23% of the total dataset, respectively. Overall, these results show that while unique CDR3α motifs are critically important for antigen recognition and for understanding the full breadth of receptor diversity, a fairly accurate picture of sequence diversity and similarity can still be generated from CDR3β sequences alone. Going beyond receptor clustering and motif analysis The biophysical analysis in this and the following section can be carried out with individual clusters of sequences derived from unstructured data, as outlined in the previous section, or from antigenically well defined populations (see S6 Fig or Boughter et al. [36] for examples). In highlighting the features of the biophysical property analysis, we select the two antigenically purest clusters of Fig 2 from both the paired-chain and single chain datasets with specificity to the NLVPMVATV and LLWNGPMAV antigens (Fig 3A and 3B). From these cluster subsets, we can appreciate that despite having nearly the same antigenic purity in each of these clusters, the paired chain clusters surprisingly display increased CDR3β diversity. While we are able to generate sequence logos for the distinct motifs in these clusters (S7A and S7B Fig), the AIMS analysis pipeline permits a more thorough biophysical characterization of these clusters. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Isolation of individual sequence clusters and subsequent position-sensitive biophysical characterization of these sequences highlights the details provided by AIMS analysis. A subset of clusters identified in Fig 2 are isolated and shown as their AIMS matrix encoding for the paired chain (A) and single-chain (B) datasets. From these encodings, we can calculate the position sensitive biophysical properties for each cluster (C, D). Opaque lines with dots represent the averages over each cluster, while wider translucent regions centered on these lines give the variance as calculated by a bootstrapping procedure (Methods). Statistically significant differences (p < 0.05, calculated via non-parametric permutation test) are denoted by asterisks, with an asterisk over a solid bar representing extended regions of statistically significant differences. https://doi.org/10.1371/journal.pcbi.1011577.g003 Specifically, from the high-dimensional biophysical property matrix for each of these clusters, we can isolate single property masks (S7C and S7D Fig) which can be averaged across repertoires (Fig 3C and 3D) or across repertoires and positions (S7E and S7F Fig), generating position sensitive or net average biophysical properties for the molecular subsets of interest. These biophysical property visualizations can be used to more carefully compare and contrast the clusters generated from the paired-chain and single chain datasets. We see from the position-sensitive biophysical property averages that the physical properties of CDR3β are matched in the paired-chain and the single-chain clusters, despite the significant difference in diversity in this region. The positively charged segment of CDR3β in TCRs recognizing the LLWNGPMAV peptide is seen in both datasets, while a corresponding negative segment is found in the CDR3β of TCRs recognizing the NLVPMVATV peptide. Given the lack of charged residues in the peptides, the conservation of charge in these pure clusters is somewhat surprising. Conversely, the hydropathy score of the CDR3β chains is far more variable within the paired-chain clusters, although the same general trend is again followed when compared to the single-chain data, with two distinct peaks in hydropathy and a region neither lacking nor enriched in hydrophobic residues. This is a common feature in the clustering of sequences in AIMS, and may reflect the nature of the hydrophobicity metrics used, or alternatively could be suggestive of the critical determinants of the formation of TCR-pMHC complexes. While charges in the interface must be delicately arranged to form interfacial interactions and offset the significant energetic penalties of desolvation, more hydrophobic residues are free to effectively “fill in the blanks” and pack as best they can. While only the position sensitive charge and hydropathy are highlighted in this section, any of the 61 standard AIMS properties can be visualized either as position-sensitive averages across receptors, or as net averages over position- and sequence-space. Generating quantitative metrics of repertoire diversity and amino acid patterning While the biophysical characterization of immune repertoires can help answer questions regarding the mechanisms of immune recognition, complementary analysis can help contextualize these findings, relating biophysical properties back to patterns in the data resulting from the source of the molecules (e.g. human, mouse, or virus) or other external influences (e.g. selection or affinity maturation). In AIMS, we can further quantify entire repertoires, cluster molecular subsets, or define antigenic groups using an array of statistical and information theoretic approaches. Information theory is built for the analysis of sequences of inputs and outputs [52]. Here, these inputs and outputs are the amino acids making up our immune repertoires quantified via the observed probability distributions across these sequences. In telecommunication, quantification of inputs and outputs determine the messages which can be sent over a given channel, whereas in the study of immune receptors this same quantification determines the range of pathogenic targets which can be recognized by a given immune system. To illustrate this we switch from the VDJdb example to two subsets of isolated peptides from the IEDB: Influenza A- and Ebolavirus-derived peptides presented by HLA-A*02:01 and HLA-B*15:01, respectively [48]. Importantly, this analysis proceeds without an initial clustering step, and direct differences between the datasets are interrogated as-is. Our approach leverages the inherent position-sensitivity of the encoded information to build up a site-specific probability distribution (Fig 4A) for these peptide inputs. We see from these position-sensitive probability distributions that the expected anchors at P2 and the C-terminal positions (or P1 and P14 in the AIMS encoding of S8 Fig) have the strongest amino acid preference, as expected for these positions [53, 54]. We see in comparing these two datasets that only P2 leucine (20% enrichment) and P2 glutamine (16% enrichment) show up as distinct anchors for HLA-A*02 and HLA-B*15, respectively, because other strong anchors are shared between these two alleles. Due to the limited overlap in the C-terminal anchors, we see strong preferences towards HLA-B*15 peptides containing PΩ tyrosine and PΩ phenylalanine (15% and 22% enrichments), and a similar preference towards PΩ isoleucine and PΩ valine anchors for HLA-A*02 (21% and 20% enrichments). However, whereas conventional sequence logo plots can generate similar inferences, albeit via a more indirect comparison, our analysis takes these probability distributions a step further. First, from the position-sensitive probability distribution, we can calculate information theoretic metrics such as the Shannon entropy and the mutual information to quantify diversity and relationships among the peptide preferences for these repertoires. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Statistical and information theoretic AIMS analysis of Influenza A- and Ebolavirus-derived peptides. (A) Position sensitive amino acid frequency difference between the two peptide datasets. (B) Position sensitive Shannon entropy quantification and encoding coverage of the two peptide datasets. Statistically significant differences in the entropy (p < 0.05, calculated via non-parametric permutation test) are denoted by asterisks. The coverage applies to all position sensitive metrics, and highlights that differences in the entropy and mutual information at positions 5 and 9 are largely due to differences in coverage. (C) Position sensitive mutual information difference between the two peptide datasets. (D) Di-gram amino acid frequency difference calculated between the two peptide datasets. In all difference plots, a deeper shade of purple represents a higher quantity for the HLA-A*02 presented Influenza A peptide dataset, while a deeper shade of orange represents a higher quantity for the HLA-B*15 presented Ebolavirus peptide dataset. Raw distributions for each individual dataset (S9 Fig) and identification of statistically significant regions (S10 Fig) can be found in the supporting information. https://doi.org/10.1371/journal.pcbi.1011577.g004 We show the position-sensitive entropy for these two defined peptide populations as well as the sequence coverage resulting from the bulge encoding scheme (Fig 4B). Immediately, we see drops in the entropy in regions of high coverage corresponding to the anchor positions (AIMS P1 and P14). Further, we see that even with peptides derived from single viral clades, the diversity at the center of the peptides is nearly maximal, i.e. all 20 amino acids occurring at nearly equal probability. Given the arguments for the necessity of cross-reactivity in T cells [55], this massive diversity from singular datasets is perhaps unsurprising. The mutual information is a quantification of the decrease in uncertainty resulting from a known condition. In the case of amino acids it quantifies relationships between amino acid correlations in distinct regions of a sequence. Looking at the difference in the mutual information between the Influenza A- and Ebolavirus-derived peptides, we see strong trends of increased information between the N- and C-terminal ends of the Ebolavirus-derived peptides (Fig 4C). We further see that although only the Influenza A-derived peptides have a varied length distribution, leading to the wider central entropy peak, there is a clear signal in these long peptides, suggesting a specific subset of amino acid usage in the peptides that tend to be longer. Generally in immune repertoire analysis, mutual information may represent instances of co-evolution or receptor cross-talk, as has been discussed in analysis of polyreactive antibody sequences [36]. The goal of this information-theoretic analysis is to identify key regions of increased diversity, conservation, or crosstalk. As a final step, we can attempt to identify the source of the patterns in the short-range mutual information through the analysis of di-gram amino acid probabilities. Removing the position sensitivity of the peptides in each dataset, we can count the raw occurrence probabilities of each peptide di-gram using a sliding window, building up a probability distribution for each amino acid pair, where the order of occurrence matters (Fig 4D). Interestingly, although there are strong differences in the raw amino acid occurrence probabilities for each dataset (S8 Fig), the di-gram differences are often concentrated in particular regions. For instance, although valine and isoleucine are more frequently found in Influenza A dataset, the valine-isoleucine digram is more common in the Ebolavirus dataset. In addition to the standard analysis pipeline outlined here, the analysis can be extended to include N-gram motifs, providing the potential to identify regions with a propensity for certain tri-grams or higher-order motifs. Care must be taken when utilizing these N-gram formulations, however, as extension to the extreme such as in the analysis of nine-gram motifs for peptide datasets will identify statistically significant but not particularly meaningful data. Comparisons to existing software While the AIMS analysis pipeline has been developed to address more than just TCR repertoire analysis and clustering, it shares some features with existing software such as GLIPH [26] and TCRdist [24, 56]. These software packages aim to identify TCR sequences enriched above background populations or to cluster sequences with similar amino acid motifs, taking different approaches but generating comparable results to the AIMS clustering outlined above. For GLIPH, we compare our ability to identify distinct motifs using the standard AIMS analysis, whereas in comparing to TCRdist we more quantitatively compare the TCRdist “distance” metric to the corresponding AIMS distance between TCRs. As a representative motif comparison, we examine the output of GLIPH from Glanville et al. [26] as applied to the Influenza antigen M1 presented by HLA-A*02:01 (Fig 5). In addition to the benchmarking of the results of each software, we can compare and contrast the distinct paths to these results. One such difference between AIMS and GLIPH is that there is no “reference population” needed as input for AIMS. AIMS takes the identified M1-reactive sequences (Fig 5A), generates the previously discussed biophysical property matrix, and identifies biophysically distinct clusters in a projected space of these biophysical properties (Fig 5B). Unlike the more diverse clusters of Fig 2 resulting from the analysis of a broad input repertoire, this more targeted analysis of tetramer-sorted sequences results in more homogeneous clustered sequences (Fig 5C). From these clusters, we can then identify the key sequence motifs of each cluster and compare these to the GLIPH results (Fig 5D). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Comparison of AIMS TCR clustering analysis to GLIPH results. (A) The Influenza-reactive TCRs identified by Glanville et al. [26] are encoded into an AIMS matrix using the bulge-encoding. (B) Each sequence is then processed using the standard AIMS pipeline, and then projected onto two dimensions using UMAP and clustered using the DBSCAN algorithm. (C) These clusters are then re-visualized as an AIMS matrix. (D) Finally, the motifs identified by GLIPH can be directly compared to the motifs identified by AIMS via the clustering in panel C. Biophysical properties of each amino acid in the motif are colored according to the key, and an “X” in the AIMS motif represents “any amino acid with this biophysical property”, i.e. the orange “X” can represent S, T, G, or A. https://doi.org/10.1371/journal.pcbi.1011577.g005 AIMS identifies thirteen biophysically distinct clusters, three of which fully encapsulate the results of GLIPH. We note that while GLIPH identifies the motifs SIRS, IRS, and SIR as distinct, AIMS identifies these sequences in the single cluster SXRS. Further, coloring the sequences by a description of the most basic amino acid properties shows that many of the distinct GLIPH motifs are biophysically degenerate. While the GLIPH results seem to suggest that arginine is a requisite for recognition of the highly hydrophobic peptide GILGFVFTL, we see suggested arginine enrichment is relaxed in the AIMS results, instead suggesting the need for smaller amino acids (S, T, G, A) or nonpolar residues. Importantly, it is clear that a hydrophilic amino acid is well tolerated or in fact required for recognition, as six of the seven highlighted AIMS clusters have such a conserved residue in CDR3β. We can furthermore quantitatively compare the AIMS analysis to TCRdist, an analysis software that clusters and annotates inputs of large TCR repertoires based on similarity. The “distance” metric critical to the TCRdist pipeline provides a useful quantitative comparison point for our AIMS analysis. Importantly, this distance metric is fundamentally based upon the BLOSUM62 substitution matrix [57]. The BLOSUM62 substitution matrix implicitly encodes biophysical similarities and differences between amino acids, whereas the AIMS encoding explicitly encodes all of these biophysical properties in a single high-dimensional matrix. As such, we should expect similar, but not necessarily identical results when calculating the TCR distances using both TCRdist and AIMS. Using the mouse TCR repertoire data of Dash et al. [24], we first generated the AIMS encodings for either the CDR3 loops of the sequenced TCRs (Fig 6A) or for all six CDR loops of these same TCRs (Fig 6B) with the intention of matching the two main distance output options in TCRdist. We next generated the high-dimensional biophysical property matrices for each of these AIMS-encoded matrices, normalized each feature as discussed previously, and removed highly correlated vectors from each matrix. While the next step in the standard AIMS analysis pipeline is the reduction of these high dimensional matrices, the UMAP and PCA projections do not conserve the distances between points. As such, we instead calculate the raw Euclidean distances in the high-dimensional space, and directly compare these to the TCRdist values. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Quantitative comparison of distance metrics used in AIMS and TCRdist. Both the CDR3-only sequences (A) and the full-CDR sequences (B) of Dash et al. [24] are encoded into AIMS matrices. Sequence distances calculated via TCRdist and AIMS are then directly compared for these CDR3-only sequences (C) or full-CDR sequences (D) for the TCRα- and β-chains. Correlation coefficients between these distance metrics are reported for the full set of sequences and for closely related sequences, which are delineated by the dashed vertical lines. https://doi.org/10.1371/journal.pcbi.1011577.g006 From these panels, we see strong correlation for the CDR3-only distances (Fig 6C, ρα = 0.73 and ρβ = 0.72) and a stronger correlation for the full CDR loop calculations (Fig 6D, ρα = 0.83 and ρβ = 0.75). In both datasets, we see that as the TCRs get more dissimilar, the AIMS distances seem to level off even as the TCRdist metric continues to get larger. This may be due in part to well known issues with the calculation of distance metrics in high-dimensional spaces [58], but is likely not a point of concern in the standard AIMS analysis as clusters are generated in the projected spaces (i.e. in the UMAP or PCA projection). In recent applications of TCRdist [56], a distance cutoff of 20 has been used to define similar CDR3 loops of TCRs. Using this same definition and then a distance cutoff of 60 for all six CDR loop distances, we see that the correlation coefficients improve for more similar TCRs, with CDR3 (all CDR) correlation coefficients of 0.85 (0.86) for the α-chain and 0.85 (0.82) for the β chain. Comparisons with a wider range of antigens from Mayer-Blackwell et al. [56] show similar agreement (S11 Fig). These new AIMSdist metrics directly inspired by TCRdist can then be used to more quantitatively assess the comparison between AIMS and GLIPH (S12 Fig), highlighting that AIMS is able to recapitulate these results and provide added insights into more biophysically distinct sequence clusters. Thus, in searching through the high-dimensional biophysical space for the same distinct repertoire clusters as GLIPH and TCRdist, AIMS provides not just the identification of these clusters but also the biophysical similarities within these clusters; effectively providing the explanations for why sequences are grouped into the same clusters. Further, comparisons between clusters or even separate datasets can be made at a level deeper than the motif scale. In highlighting the functionality and unique strengths of AIMS, we set out to show that previous results from TCRdist and GLIPH can be reproduced, and that these results can subsequently be expanded upon with new analyses. Encoding amino acid sequences and their biophysical properties Although the ideal repertoire analysis would build off of complex structures either determined experimentally or predicted computationally, the former approach is inherently low-throughput while the latter is unreliable, even for the most advanced structural prediction software to date [28, 39]. The AIMS software, conversely, takes advantage of the structural conservation inherent to immune molecules, selecting out only the regions involved in the interaction interface. These conserved interacting regions, which are highly variable at the sequence level, are then aligned in matrix form using a pseudo-structural approach that varies across the available analysis modes for each molecular species. By incorporating general structural features, rather than explicit contact predictions, AIMS reduces the bias of the analysis by minimizing the reliance on assumptions of structural accuracy. Among TCR-peptide-MHC complexes, the interaction interface is strikingly similar, with crystal structures consistently finding nearly identical docking angles between the two [37, 40–44]. TCRs contact the peptide and the MHC α-helices via their six complementarity determining region (CDR) loops, which are in turn connected via stem regions to their well conserved framework regions. These stem regions adjacent to the CDR loops are never found within 5 Å of the antigen [26], and are easily identified by highly conserved amino acids, allowing for the exclusion of framework regions from the analysis. In a majority of structures, assuming the conserved stem regions are defined as endpoints, the central 4–5 residues of the CDR3 loops contact the central residues of the peptide [26] (Fig 1A and 1B). From these general structural rules, we can inform the encoding of TCR sequences into AIMS, imposing a “central” alignment scheme as the standard. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Example of AIMS encoding for the analysis of TCR-peptide interactions. (A) Rendering of a specific TCR-pMHC interaction (PDB ID: 1OGA) with TCRα shown in blue, TCRβ in orange, MHC in white. (B) Inset shows a zoom in on this TCR-peptide interface, with the MHC now translucent. Representative AIMS encoding of the single TCR CDR3β sequence (C, central-encoding) or the peptide sequence (D, bulge-encoding) in panel A. Below these single encodings are examples of full TCR repertoire (C) or immunopeptidome (D) AIMS-encoded matrices. Each amino acid in the structures, the single encoded sequences, and the matrices is represented by a unique color. https://doi.org/10.1371/journal.pcbi.1011577.g001 In the central alignment scheme, we align each sequence to the central residue of each CDR loop. Whereas most analysis tools segregate TCR sequences by length, thereby artificially segmenting the data, the central alignment scheme of AIMS allows for receptors of all lengths to be analyzed simultaneously while focusing on the key regions of the receptor. Due to the length differences of the TCR sequences in a given dataset, signals from the CDR stem regions will be averaged out, thus prioritizing signals from the center of the CDR loops. We can visualize an example of this encoding for a test dataset of paired TCRα and TCRβ sequences from the VDJ database [16] (Fig 1C). In this matrix, we can see that each amino acid is encoded as a unique number in the matrix 1 to 21, or a unique color in the figure, with padded zeros between CDR loops represented by white space in the figure. To control for potential artifacts introduced by this approach, analysis can be repeated with “left” or “right” alignment of the sequences, aligning to the N- or C-termini, respectively, of the given sequences (S1 Fig). The standard AIMS encoding for peptides is slightly different from this central TCR alignment scheme. For class I MHC, the flanking regions of bound peptides are frequently ‘buried’ as highly conserved anchor residues that bind to the MHC platform (Fig 1A and 1B). The majority of TCR contacts are made with the central regions of the peptides that bulge out of the MHC binding groove in the case of longer peptides [45]. However, exceptions to this paradigm may not be uncommon, with TCRs capable of contacting the often-buried peptide N-terminal residue [46] and the C-terminal residue potentially extending out of the MHC pocket [47]. Nonetheless, the length distribution of peptides presented by class I MHC is narrow [33], subverting some of the sequence length concerns present in the TCR analysis. As such, peptide encoding in AIMS adopts a “bulge” scheme. The bulge scheme aligns the N- and C-terminal residues to either edge of the matrix, along with a user-defined number of additional flanking residues. Zeros are padded between these flanking regions and the remaining residues are centrally aligned as in the case of the TCR sequences, again adopting the same numeric amino acid encoding scheme (Fig 1D). We can see clearly for this subset of HLA-A2 presented Influenza peptides taken from the Immune Epitope Database (IEDB) [48] the relative conservation at anchor position 2, compared to the variability at the center of the peptide sequences. Importantly, this bulge alignment can also be applied to TCR and antibody sequences, putting more focus on their conserved stem regions. AIMS is capable of analyzing other molecules with conserved structural features and localized interfacial heterogeneity, including antibodies [36], MHC and MHC-like molecules [37], and, more generally, any molecular subset that can be successfully aligned using existing multi-sequence alignment software [38] (S2 Fig). The generalized AIMS encoding scheme allows for any molecular biologist or bioinformatician to take advantage of the downstream biophysical characterization tools of AIMS for their application of interest. All downstream repertoire characterization follows from this initial encoding, and takes identical paths regardless of the immune repertoire under consideration (S3 Fig). In the following sections we will outline the distinct AIMS modules, applying them to data that best demonstrates the utility of the analyses we perform, rather than opting for a contiguous analysis to a single dataset. More extensive descriptions of AIMS input and output options are provided in the supporting information accompanying this manuscript. Unsupervised clustering of a TCR repertoire from an unsorted dataset To illustrate the implementation of dimensionality reduction and clustering modules in AIMS, we generate a new analysis from the paired-chain data derived from VDJdb [16]. These sequences are complete with metadata regarding their epitope specificity, MHC allele presentation, and the haplotype of the individual each receptor was isolated from, if it was naturally derived. As intuition and recent quantitative work have suggested [49], paired-chain TCR sequence data greatly increases the information content from a given repertoire sequencing experiment when compared to single chain sequencing. Using the dimensionality reduction and clustering modules, we can determine precisely how strongly the analysis changes upon inclusion of the CDR3α sequences (Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Comparison of the purity of receptor clustering using paired chain (left) or single chain CDR3B (right) data. Dimensionality reduction using UMAP, followed by density based OPTICS clustering subsects the data into biophysically similar paired chain (A) and single chain (B) receptors. The first ten of these clusters are then re-visualized in their AIMS-encoded matrix, with black lines marking different clusters (C, D). The antigen specificity of each of these clusters is then quantified by percentages (E, F), with the colors corresponding to specific peptides as shown in the key. https://doi.org/10.1371/journal.pcbi.1011577.g002 We first generate the sequence encoding and biophysical property matrices (see S1 Table for list of properties) for both the paired-chain dataset and the CDR3β-only dataset. From this biophysical property matrix, the sequences are projected onto a three-dimensional space using the uniform manifold approximation and projection (UMAP) dimensionality reduction algorithm [50], and subsequently clustered using the density-based OPTICS (Ordering Points To Identify the Clustering Structure) algorithm [51] (Fig 2A and 2B). It is important to note that, for improved clarity, the projections shown here remove four outlier sequences with a proline in CDR3α and CDR3β, a very rare amino acid in TCR CDR loops (see S4 Fig for projection with outliers). From the UMAP projection and OPTICS clustering of Fig 2A and 2B we can see that the number of distinct outlier populations is increased in the paired chain dataset, suggesting the identification of a higher number of biophysically distinct sequences. Visualizing just a subset of the clusters back as encoded matrices (Fig 2C and 2D) we see that this is likely due to the clustering of the paired chain data picking up unique motifs in both CDR3α and CDR3β, suggesting that these strong outliers can come from either chain. These clusters can then be analyzed for cluster membership compared to their original dataset identifiers, here based on the antigen recognized by each TCR (Fig 2E and 2F). Importantly, cluster purity can be measured relative to the metadata of choice, such as antigenic species source, allele of presenting MHC, organism haplotype, or virtually any other identifiable characteristic that can be encoded as metadata for the sample (S5 Fig). This visualization of antigen recognition for each cluster (Fig 2E and 2F) highlights subtle differences between the paired-chain and single-chain datasets. We see that despite a nearly equal number of clusters identified, the average cluster purity is higher for the paired chain data at 0.50 ± 0.28 when compared to the single chain data at 0.37 ± 0.24, although not significantly so. We notice however that the few pure or close to pure clusters are coming from the same two antigens, NLVPMVATV and LLWNGPMAV, largely because these antigens make up 43% and 23% of the total dataset, respectively. Overall, these results show that while unique CDR3α motifs are critically important for antigen recognition and for understanding the full breadth of receptor diversity, a fairly accurate picture of sequence diversity and similarity can still be generated from CDR3β sequences alone. Going beyond receptor clustering and motif analysis The biophysical analysis in this and the following section can be carried out with individual clusters of sequences derived from unstructured data, as outlined in the previous section, or from antigenically well defined populations (see S6 Fig or Boughter et al. [36] for examples). In highlighting the features of the biophysical property analysis, we select the two antigenically purest clusters of Fig 2 from both the paired-chain and single chain datasets with specificity to the NLVPMVATV and LLWNGPMAV antigens (Fig 3A and 3B). From these cluster subsets, we can appreciate that despite having nearly the same antigenic purity in each of these clusters, the paired chain clusters surprisingly display increased CDR3β diversity. While we are able to generate sequence logos for the distinct motifs in these clusters (S7A and S7B Fig), the AIMS analysis pipeline permits a more thorough biophysical characterization of these clusters. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Isolation of individual sequence clusters and subsequent position-sensitive biophysical characterization of these sequences highlights the details provided by AIMS analysis. A subset of clusters identified in Fig 2 are isolated and shown as their AIMS matrix encoding for the paired chain (A) and single-chain (B) datasets. From these encodings, we can calculate the position sensitive biophysical properties for each cluster (C, D). Opaque lines with dots represent the averages over each cluster, while wider translucent regions centered on these lines give the variance as calculated by a bootstrapping procedure (Methods). Statistically significant differences (p < 0.05, calculated via non-parametric permutation test) are denoted by asterisks, with an asterisk over a solid bar representing extended regions of statistically significant differences. https://doi.org/10.1371/journal.pcbi.1011577.g003 Specifically, from the high-dimensional biophysical property matrix for each of these clusters, we can isolate single property masks (S7C and S7D Fig) which can be averaged across repertoires (Fig 3C and 3D) or across repertoires and positions (S7E and S7F Fig), generating position sensitive or net average biophysical properties for the molecular subsets of interest. These biophysical property visualizations can be used to more carefully compare and contrast the clusters generated from the paired-chain and single chain datasets. We see from the position-sensitive biophysical property averages that the physical properties of CDR3β are matched in the paired-chain and the single-chain clusters, despite the significant difference in diversity in this region. The positively charged segment of CDR3β in TCRs recognizing the LLWNGPMAV peptide is seen in both datasets, while a corresponding negative segment is found in the CDR3β of TCRs recognizing the NLVPMVATV peptide. Given the lack of charged residues in the peptides, the conservation of charge in these pure clusters is somewhat surprising. Conversely, the hydropathy score of the CDR3β chains is far more variable within the paired-chain clusters, although the same general trend is again followed when compared to the single-chain data, with two distinct peaks in hydropathy and a region neither lacking nor enriched in hydrophobic residues. This is a common feature in the clustering of sequences in AIMS, and may reflect the nature of the hydrophobicity metrics used, or alternatively could be suggestive of the critical determinants of the formation of TCR-pMHC complexes. While charges in the interface must be delicately arranged to form interfacial interactions and offset the significant energetic penalties of desolvation, more hydrophobic residues are free to effectively “fill in the blanks” and pack as best they can. While only the position sensitive charge and hydropathy are highlighted in this section, any of the 61 standard AIMS properties can be visualized either as position-sensitive averages across receptors, or as net averages over position- and sequence-space. Generating quantitative metrics of repertoire diversity and amino acid patterning While the biophysical characterization of immune repertoires can help answer questions regarding the mechanisms of immune recognition, complementary analysis can help contextualize these findings, relating biophysical properties back to patterns in the data resulting from the source of the molecules (e.g. human, mouse, or virus) or other external influences (e.g. selection or affinity maturation). In AIMS, we can further quantify entire repertoires, cluster molecular subsets, or define antigenic groups using an array of statistical and information theoretic approaches. Information theory is built for the analysis of sequences of inputs and outputs [52]. Here, these inputs and outputs are the amino acids making up our immune repertoires quantified via the observed probability distributions across these sequences. In telecommunication, quantification of inputs and outputs determine the messages which can be sent over a given channel, whereas in the study of immune receptors this same quantification determines the range of pathogenic targets which can be recognized by a given immune system. To illustrate this we switch from the VDJdb example to two subsets of isolated peptides from the IEDB: Influenza A- and Ebolavirus-derived peptides presented by HLA-A*02:01 and HLA-B*15:01, respectively [48]. Importantly, this analysis proceeds without an initial clustering step, and direct differences between the datasets are interrogated as-is. Our approach leverages the inherent position-sensitivity of the encoded information to build up a site-specific probability distribution (Fig 4A) for these peptide inputs. We see from these position-sensitive probability distributions that the expected anchors at P2 and the C-terminal positions (or P1 and P14 in the AIMS encoding of S8 Fig) have the strongest amino acid preference, as expected for these positions [53, 54]. We see in comparing these two datasets that only P2 leucine (20% enrichment) and P2 glutamine (16% enrichment) show up as distinct anchors for HLA-A*02 and HLA-B*15, respectively, because other strong anchors are shared between these two alleles. Due to the limited overlap in the C-terminal anchors, we see strong preferences towards HLA-B*15 peptides containing PΩ tyrosine and PΩ phenylalanine (15% and 22% enrichments), and a similar preference towards PΩ isoleucine and PΩ valine anchors for HLA-A*02 (21% and 20% enrichments). However, whereas conventional sequence logo plots can generate similar inferences, albeit via a more indirect comparison, our analysis takes these probability distributions a step further. First, from the position-sensitive probability distribution, we can calculate information theoretic metrics such as the Shannon entropy and the mutual information to quantify diversity and relationships among the peptide preferences for these repertoires. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Statistical and information theoretic AIMS analysis of Influenza A- and Ebolavirus-derived peptides. (A) Position sensitive amino acid frequency difference between the two peptide datasets. (B) Position sensitive Shannon entropy quantification and encoding coverage of the two peptide datasets. Statistically significant differences in the entropy (p < 0.05, calculated via non-parametric permutation test) are denoted by asterisks. The coverage applies to all position sensitive metrics, and highlights that differences in the entropy and mutual information at positions 5 and 9 are largely due to differences in coverage. (C) Position sensitive mutual information difference between the two peptide datasets. (D) Di-gram amino acid frequency difference calculated between the two peptide datasets. In all difference plots, a deeper shade of purple represents a higher quantity for the HLA-A*02 presented Influenza A peptide dataset, while a deeper shade of orange represents a higher quantity for the HLA-B*15 presented Ebolavirus peptide dataset. Raw distributions for each individual dataset (S9 Fig) and identification of statistically significant regions (S10 Fig) can be found in the supporting information. https://doi.org/10.1371/journal.pcbi.1011577.g004 We show the position-sensitive entropy for these two defined peptide populations as well as the sequence coverage resulting from the bulge encoding scheme (Fig 4B). Immediately, we see drops in the entropy in regions of high coverage corresponding to the anchor positions (AIMS P1 and P14). Further, we see that even with peptides derived from single viral clades, the diversity at the center of the peptides is nearly maximal, i.e. all 20 amino acids occurring at nearly equal probability. Given the arguments for the necessity of cross-reactivity in T cells [55], this massive diversity from singular datasets is perhaps unsurprising. The mutual information is a quantification of the decrease in uncertainty resulting from a known condition. In the case of amino acids it quantifies relationships between amino acid correlations in distinct regions of a sequence. Looking at the difference in the mutual information between the Influenza A- and Ebolavirus-derived peptides, we see strong trends of increased information between the N- and C-terminal ends of the Ebolavirus-derived peptides (Fig 4C). We further see that although only the Influenza A-derived peptides have a varied length distribution, leading to the wider central entropy peak, there is a clear signal in these long peptides, suggesting a specific subset of amino acid usage in the peptides that tend to be longer. Generally in immune repertoire analysis, mutual information may represent instances of co-evolution or receptor cross-talk, as has been discussed in analysis of polyreactive antibody sequences [36]. The goal of this information-theoretic analysis is to identify key regions of increased diversity, conservation, or crosstalk. As a final step, we can attempt to identify the source of the patterns in the short-range mutual information through the analysis of di-gram amino acid probabilities. Removing the position sensitivity of the peptides in each dataset, we can count the raw occurrence probabilities of each peptide di-gram using a sliding window, building up a probability distribution for each amino acid pair, where the order of occurrence matters (Fig 4D). Interestingly, although there are strong differences in the raw amino acid occurrence probabilities for each dataset (S8 Fig), the di-gram differences are often concentrated in particular regions. For instance, although valine and isoleucine are more frequently found in Influenza A dataset, the valine-isoleucine digram is more common in the Ebolavirus dataset. In addition to the standard analysis pipeline outlined here, the analysis can be extended to include N-gram motifs, providing the potential to identify regions with a propensity for certain tri-grams or higher-order motifs. Care must be taken when utilizing these N-gram formulations, however, as extension to the extreme such as in the analysis of nine-gram motifs for peptide datasets will identify statistically significant but not particularly meaningful data. Comparisons to existing software While the AIMS analysis pipeline has been developed to address more than just TCR repertoire analysis and clustering, it shares some features with existing software such as GLIPH [26] and TCRdist [24, 56]. These software packages aim to identify TCR sequences enriched above background populations or to cluster sequences with similar amino acid motifs, taking different approaches but generating comparable results to the AIMS clustering outlined above. For GLIPH, we compare our ability to identify distinct motifs using the standard AIMS analysis, whereas in comparing to TCRdist we more quantitatively compare the TCRdist “distance” metric to the corresponding AIMS distance between TCRs. As a representative motif comparison, we examine the output of GLIPH from Glanville et al. [26] as applied to the Influenza antigen M1 presented by HLA-A*02:01 (Fig 5). In addition to the benchmarking of the results of each software, we can compare and contrast the distinct paths to these results. One such difference between AIMS and GLIPH is that there is no “reference population” needed as input for AIMS. AIMS takes the identified M1-reactive sequences (Fig 5A), generates the previously discussed biophysical property matrix, and identifies biophysically distinct clusters in a projected space of these biophysical properties (Fig 5B). Unlike the more diverse clusters of Fig 2 resulting from the analysis of a broad input repertoire, this more targeted analysis of tetramer-sorted sequences results in more homogeneous clustered sequences (Fig 5C). From these clusters, we can then identify the key sequence motifs of each cluster and compare these to the GLIPH results (Fig 5D). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Comparison of AIMS TCR clustering analysis to GLIPH results. (A) The Influenza-reactive TCRs identified by Glanville et al. [26] are encoded into an AIMS matrix using the bulge-encoding. (B) Each sequence is then processed using the standard AIMS pipeline, and then projected onto two dimensions using UMAP and clustered using the DBSCAN algorithm. (C) These clusters are then re-visualized as an AIMS matrix. (D) Finally, the motifs identified by GLIPH can be directly compared to the motifs identified by AIMS via the clustering in panel C. Biophysical properties of each amino acid in the motif are colored according to the key, and an “X” in the AIMS motif represents “any amino acid with this biophysical property”, i.e. the orange “X” can represent S, T, G, or A. https://doi.org/10.1371/journal.pcbi.1011577.g005 AIMS identifies thirteen biophysically distinct clusters, three of which fully encapsulate the results of GLIPH. We note that while GLIPH identifies the motifs SIRS, IRS, and SIR as distinct, AIMS identifies these sequences in the single cluster SXRS. Further, coloring the sequences by a description of the most basic amino acid properties shows that many of the distinct GLIPH motifs are biophysically degenerate. While the GLIPH results seem to suggest that arginine is a requisite for recognition of the highly hydrophobic peptide GILGFVFTL, we see suggested arginine enrichment is relaxed in the AIMS results, instead suggesting the need for smaller amino acids (S, T, G, A) or nonpolar residues. Importantly, it is clear that a hydrophilic amino acid is well tolerated or in fact required for recognition, as six of the seven highlighted AIMS clusters have such a conserved residue in CDR3β. We can furthermore quantitatively compare the AIMS analysis to TCRdist, an analysis software that clusters and annotates inputs of large TCR repertoires based on similarity. The “distance” metric critical to the TCRdist pipeline provides a useful quantitative comparison point for our AIMS analysis. Importantly, this distance metric is fundamentally based upon the BLOSUM62 substitution matrix [57]. The BLOSUM62 substitution matrix implicitly encodes biophysical similarities and differences between amino acids, whereas the AIMS encoding explicitly encodes all of these biophysical properties in a single high-dimensional matrix. As such, we should expect similar, but not necessarily identical results when calculating the TCR distances using both TCRdist and AIMS. Using the mouse TCR repertoire data of Dash et al. [24], we first generated the AIMS encodings for either the CDR3 loops of the sequenced TCRs (Fig 6A) or for all six CDR loops of these same TCRs (Fig 6B) with the intention of matching the two main distance output options in TCRdist. We next generated the high-dimensional biophysical property matrices for each of these AIMS-encoded matrices, normalized each feature as discussed previously, and removed highly correlated vectors from each matrix. While the next step in the standard AIMS analysis pipeline is the reduction of these high dimensional matrices, the UMAP and PCA projections do not conserve the distances between points. As such, we instead calculate the raw Euclidean distances in the high-dimensional space, and directly compare these to the TCRdist values. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Quantitative comparison of distance metrics used in AIMS and TCRdist. Both the CDR3-only sequences (A) and the full-CDR sequences (B) of Dash et al. [24] are encoded into AIMS matrices. Sequence distances calculated via TCRdist and AIMS are then directly compared for these CDR3-only sequences (C) or full-CDR sequences (D) for the TCRα- and β-chains. Correlation coefficients between these distance metrics are reported for the full set of sequences and for closely related sequences, which are delineated by the dashed vertical lines. https://doi.org/10.1371/journal.pcbi.1011577.g006 From these panels, we see strong correlation for the CDR3-only distances (Fig 6C, ρα = 0.73 and ρβ = 0.72) and a stronger correlation for the full CDR loop calculations (Fig 6D, ρα = 0.83 and ρβ = 0.75). In both datasets, we see that as the TCRs get more dissimilar, the AIMS distances seem to level off even as the TCRdist metric continues to get larger. This may be due in part to well known issues with the calculation of distance metrics in high-dimensional spaces [58], but is likely not a point of concern in the standard AIMS analysis as clusters are generated in the projected spaces (i.e. in the UMAP or PCA projection). In recent applications of TCRdist [56], a distance cutoff of 20 has been used to define similar CDR3 loops of TCRs. Using this same definition and then a distance cutoff of 60 for all six CDR loop distances, we see that the correlation coefficients improve for more similar TCRs, with CDR3 (all CDR) correlation coefficients of 0.85 (0.86) for the α-chain and 0.85 (0.82) for the β chain. Comparisons with a wider range of antigens from Mayer-Blackwell et al. [56] show similar agreement (S11 Fig). These new AIMSdist metrics directly inspired by TCRdist can then be used to more quantitatively assess the comparison between AIMS and GLIPH (S12 Fig), highlighting that AIMS is able to recapitulate these results and provide added insights into more biophysically distinct sequence clusters. Thus, in searching through the high-dimensional biophysical space for the same distinct repertoire clusters as GLIPH and TCRdist, AIMS provides not just the identification of these clusters but also the biophysical similarities within these clusters; effectively providing the explanations for why sequences are grouped into the same clusters. Further, comparisons between clusters or even separate datasets can be made at a level deeper than the motif scale. In highlighting the functionality and unique strengths of AIMS, we set out to show that previous results from TCRdist and GLIPH can be reproduced, and that these results can subsequently be expanded upon with new analyses. Discussion Immune molecules and the pathogens which they protect against represent a unique case study for bioinformatic analysis approaches. The polymorphic regions of TCRs, antibodies, and MHC molecules in humans are concentrated in select regions of these proteins, specifically in their key intermolecular interaction sites, while the rest of their three-dimensional structures are exceptionally well conserved. This structural conservation and localized variability appears to point towards molecular modeling as a key tool, but modern computational approaches are either too costly i.e., slow and inefficient, or too inaccurate to allow for detailed conclusions regarding the proximity of specific amino acids [28, 39, 59]. Many of the best performing machine learning approaches are “black box” algorithms that do not allow the user to determine how or why certain classifications are made. While more interpretable structural modeling approaches are capable of approximate placement of TCR protein backbones on the pMHC surface, even the best structure prediction software struggles to place receptor side chains in the proper position on the antigenic surface [28, 39]. Proper placement of these side chains, to angstrom-level precision, is key for proper inference of the interaction strength between receptors and their cognate antigens. AIMS was developed specifically as an interpretable tool for immune repertoire analysis, capable of utilizing the information provided by conserved structural features of immune molecules for unique analytical approaches while reducing the potential for errors arising from insufficient resolution offered by structural predictions. The generality of the AIMS analysis pipeline allows for the simultaneous characterization of all adaptive immune molecules, including peptides, conserved viral structures, antibodies [36], MHC molecules [37], T cell receptors, and broadly any protein subset with structurally conserved features and localized diversity [38]. Currently the standard AIMS pipeline analyzes each of these molecular subsets individually, and subsequently allows users to compare and contrast biophysical features of paired TCR-pMHC or antibody-antigen interactions. Recently developed features and further ongoing work is aimed at incorporating mixed repertoire analysis from the pipeline initialization. These features will provide insights into the intricacies and inter-relationships of all of the molecular players involved in an adaptive immune response. Of the existing software packages available for the analysis of molecular subsets of adaptive immunity, T cell receptor repertoire analysis is the most mature of the fields, and therefore serves as the primary point of comparison for AIMS. Despite not being developed explicitly for T cell receptor analysis, we find that AIMS can reproduce the results of two of the most popular programs developed for TCR analysis, namely GLIPH and TCRdist [24, 26, 56]. GLIPH and TCRdist both utilize distance based metrics to identify unique motifs from TCR repertoire datasets. Extending sequence analysis beyond clustering, AIMS quantifies the biophysical differences between identified clusters of sequences to determine the various approaches TCRs take to recognize single antigenic targets. In the analysis of large experimental datasets, users have the option of clustering using GLIPH or TCRdist, and subsequently importing these sequences into AIMS for the downstream analysis, providing an opportunity to extend the analysis from where other approaches conclude. One key AIMS approach not discussed in this manuscript is a more targeted, supervised learning approach that is also available for the identification of specific differences in distinct repertoire datasets. This supervised learning approach utilizes a linear discriminant analysis (LDA)-based classifier to simultaneously sort individual sequences into their respective classes and identify key features that delineate distinct repertoires. While other algorithms may perform better in classifying repertoire data, the strength of LDA lies in its interpretability. Unlike other machine learning methods, the vectors most important for discriminating between distinct repertoires are included as output, along with their associated linear weights. Further details on linear discriminant analysis can be found in the extended methods. In this targeted supervised approach, users immediately identify key position-sensitive differences between their datasets. AIMS fundamentally is built around the identification of biophysically distinct molecular subsets. Independent of any biases in the experimental approaches used, AIMS aims to find the receptor subsets that most strongly span the biophysical space of the input dataset. While still susceptible to some of the issues that plague all analysis in the regime of strong undersampling, by focusing on those receptors that are the most biophysically distinct, rather than those that are the most similar according to a variety of distance metrics, we can identify the limits of molecular recognition. This may under-emphasize the importance of convergence towards certain motifs in immune responses, but it enhances our understanding of the diverse paths adaptive immune systems take towards solving the same problems of pathogenic recognition. It is important to note that the analytical tools provided by AIMS are best utilized both as a means of identifying differences between datasets and as an exploratory tool. Many of the decisions made at each stage of the analysis can alter the downstream interpretations, so users are encouraged to test different alignments, projection methods, clustering algorithms, and clustering options. While choices in alignment do not strongly alter the underlying structure of the input dataset (S12 Fig and S2 Table), they can generate differential emphasis on distinct features. Further, as seen in S4 Fig, the inclusion or exclusion of certain sequences can distort the projected spaces used for sequence clustering. Thorough investigation should include multiple iterations of analysis, testing how single or paired chain data alter outputs, and how inclusion of mixed or single antigenic specificities in a given AIMS run can provide new and exciting insights. Importantly, AIMS can be easily extended by adding code with the desired additional functionalities, building on the algorithms already present in the software. Materials and methods This section serves as a reference for reproducing the analyses used to create the figures in this manuscript. For a more conceptual overview, readers are referred to the Extended Methods section. For a more practical discussion of the implementation of AIMS using either the graphical user interface (GUI) for non-computational researchers or the Jupyter notebooks and command line interface (CLI) for more advanced users, we direct the reader to download the code through GitHub [https://github.com/ctboughter/AIMS] and follow the walkthroughs available at [https://aims-doc.readthedocs.io/en/latest/]. AIMS encoding All sequencing data is first processed into an AIMS-readable format, a simple comma-separated value file with each column corresponding to each structural feature (CDR loops for TCRs, α-helices and β-strands for MHC, or specific regions of interest for multi-sequence alignments). These files are then read into AIMS, parsing out sequences with missing residues, improper characters, or fewer structural features of interest than those defined by the user. The sequences are then aligned according to user input. For datasets with multiple structural features, alignment is performed independently on each feature. Only “Central” and “Bulge” alignment strategies require special consideration, as both deal with an alignment to a sequence center. The “center” of sequences with an even number of amino acids is chosen to be the amino acid preceding the midway point of the sequence. “Bulge” alignments require additional input specifying the total number of amino acids “padded” on either side of the centrally aligned region. A pad length of 6 (3 AA of the N- and C-termini) was used for peptide analysis of Fig 4, while a pad length of 8 (4 AA of the N- and C- termini) was used for Fig 5. Generation of biophysical property matrices The initial AIMS encoding is used as the template for all downstream analysis. If the sequence encoding matrix utilizes a bulge scheme, then all resultant position-sensitive figures adopt this same alignment. This is accomplished via a simple dictionary, where each amino acid is associated with 62 other values (1 value for positional encoding visualization and 61 for biophysical properties) to generate an i x j x k property matrix. Importantly, a Z-score normalization transforms each individual biophysical property in our dictionary, not the given dataset. Propensities for biophysical interactions can optionally be scored using the pairwise interaction scores of S3 Table. In the analysis throughout this manuscript, vectors with a correlation coefficient above 0.75 with another vector, and all empty vectors (i.e. those corresponding to white space in the positional matrices) are dropped from the biophysical property matrix. Biophysical property measurements (like position sensitive charge, net hydrophobicity, etc.) utilize the full, non-parsed matrices, whereas all projection and clustering is done on these parsed matrices. Dimensionality reduction and unsupervised clustering Dimensionality reduction modules to collapse the high-dimensional biophysical property matrices using either principal component analysis (PCA) or uniform manifold approximation and projection (UMAP) utilize the SciKit-learn [60] and UMAP [50] Python packages. The parsed biophysical property matrices discussed in the previous section are first subject to dimensionality reduction with specified parameters of n_components = 3 for both UMAP and PCA, svd_solver = full for PCA, and n_neighbors = 25 for UMAP. These parameters are defaults in AIMS but can be changed by the user. For the purposes of reproducibility, the UMAP random seed was set to 617 for all projections in this manuscript. A discussion of reproducibility when using UMAP can be found within both the AIMS and UMAP Read the Docs pages. All other parameters are SciKit-Learn defaults. The outputs from these projection algorithms are then fed into either the OPTICS (ordering points to identify the clustering structure) [51] or DBSCAN (density-based spatial clustering of applications with noise) [61] algorithms. For all clustering in this manuscript, the default AIMS specified parameters are used; min_samples = 10 for OPTICS and eps = 0.15 for DBSCAN. All other parameters are SciKit-Learn defaults. It is important to note that these two parameters should typically be the most variable across applications of AIMS, with the “proper” settings varying strongly across different projection algorithms and input datasets. Information theoretic calculations Information theory, a theory classically applied to communication across noisy channels, is incredibly versatile in its applications, with high potential for further applications in immunology [52, 62–66]. In AIMS, we utilize two powerful concepts from information theory, namely Shannon entropy and mutual information. Shannon entropy, in its simplest form, can be used as a proxy for the diversity in a given input population. This entropy, denoted as H, has the general form: (1) Where p(x) is the occurrence probability of a given event, and X is the set of all events. We can then calculate this entropy at every position across AIMS-encoded matrices, where X is the set of all amino acids, and p(x) is the probability of seeing a specific amino acid at the given position. In other words, we want to determine, for a given site in the AIMS matrix, how much diversity (or entropy) is present. Given there are only 20 amino acids used in naturally derived sequences, we can calculate a theoretical maximum entropy of 4.32 bits, which assumes that every amino acid occurs at a given position with equal probability. Importantly, from this entropy we can calculate an equally interesting property of the dataset, namely the mutual information. Mutual information is similar, but not identical to, correlation. Whereas correlations are required to be linear, if two amino acids vary in any linked way, this will be reflected as an increase in mutual information. In AIMS, mutual information I(X; Y) is calculated by subtracting the Shannon entropy described above from the conditional Shannon entropy H(X|Y) at each given position as seen in Eqs 2 and 3: (2) (3) Putting these equations into words, we are effectively asking how the knowledge of the identity of an amino acid at one site changes the entropy at another site. If the entropy at the “test site” is zero, i.e. H(X) = 0, then no matter what we know of the amino acid identity at another site the change in entropy at the test site will still be zero, and therefore the mutual information will be zero. Likewise, if the entropy remains unchanged at this test site despite the knowledge of the amino acid identity at another site, the mutual information will again be zero. There is only a meaningful mutual information between a test site and a given amino acid site if the knowledge of that given amino acid reduces the entropy at the test site. The mutual information cannot be negative, so the reverse situation, i.e. an increase in diversity with the knowledge of amino acid identity at a given site, cannot occur. Statistical considerations in AIMS The position sensitive averages in AIMS in particular are critical for comparing repertoires from distinct sources or subsets of repertoires, and are capable of identifying different modes of recognition for these molecules. Importantly, these averages are difficult to directly compare statistically, as they are not normally distributed due to the discrete nature of the 20 amino acids composing protein sequences. As such, the AIMS standard for plotting properties of these repertoires is to bootstrap a normal distribution of receptor averages, with the bootstrapped average and the bootstrapped standard deviation plotted in the final figure. The bootsrapped distribution is sampled 1000 times as a default. Statistical significance in AIMS is then calculated using either a two-sided nonparametric Studentized bootstrap or a two-sided nonparametric permutation test as outlined in “Bootstrap Methods and Their Application” [67]. In this manuscript, the two-sided nonparametric permutation test was exclusively utilized to calculate statistical significance. Here, the test statistic z is set to a simple difference of means, and we randomly permute the data into two bins. We then count the number of permutations where the randomly permuted test statistic is greater than or equal to the empirical test statistic. The p-value is then calculated as: (4) where z is the permuted test statistic and z0 is the empirical test statistic. R is then the number of tested permutations, here 1000, and is the count of permutations where the square of the permuted test statistic is greater than the square of the empirical test statistic. Assorted p-value cutoffs are reported within the figure legends throughout the manuscript. Generation of simulated TCR Datasets To benchmark quantitative comparisons between different implementations of the AIMS analysis and existing software such as GLIPH and TCRdist, we created a new AIMS module for the generation of simulated TCR datasets. Results of these comparisons can be found in S2 Table. These simulated datasets are generated via a random selection of human V- and J-gene segments, followed by a random deletion of 0–2 amino acids from these selected segments. From there, user inputs determine the number of randomly added amino acids and the probability distributions of these amino acids. It is important to note that the V- and J-gene selection probabilities, the deletion probabilities, and the added amino acid probabilities are pseudo-randomly generated, and are not meant to match biological frequencies. Our test simulated dataset was comprised of 15,000 total receptors, from three discrete simulated datasets of 5,000 receptors with lengths varying from 11–14 amino acids. The three datasets are named after the amino acid insertion distributions they draw from, with all three datasets excluding cysteine and proline from the distribution. The “Random” dataset sets the probability of insertion of all other amino acids to 1/18. The “KRQN” dataset sets the probability of insertion 20-fold more likely for positive charged amino acids K and R and 10-fold more likely for hydrophilic amino acids Q and N. Likewise, the “DEHY” dataset sets the probability of insertion 20-fold more likely for negatively charged amino acids D and E and 10-fold more likely for amphipathic amino acids H and Y. Such strong trends should generate relatively clean separations using any clustering approach. This list of 15,000 single chain sequences from these three generated datasets are used as the input for each analysis. In AIMS analysis, the “Standard” approach uses the central encoding scheme, vector normalization and an entropy re-weighting of the biophysical property matrix. This matrix is then projected onto 3 UMAP dimensions and clustered using the DBSCAN algorithm. The remaining entries in S2 Table are deviations from this standard, with the analysis name highlighting which step is changed. So “AIMS Left” utilizes the left alignment scheme, while “AIMS PCA-Kmeans” utilizes the PCA for projection of the data and Kmeans for clustering. TCRdist and AIMSdist clusters are determined using a hierarchical clustering approach with a cutoff of 30 TCRdist units and 4 AIMSdist units. GLIPH clusters are defined by the identified statistically significant motifs with a length of 3 or more and 10 or more sequences per motif, as the clustering algorithm did not converge after 48 hours of continuous calculation. It should be noted that a true metric of “clustering success” is difficult to quantitatively determine. As such we report a range of statistics for each analysis. For instance, while hierarchical clustering of TCRdist and AIMSdist metrics give nearly 100% cluster purity, the large number of clusters (nearly 400 for AIMSdist and over 500 for TCRdist) may make these results hard to parse. Further, the number of clusters per dataset is unevenly weighted. We note that while AIMS standard analysis cluster purity appears to be low (75%) the majority of the “contaminants” are from the randomly generated dataset, which likely includes sequences enriched in positive or negative charge. In a way, such “impurities” may be desired in the analysis of especially TCRs, as crossreactivity may make it likely that TCRs from distinct datasets have similar biophysical properties. As a final note, GLIPH performs the worst of nearly all analyses, clustering only 28% of the sequences with single sequences belonging to multiple clusters. This is likely because GLIPH is not meant for the analysis of simulated data, making such comparisons inherently unfair. AIMS encoding All sequencing data is first processed into an AIMS-readable format, a simple comma-separated value file with each column corresponding to each structural feature (CDR loops for TCRs, α-helices and β-strands for MHC, or specific regions of interest for multi-sequence alignments). These files are then read into AIMS, parsing out sequences with missing residues, improper characters, or fewer structural features of interest than those defined by the user. The sequences are then aligned according to user input. For datasets with multiple structural features, alignment is performed independently on each feature. Only “Central” and “Bulge” alignment strategies require special consideration, as both deal with an alignment to a sequence center. The “center” of sequences with an even number of amino acids is chosen to be the amino acid preceding the midway point of the sequence. “Bulge” alignments require additional input specifying the total number of amino acids “padded” on either side of the centrally aligned region. A pad length of 6 (3 AA of the N- and C-termini) was used for peptide analysis of Fig 4, while a pad length of 8 (4 AA of the N- and C- termini) was used for Fig 5. Generation of biophysical property matrices The initial AIMS encoding is used as the template for all downstream analysis. If the sequence encoding matrix utilizes a bulge scheme, then all resultant position-sensitive figures adopt this same alignment. This is accomplished via a simple dictionary, where each amino acid is associated with 62 other values (1 value for positional encoding visualization and 61 for biophysical properties) to generate an i x j x k property matrix. Importantly, a Z-score normalization transforms each individual biophysical property in our dictionary, not the given dataset. Propensities for biophysical interactions can optionally be scored using the pairwise interaction scores of S3 Table. In the analysis throughout this manuscript, vectors with a correlation coefficient above 0.75 with another vector, and all empty vectors (i.e. those corresponding to white space in the positional matrices) are dropped from the biophysical property matrix. Biophysical property measurements (like position sensitive charge, net hydrophobicity, etc.) utilize the full, non-parsed matrices, whereas all projection and clustering is done on these parsed matrices. Dimensionality reduction and unsupervised clustering Dimensionality reduction modules to collapse the high-dimensional biophysical property matrices using either principal component analysis (PCA) or uniform manifold approximation and projection (UMAP) utilize the SciKit-learn [60] and UMAP [50] Python packages. The parsed biophysical property matrices discussed in the previous section are first subject to dimensionality reduction with specified parameters of n_components = 3 for both UMAP and PCA, svd_solver = full for PCA, and n_neighbors = 25 for UMAP. These parameters are defaults in AIMS but can be changed by the user. For the purposes of reproducibility, the UMAP random seed was set to 617 for all projections in this manuscript. A discussion of reproducibility when using UMAP can be found within both the AIMS and UMAP Read the Docs pages. All other parameters are SciKit-Learn defaults. The outputs from these projection algorithms are then fed into either the OPTICS (ordering points to identify the clustering structure) [51] or DBSCAN (density-based spatial clustering of applications with noise) [61] algorithms. For all clustering in this manuscript, the default AIMS specified parameters are used; min_samples = 10 for OPTICS and eps = 0.15 for DBSCAN. All other parameters are SciKit-Learn defaults. It is important to note that these two parameters should typically be the most variable across applications of AIMS, with the “proper” settings varying strongly across different projection algorithms and input datasets. Information theoretic calculations Information theory, a theory classically applied to communication across noisy channels, is incredibly versatile in its applications, with high potential for further applications in immunology [52, 62–66]. In AIMS, we utilize two powerful concepts from information theory, namely Shannon entropy and mutual information. Shannon entropy, in its simplest form, can be used as a proxy for the diversity in a given input population. This entropy, denoted as H, has the general form: (1) Where p(x) is the occurrence probability of a given event, and X is the set of all events. We can then calculate this entropy at every position across AIMS-encoded matrices, where X is the set of all amino acids, and p(x) is the probability of seeing a specific amino acid at the given position. In other words, we want to determine, for a given site in the AIMS matrix, how much diversity (or entropy) is present. Given there are only 20 amino acids used in naturally derived sequences, we can calculate a theoretical maximum entropy of 4.32 bits, which assumes that every amino acid occurs at a given position with equal probability. Importantly, from this entropy we can calculate an equally interesting property of the dataset, namely the mutual information. Mutual information is similar, but not identical to, correlation. Whereas correlations are required to be linear, if two amino acids vary in any linked way, this will be reflected as an increase in mutual information. In AIMS, mutual information I(X; Y) is calculated by subtracting the Shannon entropy described above from the conditional Shannon entropy H(X|Y) at each given position as seen in Eqs 2 and 3: (2) (3) Putting these equations into words, we are effectively asking how the knowledge of the identity of an amino acid at one site changes the entropy at another site. If the entropy at the “test site” is zero, i.e. H(X) = 0, then no matter what we know of the amino acid identity at another site the change in entropy at the test site will still be zero, and therefore the mutual information will be zero. Likewise, if the entropy remains unchanged at this test site despite the knowledge of the amino acid identity at another site, the mutual information will again be zero. There is only a meaningful mutual information between a test site and a given amino acid site if the knowledge of that given amino acid reduces the entropy at the test site. The mutual information cannot be negative, so the reverse situation, i.e. an increase in diversity with the knowledge of amino acid identity at a given site, cannot occur. Statistical considerations in AIMS The position sensitive averages in AIMS in particular are critical for comparing repertoires from distinct sources or subsets of repertoires, and are capable of identifying different modes of recognition for these molecules. Importantly, these averages are difficult to directly compare statistically, as they are not normally distributed due to the discrete nature of the 20 amino acids composing protein sequences. As such, the AIMS standard for plotting properties of these repertoires is to bootstrap a normal distribution of receptor averages, with the bootstrapped average and the bootstrapped standard deviation plotted in the final figure. The bootsrapped distribution is sampled 1000 times as a default. Statistical significance in AIMS is then calculated using either a two-sided nonparametric Studentized bootstrap or a two-sided nonparametric permutation test as outlined in “Bootstrap Methods and Their Application” [67]. In this manuscript, the two-sided nonparametric permutation test was exclusively utilized to calculate statistical significance. Here, the test statistic z is set to a simple difference of means, and we randomly permute the data into two bins. We then count the number of permutations where the randomly permuted test statistic is greater than or equal to the empirical test statistic. The p-value is then calculated as: (4) where z is the permuted test statistic and z0 is the empirical test statistic. R is then the number of tested permutations, here 1000, and is the count of permutations where the square of the permuted test statistic is greater than the square of the empirical test statistic. Assorted p-value cutoffs are reported within the figure legends throughout the manuscript. Generation of simulated TCR Datasets To benchmark quantitative comparisons between different implementations of the AIMS analysis and existing software such as GLIPH and TCRdist, we created a new AIMS module for the generation of simulated TCR datasets. Results of these comparisons can be found in S2 Table. These simulated datasets are generated via a random selection of human V- and J-gene segments, followed by a random deletion of 0–2 amino acids from these selected segments. From there, user inputs determine the number of randomly added amino acids and the probability distributions of these amino acids. It is important to note that the V- and J-gene selection probabilities, the deletion probabilities, and the added amino acid probabilities are pseudo-randomly generated, and are not meant to match biological frequencies. Our test simulated dataset was comprised of 15,000 total receptors, from three discrete simulated datasets of 5,000 receptors with lengths varying from 11–14 amino acids. The three datasets are named after the amino acid insertion distributions they draw from, with all three datasets excluding cysteine and proline from the distribution. The “Random” dataset sets the probability of insertion of all other amino acids to 1/18. The “KRQN” dataset sets the probability of insertion 20-fold more likely for positive charged amino acids K and R and 10-fold more likely for hydrophilic amino acids Q and N. Likewise, the “DEHY” dataset sets the probability of insertion 20-fold more likely for negatively charged amino acids D and E and 10-fold more likely for amphipathic amino acids H and Y. Such strong trends should generate relatively clean separations using any clustering approach. This list of 15,000 single chain sequences from these three generated datasets are used as the input for each analysis. In AIMS analysis, the “Standard” approach uses the central encoding scheme, vector normalization and an entropy re-weighting of the biophysical property matrix. This matrix is then projected onto 3 UMAP dimensions and clustered using the DBSCAN algorithm. The remaining entries in S2 Table are deviations from this standard, with the analysis name highlighting which step is changed. So “AIMS Left” utilizes the left alignment scheme, while “AIMS PCA-Kmeans” utilizes the PCA for projection of the data and Kmeans for clustering. TCRdist and AIMSdist clusters are determined using a hierarchical clustering approach with a cutoff of 30 TCRdist units and 4 AIMSdist units. GLIPH clusters are defined by the identified statistically significant motifs with a length of 3 or more and 10 or more sequences per motif, as the clustering algorithm did not converge after 48 hours of continuous calculation. It should be noted that a true metric of “clustering success” is difficult to quantitatively determine. As such we report a range of statistics for each analysis. For instance, while hierarchical clustering of TCRdist and AIMSdist metrics give nearly 100% cluster purity, the large number of clusters (nearly 400 for AIMSdist and over 500 for TCRdist) may make these results hard to parse. Further, the number of clusters per dataset is unevenly weighted. We note that while AIMS standard analysis cluster purity appears to be low (75%) the majority of the “contaminants” are from the randomly generated dataset, which likely includes sequences enriched in positive or negative charge. In a way, such “impurities” may be desired in the analysis of especially TCRs, as crossreactivity may make it likely that TCRs from distinct datasets have similar biophysical properties. As a final note, GLIPH performs the worst of nearly all analyses, clustering only 28% of the sequences with single sequences belonging to multiple clusters. This is likely because GLIPH is not meant for the analysis of simulated data, making such comparisons inherently unfair. Extended methods Dimensionality reduction and unsupervised clustering In the analysis of immune repertoires, and broadly of amino acid sequences, thorough characterization of these sequences requires the generation of a high-dimensional space comprised of assorted descriptive properties. To systematically analyze this high-dimensional space of data, AIMS employs both linear and non-linear dimensionality reduction techniques with extensive flexibility in the application of these techniques given to users. Often, we recommend starting with the linear dimensionality reduction, principal component analysis (PCA). PCA is a highly interpretable dimensionality reduction technique that projects the data onto linear combinations of the input vectors corresponding to the orthogonal vectors spanning the dimensions of highest variance in the data. PCA is a powerful and interpretable tool for analysis of high-dimensional datasets, as the identified principal components are fundamental linear algebraic properties of the matrix. Due to the linear nature of PCA, the precise biophysical properties used to create the principal components are easily inferred from the data. Unfortunately, in the analysis of immune repertoires, the axes of highest variance are not always those that best separate the key biophysical features in a given dataset. In antibody and TCR data particularly, the vectors of highest variance in the data will generally be in the center of the CDR3 loops. While this frequently also means that CDR3 will be the most distinguishing feature of a given antibody or TCR sequence, this need not always be the case. If needed, users can instead turn to the nonlinear dimensionality reduction techniques, specifically t-stochastic neighborhood embedding (t-SNE) and uniform manifold approximation projection (UMAP). Fundamentally these nonlinear algorithms attempt to reduce dimensionality while preserving distance between and within clusters of data points in the original input space. Users should become familiar with each algorithm by reading the relevant literature, but certain key features will be discussed here. Perhaps most importantly, both t-SNE and UMAP as implemented in python are inherently stochastic algorithms. This means that if users want reproducible analysis, care must be taken to first specify a specific seed from which the stochastic algorithm will start from. Further, as nonlinear algorithms the resultant projections are not easily interpreted, making the identification of biophysical differences in localized clusters of data points difficult. Notably however, some of the downstream analytical tools in AIMS can help overcome this shortcoming. Once the data have been projected onto a lower-dimensional space, users must define their clustering algorithm of choice. The default, Kmeans clustering, is the most conceptually straightforward, breaking the data into N clusters, where N is defined by the user. Kmeans clustering is especially useful if users should expect a priori some specific number of clusters arise within their data. In using AIMS for more exploratory studies, density-based clustering is recommended instead, using either OPTICS or DBSCAN algorithms. These algorithms are not biased by a user-defined number of clusters, and instead identify clusters based on localized concentrations of data points. Each algorithm comes with its own advantages and disadvantages, so again users are encouraged to read further on these to inform their analysis. In the body of this manuscript, UMAP is used to reduce the dimensionality, while the OPTICS algorithm is used to cluster the data. Linear discriminant analysis As discussed briefly in the main text, linear discriminant analysis (LDA) is described in greater detail in Boughter et al. [36] and Nandigrami et al. [38]. Briefly, LDA is useful for two distinct purposes, the generation of classifiers to be used in future applications, and the identification of key features that discriminate between two well-defined datasets. Importantly, both of these applications require large amounts of well-defined data to be confident in the results and a discrete number of labels that can be applied to these data. Unlike much of the data used as examples here, which are more heterogeneous or comprised of mixed populations of data. Despite this, here we briefly define how linear discriminant analysis works in AIMS. LDA is conceptually similar to PCA, in that the data are projected onto axes generated via a linear combination of the input vectors. However, in LDA the classes which each sequence belongs to are added as input, and the identified axes are those which both minimize within-class distance while maximizing distance between classes. Further, unlike PCA, users must be aware of a potential for overfitting. To avoid this, AIMS includes multiple pre-processing steps before the LDA calculation, including the removal of highly correlated vectors in the biophysical property matrices and a range of algorithms for the selection of a subset of key vectors. When the LDA calculation is completed, the key outputs used in AIMS are the linear weights for each input vector. These weights can then be sorted by magnitude to identify the key properties that best discriminate between the input datasets. These properties can then be visualized in AIMS using the standard biophysical property analysis. For the generation of interpretable classifiers using LDA, more advanced knowledge of machine learning is recommended. Dimensionality reduction and unsupervised clustering In the analysis of immune repertoires, and broadly of amino acid sequences, thorough characterization of these sequences requires the generation of a high-dimensional space comprised of assorted descriptive properties. To systematically analyze this high-dimensional space of data, AIMS employs both linear and non-linear dimensionality reduction techniques with extensive flexibility in the application of these techniques given to users. Often, we recommend starting with the linear dimensionality reduction, principal component analysis (PCA). PCA is a highly interpretable dimensionality reduction technique that projects the data onto linear combinations of the input vectors corresponding to the orthogonal vectors spanning the dimensions of highest variance in the data. PCA is a powerful and interpretable tool for analysis of high-dimensional datasets, as the identified principal components are fundamental linear algebraic properties of the matrix. Due to the linear nature of PCA, the precise biophysical properties used to create the principal components are easily inferred from the data. Unfortunately, in the analysis of immune repertoires, the axes of highest variance are not always those that best separate the key biophysical features in a given dataset. In antibody and TCR data particularly, the vectors of highest variance in the data will generally be in the center of the CDR3 loops. While this frequently also means that CDR3 will be the most distinguishing feature of a given antibody or TCR sequence, this need not always be the case. If needed, users can instead turn to the nonlinear dimensionality reduction techniques, specifically t-stochastic neighborhood embedding (t-SNE) and uniform manifold approximation projection (UMAP). Fundamentally these nonlinear algorithms attempt to reduce dimensionality while preserving distance between and within clusters of data points in the original input space. Users should become familiar with each algorithm by reading the relevant literature, but certain key features will be discussed here. Perhaps most importantly, both t-SNE and UMAP as implemented in python are inherently stochastic algorithms. This means that if users want reproducible analysis, care must be taken to first specify a specific seed from which the stochastic algorithm will start from. Further, as nonlinear algorithms the resultant projections are not easily interpreted, making the identification of biophysical differences in localized clusters of data points difficult. Notably however, some of the downstream analytical tools in AIMS can help overcome this shortcoming. Once the data have been projected onto a lower-dimensional space, users must define their clustering algorithm of choice. The default, Kmeans clustering, is the most conceptually straightforward, breaking the data into N clusters, where N is defined by the user. Kmeans clustering is especially useful if users should expect a priori some specific number of clusters arise within their data. In using AIMS for more exploratory studies, density-based clustering is recommended instead, using either OPTICS or DBSCAN algorithms. These algorithms are not biased by a user-defined number of clusters, and instead identify clusters based on localized concentrations of data points. Each algorithm comes with its own advantages and disadvantages, so again users are encouraged to read further on these to inform their analysis. In the body of this manuscript, UMAP is used to reduce the dimensionality, while the OPTICS algorithm is used to cluster the data. Linear discriminant analysis As discussed briefly in the main text, linear discriminant analysis (LDA) is described in greater detail in Boughter et al. [36] and Nandigrami et al. [38]. Briefly, LDA is useful for two distinct purposes, the generation of classifiers to be used in future applications, and the identification of key features that discriminate between two well-defined datasets. Importantly, both of these applications require large amounts of well-defined data to be confident in the results and a discrete number of labels that can be applied to these data. Unlike much of the data used as examples here, which are more heterogeneous or comprised of mixed populations of data. Despite this, here we briefly define how linear discriminant analysis works in AIMS. LDA is conceptually similar to PCA, in that the data are projected onto axes generated via a linear combination of the input vectors. However, in LDA the classes which each sequence belongs to are added as input, and the identified axes are those which both minimize within-class distance while maximizing distance between classes. Further, unlike PCA, users must be aware of a potential for overfitting. To avoid this, AIMS includes multiple pre-processing steps before the LDA calculation, including the removal of highly correlated vectors in the biophysical property matrices and a range of algorithms for the selection of a subset of key vectors. When the LDA calculation is completed, the key outputs used in AIMS are the linear weights for each input vector. These weights can then be sorted by magnitude to identify the key properties that best discriminate between the input datasets. These properties can then be visualized in AIMS using the standard biophysical property analysis. For the generation of interpretable classifiers using LDA, more advanced knowledge of machine learning is recommended. Supporting information S1 Fig. Alternate numerical encoding alignment schemes in AIMS using the same repertoire data as Fig 1C. Each of these schemes are independently applied to individual key structural features by aligning to the (A) N-terminal amino acids, (B) C-terminal amino acids, or (C) “bulge” encoding as discussed in the peptide analysis described in the main text. Here the bulge padding is set to 6, i.e. 3 amino acids padding the N- and C- termini are separated for alignment, and the remaining amino acids are centrally aligned. https://doi.org/10.1371/journal.pcbi.1011577.s001 (TIFF) S2 Fig. Examples of the input flexibility available in AIMS, with molecular structures on the left and AIMS encodings of subsets of these structures on the right. (A) Antibody encoding of all six CDR loops, structure via Borowska & Boughter et al. [68]. (B) Multiple sequence alignment encoding as discussed in Nandigrami et al. [38]. (C) MHC and MHC-like encoding of the α-helices and β-strands of these related molecules, structures via PDBs: 2XPG, 1ZT4. (D) Multiple sequence alignment encoding of Influenza hemagglutinin (HA) protein, structure via PDB: 1RUZ. Influenza MSA via 3DFlu [69]. https://doi.org/10.1371/journal.pcbi.1011577.s002 (TIFF) S3 Fig. A graphical overview of the standard AIMS analytical pipeline. (A) Visual representation of the high-dimensional biophysical property matrix. (B) Representative parsed biophysical property matrix reshaped into two dimensions. (C) Exemplary dimensionality reduction, clustering, and re-visualization of the data in B. (D) Simplified matrix representation of the linear discriminant analysis workflow. Final repertoire characterization steps using (E) biophysical property analysis or (F) information theoretic analysis of this specific example dataset. Here and throughout AIMS outputs, “sequence position” refers to the encoded position in the AIMS alignment matrix. Vertical black lines in panels E, F, delineate core structural features (here, distinct CDR loops). All position-sensitive figures utilize the same AIMS encoding. Lines and colored boxes help guide the reader through the workflow. https://doi.org/10.1371/journal.pcbi.1011577.s003 (TIFF) S4 Fig. VDJdb repertoire dimensionality reduction and clustering analysis including proline-containing outlier sequences. Shown are the three-dimensional clustering results for the paired chain (A) and single chain (B) data and the two-dimensional projections of these same figures for the paired chain (C) and single chain (D) data. CDR3β amino acid sequences of these outliers are highlighted in the center of the figure. Both sequences were confirmed to be the outliers in the paired chain and single-chain data. https://doi.org/10.1371/journal.pcbi.1011577.s004 (TIFF) S5 Fig. Cluster purity quantification for an array of metadata from the VDJ database when comparing paired chain (left column) and single chain (right column) clustering results. Antigen species source for each cluster member, with a cluster purity of 0.52 ± 0.26 (paired, A) and 0.39 ± 0.24 (single, B). Presenting MHC of each tested epitope for each cluster member, with a cluster purity of 0.68 ± 0.35 (paired, C) and 0.646 ± 0.37 (single, D). Organism haplotype for each cluster member, with a cluster purity of 0.46 ± 0.26 (paired, E) and 0.38 ± 0.22 (single, F). Legends are comprehensive for panels A, B, C, and D but only show a subset of the groups for panels E, F. https://doi.org/10.1371/journal.pcbi.1011577.s005 (TIFF) S6 Fig. Biophysical and information theoretic analysis, as in Figs 3 and 4, for the defined antigenic sequences reactive to either Influenza or EBV peptides as used in Glanville et al. [26]. (A) Initial AIMS encoding separated by antigenic reactivity, using the central alignment scheme. (B) Example of a biophysical property mask applied to the data in (A), here specifically showing the position- and sequence-sensitive normalized charge of each sequence. (C) Net biophysical properties, i.e. averaged over all positions and all sequences, for each antigen specificity. (D) Position sensitive charge and hydropathy, i.e. averaged over the y-axis of panel B, for each antigen specificity. Information theoretic analysis concludes the characterization of an antigen-specific repertoire, with the position sensitive entropy (E) and mutual information difference (F). Statistical significance of differences between these two populations are calculated for panels C, D, and E using a non-parametric permutation test (Methods). Averages and standard deviations in panels C, D, and E calculated using a bootstrapping procedure (Methods), with standard deviation in panels D and E represented as a shaded region about the solid line averages. ns—not significant, *—p < 0.05, solid bar with * above—contiguous region of p < 0.05. https://doi.org/10.1371/journal.pcbi.1011577.s006 (TIFF) S7 Fig. Detailed biophysical analysis done in parallel with Fig 3 for the paired chain (left column) and single chain (right column) selected clusters. (A, B) Sequence logos of the selected clusters of Fig 3A and 3B, as generated by WebLogo [70]. (C, D) Biophysical property masks of charge (top) and hydropathy (bottom) for each cluster of sequences. The position-sensitive biophysical property masks of Fig 3C and 3D are generated by averaging over the y-axis of these plots. (E, F) Net averaged biophysical properties, i.e. averages over the x- and y-axes of panels C and D, of four out of the sixty-one available AIMS properties for each cluster of sequences. Statistical significance of differences between these two populations are calculated for panels E and F using a non-parametric permutation test (Methods). *—p < 0.05, **—p < 0.025, ***—p < 0.01, ****—p < 0.001. https://doi.org/10.1371/journal.pcbi.1011577.s007 (TIFF) S8 Fig. Associated figures for the peptide analysis of Fig 4. (A) AIMS-encoding using the bulge alignment scheme of the dataset of Influenza A and Ebolavirus derived peptides. (B) Position-independent amino acid frequencies for the HLA-A2 presented Influenza A peptides and the HLA-B15 Ebolavirus peptides. Statistical significance of differences between these two populations are calculated for panel B using a non-parametric permutation test (Methods). *—p < 0.05. https://doi.org/10.1371/journal.pcbi.1011577.s008 (TIFF) S9 Fig. Raw distributions for the population differences highlighted in Fig 4. Position-sensitive amino acid probability distributions for (A) Influenza A and (B) Ebolavirus derived peptides. Position-sensitive mutual information calculated between each encoded sequence position for (C) Influenza A and (D) Ebolavirus derived peptides. Amino acid digram frequencies for (E) Influenza A and (F) Ebolavirus derived peptides. https://doi.org/10.1371/journal.pcbi.1011577.s009 (TIFF) S10 Fig. Associated figures for the statistical significance of the peptide analysis of Fig 4. Statistical significance is shown for the amino acid frequency difference (A), the average Shannon entropy difference (B), the mutual information difference (C), and the digram frequency difference (D). Statistical significance of differences between these two populations are calculated using a non-parametric permutation test (Methods). For all tests, a threshold of p < 0.05 is used, denoted by either the solid red line in panel B or the presence of a filled (black) square in the matrices of panels A, C, and D. https://doi.org/10.1371/journal.pcbi.1011577.s010 (TIFF) S11 Fig. Quantitative comparison of distance metrics used in AIMS and TCRdist. Here only the sequence distances calculated via TCRdist and AIMS between full-CDR sequences of Mayer-Blackwell et al. [56] are compared directly for the TCRα- and β-chains. Correlation coefficients between these distance metrics are reported for the full set of sequences and for closely related sequences, which are delineated by the dashed vertical lines at a TCRdist of 60 units. TCRs are isolated from human T cells in response to each antigen listed above the plots (A-F) or for the full dataset of TCRs (G). https://doi.org/10.1371/journal.pcbi.1011577.s011 (TIFF) S12 Fig. Quantitative comparison of clustering performance of AIMS and GLIPH on the same dataset. The AIMS clusters (A) are generated from the curated Influenza A reactive sequences from Glanville et al. Supplementary Table 1 [26] subject to a UMAP projection and DBSCAN clustering (eps = 0.15) of the biophysical property matrix. while the GLIPH clusters (B) are taken directly from Glanville et al. Supplementary Table 7 [26]. Calculating the AIMS distances between the sequences within the AIMS clustering (C) or the GLIPH clustering (D) shows highly similar patterns for the most similar sequences, suggesting both methods are capable of identifying highly pure clusters, albeit with a higher resolution in AIMS. AIMS additionally identifies highly biophysically distinct yet self-similar clusters (sequences 50–150) compared to the previously identified specificity groups. https://doi.org/10.1371/journal.pcbi.1011577.s012 (TIFF) S13 Fig. Comparison of the effect of different alignment strategies on the AIMS distance using all TCRs in the Glanville et al. dataset [26]. We see that largely the distances are preserved for similar TCRs, yet there is some divergence in the calculated metric at higher distances for comparisons between the bulge- and center-alignments (left) and the left- and center-alignments (right). https://doi.org/10.1371/journal.pcbi.1011577.s013 (TIFF) S1 Table. List of all of biophysical properties used for this study. For hotspot detecting variables (HS) a simplified form of the description is used. For more in-depth descriptions, the original reference should be used. https://doi.org/10.1371/journal.pcbi.1011577.s014 (CSV) S2 Table. A quantitative comparison of the accuracy of TCR sequence clustering using a simulated dataset between the different modes of AIMS analysis and TCRdist and GLIPH software. Details of the simulated dataset and the details of the comparisons can be found in the methods. While “purity” may be considered a useful metric of successful clustering, different approaches will yield different types of clustered receptors, so it is unlikely that a “best” approach exists. https://doi.org/10.1371/journal.pcbi.1011577.s015 (CSV) S3 Table. Table used for the second version of the AIMS scoring of pairwise amino acid interactions. The table attempts to recapitulate the interactions between amino acids at the level of an introductory biochemistry course. https://doi.org/10.1371/journal.pcbi.1011577.s016 (CSV) Acknowledgments We thank David Margulies, Caitlin Castro, Ryan Duncombe, and Augusta Broughton for insightful comments and discussions.

Kainth, Amoldeep S.;Haddad, Gabriela A.;Hall, Johnathon M.;Ruthenburg, Alexander J.

doi: 10.1371/journal.pcbi.1011576pmid: 37883581

Introduction Characterization of transcripts and their variants is essential to advancing our understanding of gene expression in development, responses to stimuli, and disease etiology. The advent of massively parallel short-read sequencing enabled the development of transcriptomics and discovery of biological features such as gene expression levels and pervasive genome-wide low-level transcription [1,2]. High depth of sequencing, relatively low error rates, information about strand of origin and ability to sequence a wide range of RNA from varied upstream sources are some of the key advantages of short-read RNA-seq [3,4]. Furthermore, multiple computational tools have been developed and benchmarked for the alignment, assembly and relative quantitation of short-read transcripts for general as well as application-specific purposes [5,6]. However, short-read sequencing approaches bear several caveats: i) library preparation for short-read RNA-seq typically involves PCR amplification which is prone to length and sequence-based biases that can compromise quantification and in some cases detection [7,8]; ii) the length of reads (typically 50–200 bp) is much shorter than most of the transcripts, such that transcript assembly and quantitation profoundly relies on inferential re-construction of transcripts; iii) differences in the computational algorithms lead to substantial inconsistencies in detection of transcripts and their splicing isoforms, variations in abundance estimation, and potential segmented assembly of a continuous transcript [6,9–12]. These limitations are exacerbated for non-canonical transcripts such as long noncoding RNA (lncRNA) that are typically low abundance and lack canonical features of coding transcripts [9,13]. Towards remediating these issues, new tools have become recently available. Long-read sequencing, also known as third generation sequencing, has permitted longer spans or in some cases full-length transcripts to be sequenced [14]. The MinION device from Oxford Nanopore Technologies (ONT) can directly sequence cDNA which could potentially overcome the segmentation and PCR amplification limitations of short-read sequencing [15]. While the typical read length from short-read sequencing is less than 250 nt, long reads from ONT can reach ≥10kb [16]. This can result in improved and consistent detection of transcripts and their splicing isoforms, better mapping of repeat regions, and potential mitigation of the short-read segmentation problem [17–20]. While long-read sequencing has several advantages, it too has its share of caveats, being prone to higher error rates [18,21,22], lower transcript coverage and depth [23,24], lack of strand of origin information for reads obtained from methods such as ONT direct and PCR cDNA sequencing, and having a more limited toolkit for analysis in comparison to short-read sequencing. As many of the analysis tools were developed for DNA-level genomic assembly, they are ill-equipped to detect variations in transcript copy number and isoforms [25]. Although an array of tools has been developed for long-read RNA-seq analysis [catalogued in 26], many of them are still in their infancy and lack robustness [27]. Hybrid algorithms developed to combine short- and long-read datasets are largely built either for error correction in the long-read data [21,28] or employ short-read as a template for long-read assembly [29]. Consequently, the output is reduced in long-read information with bias towards short-read attributes [28]. There remains a dearth of approaches that truly integrate short- and long-read datasets for transcript calling. Critically, assembly frameworks of many of the current approaches are modeled on well-annotated high abundance canonical transcripts. Further refinement and development of long-read transcriptomic tools is warranted for analyses of non-canonical transcripts such as lncRNA which constitute a large fraction of the transcriptome [30]. LncRNA have emerged as important regulators of mammalian genomes, implicated in the regulation of genes through a diverse and growing set of mechanisms [31–36]. One class of transcripts that are highly enriched in lncRNA are the recently described chromatin-enriched RNA (cheRNA) [37–40]. CheRNA have been particularly difficult to characterize as they are highly cell-type divergent, predominantly unannotated, and so low abundance that they were largely unobserved prior to biochemical enrichment [37,38]. Owing to these features, we reasoned that cheRNA would provide a challenging testbed to develop and benchmark long-read RNA-seq analyses for assembly and quantitation of transcripts. In this study, we improve existing bioinformatic workflows to integrate the quantitative depth of short-read sequencing with the qualitative strengths of long-read sequencing for improved characterization of transcripts. Using short- and long-read datasets we generated in two cell lines with mono-exonic ERCC spike-in standards, as well as existing datasets of multi-exonic sequin spike-in standards, we systematically compare key parameters and biases in the read alignment and assembly of transcripts. We also report novel artifacts and further document known pitfalls in long-read datasets that can substantially impact the identity and abundance of assembled transcripts. We develop a computational pipeline, SLURP (Stranding Long Unstranded Reads using Primer sequences), to infer strand-of-origin from ONT long-read cDNA libraries (hereafter “stranding”), markedly improving assembly of transcripts from long reads. Incorporating our stranding method, we devise a hybrid transcript assembly pipeline, TASSEL (Transcript Assembly using Short and Strand Emended Long reads), that merges qualitative features of stranded long reads with the quantitative depth of short-read sequencing. TASSEL outperforms other assembly methods in terms of sensitivity and complete assembly on the correct strand. At the molecular level, TASSEL resulted in substantially improved capture of key transcriptomic features such as transcription start and termination sites as well as better enrichment of active histone marks and RNA Pol II. To demonstrate the efficacy of TASSEL, we implement it for improved characterization of cheRNA, resolving previously ambiguous transcripts into coherent and discrete molecules. SLURP is generally applicable to other cDNA-based ONT long-read datasets and TASSEL can be applied to ONT as well as PacBio long-read RNA-seq to refine transcript identification and isoform structure, which can enable better molecular manipulations to illuminate their mechanisms. Results Using short- and long-read RNA-seq to capture chromatin-enriched RNA To evaluate the merits of short- and long-read sequencing platforms for assembly and molecular feature characterization of transcripts, we focused our efforts on a specific but challenging class of transcripts called cheRNA (chromatin-enriched RNA). We leveraged biochemical fractionation of nuclei [37,41] to obtain cheRNA (Fig 1A) in two cell lines—HAP1, a near-haploid leukemia cell line derived from KBM-7 [42], and HL1, a cardiac muscle cell line derived from AT-1 mouse atrial cardiomyocytes [43]. Importantly, at the nuclear extraction step, mono-exonic ERCC RNA standards [44] were spiked in to allow for quality control analyses and direct comparisons between samples for both short- and long-read libraries. Fractionated RNA was then depleted of ribosomal RNA and subjected to either short-read or long-read RNA sequencing. Owing to higher quantitative depth, short-read sequencing (n = 3) was done on both fractions to identify transcripts which are significantly enriched in the chromatin fraction as compared to the nucleoplasmic fraction. We define these transcripts as cheRNA (S1A Fig). We performed long-read sequencing (n = 2) for only the chromatin fraction for comparison and conjugation with short-read transcripts from the chromatin fraction and to refine cheRNA transcript models. For long-read RNA-seq, RNA was polyadenylated for use with polyT primers, subjected to library preparation using the Oxford Nanopore direct cDNA sequencing kit, then run on a MinION sequencer. We chose the cDNA sequencing platform as it has higher yield than the direct RNA sequencing platform [45]. We obtained ~1.2M and 3M reads for HAP1 replicates, and ~1.8M and 2.5M reads for HL1 replicates and subjected them to our analysis pipeline (S1 Table). The median sizes of reads in the long-read libraries were 1.7–1.9kb for HAP1 samples and 1.2–1.4kb for HL1 samples; replicates were nearly identical in read length profiles (Figs 1B and S1B and S1 Table). Thus, our long-read sequencing captures much longer reads than typical short-read sequencing. Next, long reads were aligned using minimap2 [46] to human (hg38) and mouse (mm10) genome assemblies for HAP1 and HL1, respectively. Aligned long reads showed strong correlation for genomic coverage between the replicates (pearson r = 0.92 and 0.86 for HAP1 and HL1, respectively) (Figs 1C and S1C). Similarly, short-read alignments with hisat2 [47] also showed high correlation between replicates (S1D Fig) for both cell lines, attesting to the reproducibility of the RNA fractionation, library preparation, and alignment methodology used here. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Short- and long-read RNA-seq to capture chromatin-enriched RNA. A. Schematic illustrating the nuclear fractionation of cells to isolate chromatin-enriched RNA (cheRNA) then subjected to short- and long-read sequencing. Nuclei from HAP1 and HL1 cells were fractionated into nucleoplasm and chromatin fractions using urea/detergent buffer [37], and ERCC standard RNA [44] were spiked in before RNA extraction and sequencing. For short-read, both the nucleoplasm and chromatin fraction RNA were subjected to single end 50 bp Illumina sequencing (n = 3). For long-read, only the chromatin fraction was sequenced using Oxford Nanopore Technology (ONT; n = 2). B. Density plot of read lengths from long-read sequencing of HAP1 replicates (n = 2). C. Correlation between HAP1 replicates (n = 2) for genomic coverage of long-read alignments binned at 1 kb. Pearson correlation is shown. D. Average counts per million (average CPM, dark color; +/- SD, lighter color) of aligned reads across all ERCC transcripts, meta-scaled to 1000 nt, in the HAP1 short-read (n = 3) and long-read (n = 2) samples. E. Relationship between the average fraction of ERCC transcripts covered by short (left) or long (right) read alignments as a function of their amount (attomoles) or length (nt) for HAP1 samples. F. Metagene plots and heatmaps of mapped read coverage (average CPM) for HAP1 short- and long-read alignments, scaled to transcription start sites (TSS) and transcription termination sites (TTS) of gencode hg38v41 genes. G. Enrichment of TSS ± 50 bp as well as TTS ± 50 bp in short-read replicates (n = 3) and long-read replicates (n = 2) in HAP1 samples. Average counts in the indicated region for each of the replicates were normalized to a background of random 100 bp regions in the same library to account for variations in the depth of the libraries. * p < 0.05; ** p < 0.01; unpaired Student’s t-test. H. Metagene plots of mapped read coverage (average CPM) for HAP1 short (left) and long (right) reads, centered on curated transcription start sites (TSS; [50]) or termination sites (polyA; [51]). https://doi.org/10.1371/journal.pcbi.1011576.g001 Spike-in standards highlight similarities and limitations in short- and long-read alignment Transcript coverage is critical for correct assembly and estimation of abundance. The use of ERCC spike-in RNA enables us to directly compare coverage of short- and long-read alignments. Short-read alignments were highly variable in the gene body with clear lack of coverage at the 5’ and 3’ ends, whereas long-read alignments showed largely homogenous coverage across ERCC transcripts (Figs 1D and S1E). Next, we tested the extent to which a given transcript is covered as a function of its abundance or length. We observed that the fraction of a given transcript covered by short-read alignment showed a sigmoidal relationship with the amount added: near-linear sensitivity for low abundance transcripts, with coverage saturating at mid- to high-abundance transcripts (Figs 1E and S1F). While long-read alignments also showed a similar trend, they had less dynamic range, especially for low abundance transcripts. Of the 92 ERCC transcripts, 78 and 82 were detected for HAP1 and HL1 short-read datasets respectively (Figs 1E and S1F, left), as compared to 51 and 58 in the long-read datasets (Figs 1E and S1F, right). This is consistent with higher depth of the short-read datasets as compared to the long-read datasets (S1 Table). Importantly, the fraction coverage of ERCC transcripts was independent of the length of the transcripts for both short- and long-read alignments. Similarly, the coverage of known genes in our HAP1 and HL1 datasets is also independent of the length of the gene (S1G Fig), indicating that our libraries are free of any artifactual length biases. Thus, high correlation between replicates, sensitivity to abundance, and absence of length bias demonstrate the reliability of our library-prep and alignment methods for both sequencing platforms. Long-read alignments are better in capturing transcript termini As short-read libraries are generated using random hexamers from fragmented RNA-samples and sequenced often with read lengths of less than 100 bp, a given read covers only a fraction of the transcript. Since long reads have the potential to capture a full-length transcript (Figs 1B and S1B), we reasoned that long-read alignment might be able to capture ends of transcripts better than short-read alignment. To formally test this hypothesis, we compared the distributions of long- and short-read alignments over known genes. While the short-read alignments showed better coverage in the gene bodies, the long-read alignments evinced better demarcation of transcription start sites and transcription termination sites of annotated genes (Figs 1F, S1H and S1I). A direct comparison showed that the 5’ and 3’ ends of a gene are more prevalent in the long-read alignments than the short-read alignments (Fig 1G). The lower gene-body coverage in the long-read alignments is consistent with previous findings [20,48]. Additionally, we observed strong 3’-end coverage bias for long reads as reported by previous studies [20,49]. A limitation of the foregoing analyses is that they are restricted to reference genes. However, there are many more experimentally derived TSS and TTS independent of the consensus reference annotation. To expand our analysis, we calculated the coverage of short- and long-read alignments on transcription start sites annotated on the basis of CAGE, TSS-Seq and RAMPAGE [50] as well as highly curated polyA sites [51]. Alignments from both sequencing platforms showed enrichment over these key ends of transcripts (Figs 1H, S1J and S1K). The metagene plots of long-read alignments centered over curated TSS showed more density going into the gene body relative to upstream of the TSS for both cell lines. Conversely, long-reads centered over TTS showed more density coming from the gene body relative to downstream of the TTS for both cell lines. Corresponding short-read plots showed noisier profiles and symmetrical density distributions, indicating less biologically representative capturing of 5’ and 3’ ends (Figs 1H, S1J and S1K). Thus, we conclude that long reads are capturing transcript molecule ends more effectively than short reads. Enhanced capture of intact transcripts in long-read sequencing can improve the detection of rare isoforms and gene-fusion events (see S1 Note). Taken together, these analyses suggest that short- and long-read datasets can complement each other for better coverage of transcriptomic elements. Benchmarking long-read transcript assembly One of the crucial steps of RNA-seq analysis is assembly of transcripts from the reads as it dictates the identification of transcript structure, isoform variants, gene abundance and differential gene expression. Therefore, we compared three different transcript assembly programs: Cufflinks [52], StringTie2 [53,54] and FLAIR [55]. First, we compared these programs for the assembly of the ERCC spike-in standards as we know the exact transcript structure of these transcripts. We observed that guided assembly with StringTie yielded highest sensitivity and precision across short- and long-read assembly (S2A Fig). A major limitation of the ERCC spike-in standards is that they are unspliced transcripts with no alternative isoforms; as such they may fail to capture the complexity of a typical mammalian transcriptome. To overcome this caveat, we also tested the assembly methods on the sequin spike-in standards in the existing short- and long-read datasets from the SG-Nex consortium [48]. Sequins are multi-exonic spike-in RNA standards with alternative isoforms [56]. Similar to ERCC standards, guided assembly with StringTie outperformed other short- or long-read methods (S2A Fig). Furthermore, StringTie also performed better than other methods for transcript assembly in our HAP1 and HL1 datasets. In general, we observed that short-read transcript assembly has higher accuracy as well as precision than long-read transcript assembly. This could arise due to multiple factors such as lower depth, higher error rate and suboptimal assembly of long-read sequencing [57]. As estimation of transcript and gene abundance is one of the desired outcomes of most RNA-seq analyses, we assessed whether our short- and long-read alignment and transcript assembly are sensitive to variations in abundance and lengths of transcripts in the transcriptome. To do so, we computed abundance of mono-exonic ERCC and multi-exonic sequin spike-in standards, as their identity, abundance, and lengths are known a priori. We observed that the abundance of each spike-in standard in the short- and long-read libraries via StringTie assembly and quantitation showed a near-linear relationship with the concentration of each standard transcript added during library preparation (Figs 2A and S2B). A similar relationship was also observed using BedTools [58] which computes coverage based on alignments and is independent of transcript assembly (S2C Fig). Short-read library preparation involves PCR amplification whereas the long-read direct cDNA libraries were made without PCR. The linear range of transcript quantitation by short-read assembly and its similarity with the long-read assembly suggests that it is largely free of PCR biases typically thought to be associated with short-read RNA-seq analyses [7,8]. We also observed that quantitation of short- and long-read data are independent of the length of spike-in transcripts (Figs 2A and S2B). This is particularly informative for short-read RNA-seq as it demonstrates that although individual RNA is fragmented during short-read library prep, StringTie can efficiently assemble and quantitate a wide range (0.2–7 kb) of well-annotated transcripts from short reads. However, in the absence of an adequate ground truth knowledge of the precise transcriptome in these cell lines, we caution the extrapolation of this finding to more complex and diverse mammalian transcriptomes. Assembled transcript counts from long-read replicates were well correlated for medium and high abundance transcripts (Fig 2B, HAP1 R2 = 0.83; S2D Fig, HL1 R2 = 0.74), indicating reproducibility of the extraction, sequencing, and analysis pipeline. Consistent with higher counting error at the lower ends of the distribution, we detected increased divergence in quantitation of low abundance transcripts. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Optimization of short- and long-read transcript assembly. A. Average abundance (Tags Per Million; TPM) of ERCC transcripts determined by StringTie versus amount added (attomoles) or length (nt) of the transcript in HAP1 samples (n = 3 for short reads, n = 2 for long reads). Short-read plots are shown in red (left), long-read plots in blue (right). R2, correlation coefficient for linear regression. B. Correlation between HAP1 replicates for long-read transcriptome assembly by StringTie. Shown are the scatter plots of abundance (TPM) of each transcript in the two replicates. Colors correspond to kernel density estimations of scatter plot distribution. R2, correlation coefficient for linear regression. C. Comparison of structure of transcripts assembled by StringTie for HAP1 and HL1 long-read samples with structure of reference genome transcripts. The class codes for relationship between the assembled transcript and the closest reference transcript were deduced from gffcompare. D. Correlation plot for the average abundance (TPM) of ERCC spike-in transcripts in the short-read datasets (n = 3) with the average abundance in the long-read datasets (n = 2) of HAP1 samples, determined by StringTie. Shown are the transcripts detected by both sequencing platforms. E. Scatter plot comparing the average abundance (TPM) of all transcripts in the short-read datasets (n = 3) and the long-read datasets (n = 2) in HAP1 samples. Transcripts assembled by short- and long-read assembly were merged using StringTie merge to obtain a non-redundant pool of transcripts across samples. Abundance in each sample was then calculated using the re-estimation function of StringTie using this merged transcriptome as the template. Color bar corresponds to kernel density estimations of scatter plot distribution. F. Number of long-read transcripts with or without any overlapping transcript in the short-read datasets (left); number of short-read transcripts with or without any overlapping transcript in the long-read datasets (right). G. Structural comparison of short-read assembled transcripts with the long-read assembled transcripts for HAP1 and HL1 samples. The long-read transcriptome assembly was used as the reference and short-read as the query in gffcompare. H. Percent of transcripts in the down-sampled short-read dataset for several key transcript structures (class codes in panel 2G) using long-read transcripts as a reference in HL1 samples. Note that the depth of 10% of HL1 short-read dataset (dotted line) is approximately equivalent to the depth of long-read dataset in terms of number of mapped bases. https://doi.org/10.1371/journal.pcbi.1011576.g002 Having established the efficacy of StringTie in the assembly of transcripts from long reads, we compared the assembled long-read transcriptomes with reference transcriptomes (Fig 2C). We observed that more than 80% of long-read transcripts overlapped with the reference transcriptome: 5–14% of transcripts matched exactly with the reference transcriptome, while 6–8% showed a match of at least one intron-exon junction. In contrast, 45–60% of transcripts assembled from short-read samples showed an exact match with the reference transcriptome (S2E Fig). Similar to previous analyses [54], we observed that a large percentage of the long-read assembled transcripts (>40%) were fully contained in reference gene introns. These truncated transcripts could originate from inherent limitations of long-read sequencing, such as incomplete assembly due to low coverage and internal priming of oligo-dT primers [reviewed in 20]. The prevalence of such transcripts is higher in our dataset, possibly because of the enrichment of nascent transcripts in the chromatin-bound transcripts [59–61]. Notably, many transcripts in our long-read datasets (~7%; Fig 2C class code “s”), in contrast to the short-read datasets (S2E Fig, class code “s”), were assembled to the opposite strand of the reference transcriptome. This is because strand information is lost during the preparation of cDNA for ONT direct cDNA long-read RNA-seq. In contrast, the short-read libraries were prepared using a method that preserves the strand information. Comparison of short- and long-read data After establishing that our long-read datasets pass initial quality control measures, we assessed long-read sequencing performance relative to the short-read sequencing. To do so, we leveraged matching short-read datasets for each cell line, which also contained spiked in ERCC RNA. Analysis of averaged TPMs of ERCC standards from StringTie for long reads (n = 2) against short reads (n = 3) showed strong correlation between the HAP1 and HL1 datasets (R2 = 0.88 and 0.91, respectively) (Figs 2D and S2F). Expanding this analysis to all transcripts assembled by StringTie showed high correlation of quantitation for highly abundant transcripts (Figs 2E and S2G). However, we observed increased divergence for low abundance transcripts (log10(TPM) < 0), as short-read sequencing showed more dynamic range than long-read sequencing for these transcripts. This is consistent with higher sensitivity of short-read sequencing for low abundance transcripts (Figs 2A and S2B). While both transcript assembly platforms peaked at similar lengths in their distributions, short-read assemblies had a higher number of long transcripts (S2H Fig). We observed more than 70% of long-read transcripts in the HAP1 dataset and more than 60% in the HL1 dataset had an overlapping transcript in the corresponding short-read dataset (Fig 2F). Conversely, more than 90% of short-read transcripts had an overlap in the corresponding long-read dataset (Fig 2F). Then, we compared the fine-scale structure of short- and long-read assembled transcripts. More than 20% of short-read transcripts in the HAP1 dataset and more than 25% of short-read transcripts in the HL1 dataset showed an exact match with their long-read counterparts (Fig 2G). Additionally, ~30% of multi-exon short-read transcripts had at least one intron-exon junction matched with the corresponding long-read transcripts. The differences in transcript structure between the two platforms could arise due to differences in the depth of sequencing or the inherent differences in the sequencing and assembly methodologies. To test the effect of depth differences, we compared the long-read transcriptome structure with that derived from down-sampled short reads (Fig 2H). The percentage of short-read transcripts matching exactly with a long-read transcript increases concomitantly with the depth of short-read sequencing (Fig 2H, class code “=“) until the short-read coverage equals that of long-read coverage. We detected minimal change thereafter as it denotes the maximum limit of coherent assembly for long-read assemblies and saturation of the total number of transcripts assembled in the short-read assembly (S2I Fig). The inverse trend is seen for short-read transcripts that are contained within the long-read transcripts (Fig 2H, class code “c”), emphasizing the fact that assembly from low-depth short-read RNA-seq leads to artifactual segmentation of transcripts. Conversely, we detected that a higher proportion of short-read transcripts contained a long-read transcript with increase in the depth of short-read sequencing (Fig 2H, class code “k”). This suggests that lower depth of long-read sequencing can cause incomplete assembly of transcripts, consistent with our observation that short-read assemblies had a higher number of long transcripts (S2H). The proportion of short-read transcripts with no counterpart in the long-read dataset was typically higher in the high-depth short-read dataset (Fig 2H, class code “u”). These analyses indicate that sequencing depth is an important but not the only factor that limits concordant assembly of transcripts between the short- and long-read platforms. Our analyses highlight two main caveats of transcript assembly by long reads: assembly of long-read transcripts on the incorrect strand due to the lack of strand of origin information, and containment of long-read transcripts in the reference genes due to the lack of sequencing depth. We undertook to develop a computational pipeline to overcome these limitations. Development of computational stranding of long reads to improve transcript assembly There are many methods to prepare and sequence stranded libraries for short-read RNA-seq [62] which culminate in accurate assembly and analyses of transcripts [63]. However, libraries prepared from ONT direct and PCR-cDNA methods inherently lack strand of origin information. Consequently, these methods suffer from alignment of reads to the incorrect strand of the locus of origin, leading to assembly of transcripts opposite to their true strand (Fig 2C). As the existing software for stranding of long reads discarded >70% of the reads in our datasets (S3A Fig), we sought to develop a computational “stranding” pipeline, SLURP (Stranding Long Unstranded Reads using Primer sequences). Our method exploits the sequences of primers used during preparation of cDNA long-read libraries to infer strand of origin (Fig 3A). Specifically, primer 1 (with a polyT 3’ end) is used to synthesize the first strand of cDNA by reverse transcriptase, which subsequently switches its template strand using primer 2 (with a GGG 3’ end) to synthesize the second cDNA strand. Therefore, we reasoned that the presence of primer 1 or the reverse complement of primer 2 in a read would indicate that it originates from the first strand, whereas the presence of primer 2 or the reverse complement of primer 1 would denote that the read is derived from the second strand. As the primers are incorporated in the majority of the reads, we took advantage of a motif-enrichment tool, MEME [64], to predict the sequences of primer 1 and primer 2 (S3B Fig). We checked the location and prevalence of the two primer sequences in the long reads. Consistent with priming from the transcript termini, primer 1 and primer 2 were highly enriched within the first 100 bp of the reads while their reverse complements were enriched in the last 100 bp of the reads (Fig 3B). To our surprise, we observed that the prevalence of primer 1 (>53% of reads) is considerably higher than primer 2 (<9%) in the direct-cDNA kit, suggesting that the first strand synthesis step is more efficient than strand-switching and second strand synthesis. This skew may account for the coverage bias at the 3’-end of long-read RNA-seq reported in many previous studies [reviewed in 20]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Primer-based elucidation of the strand of origin of long reads. A. Schematic of ONT long-read cDNA library preparation. Primer 1 with 3’ polyT initiates first strand synthesis by reverse transcriptase that adds non-templated -CCC- at the end. Primer 2 with -GGG- end anneals to the first strand and mediates strand switching for synthesis of the second strand. Sequencing adapters are subsequently ligated to the double stranded cDNA molecule. Either of the two strands (F: first, S: second) can be loaded in a given sequencing nanopore. B. Location and enrichment of sequencing primers (primer 1 and primer 2) and their reverse complements in the long reads. C. Change in the correct strand mapping of unstranded and stranded long-reads to ERCC transcripts. D. Assembly of ERCC-00092 transcript by unstranded and stranded long reads. E. Comparison of Pychopper and SLURP for the sensitivity of assembling the ERCC (left) or human (right) transcripts in the HAP1 dataset. F. Comparison of user time required by Pychopper and SLURP to strand reads in the HAP1 dataset. G. Comparison of percentage of sequin spike-in standard transcripts mapping to the opposite strand of the reference annotation in the unstranded and stranded long-read assemblies using direct cDNA and PCR-cDNA datasets with StringTie as well as direct cDNA dataset with FLAIR (n = 2–3; mean ±SD; * p < 0.05; ** p < 0.01; *** p < 0.001 unpaired Student’s t-test). H. Comparison of SLURP-mediated change in the sensitivity and precision of the sequin spike-in standards transcript assembly. I. Example of a sequin spike-in standard transcript correctly assembled upon SLURP-mediated stranding of long reads. J. Comparison of percentage of transcripts mapping to the opposite strand of reference transcripts in the unstranded and stranded long-read transcriptome of HAP1 cells (n = 2; mean ±SD; ** p < 0.01 unpaired Student’s t-test). https://doi.org/10.1371/journal.pcbi.1011576.g003 We compared different combinations of primer searches as well as permissible number of mismatches to strand our long-read libraries. Guided by their prevalence (Fig 3B), we restricted our search of primer sequences to the first or last 100 bp of reads to minimize artifactual stranding due to coincidental occurrence of these sequences in read interiors. We observed that a 3-criteria (primer 1, primer 2 and reverse complement of primer 2) search combined with 2 mismatches provided an appropriate balance between the number of assembled transcripts and extent of correct stranding with respect to the reference transcriptome (S3C and S3D Fig). SLURP yielded significantly more stranded reads than UNAGI [65] and Pychopper (https://github.com/epi2me-labs/pychopper) (S3A Fig). We checked the efficacy of SLURP on ERCC spike-in standards, as their precise strand of origin is known a priori. SLURP successfully reassigned the majority (>90%) of the long reads originated from ERCC loci to their correct strand (Fig 3C). Importantly, while the alignment of unstranded long reads led to ambiguous assembly of two ERCC-00092 transcripts in opposite directions, alignment of stranded long reads led to assembly of one correct transcript (Fig 3D). Notably, SLURP demonstrated higher sensitivity of ERCC transcript assembly than Pychopper, with equivalent or better precision (Figs 3E and S3E) in the HAP1 dataset. Similar increase in sensitivity of detection was also observed for human transcripts in the HAP1 datasets (Figs 3E and S3E). As a result, use of SLURP led to assembly of a substantially greater number of matching transcripts than Pychopper (S3F Fig). We reason that the increase in the yield of stranded reads by SLURP results in increased sensitivity of detection as well as precise end-to-end assembly. For example, use of Pychopper led to a truncated assembly of ERCC-00108 transcript whereas use of SLURP led to assembly of the full transcript (S3G Fig). Furthermore, use of SLURP had markedly lower computational load than Pychopper (Fig 3F). SLURP-mediated improvement is not limited to StringTie as similar increase in sensitivity and precision was also detected for transcripts assembled by FLAIR (S3H Fig). When tested on HL1 (mouse transcripts) and HCT116 datasets from the SG-Nex consortium, SLURP outperformed Pychopper in terms of sensitivity and precision of assembly as well as the computational overhead of stranding (S3I and S3J Fig). Then we tested SLURP on multi-exonic sequin spike-in standards. Use of SLURP markedly reduced the proportion of transcripts assembled on the wrong strand of the reference annotation for both direct cDNA and PCR-cDNA kits of the ONT platform (Fig 3G). In addition to the transcript assembly by StringTie, SLURP also rectified wrong-strand assembly by FLAIR. SLURP not only corrected the strand of transcript assembly but also improved the overall sensitivity and precision of the sequin transcriptome in the direct- and PCR-cDNA datasets with StringTie and FLAIR (Fig 3H). For example, neither of the two isoforms of the multi-exonic sequin transcript R1_22 were correctly assembled using the uncorrected long reads (Fig 3I). SLURP correction led to assembly of both the isoforms in the correct direction. When tested on sequin spike-in standards with alternate TSS/TTS, SLURP improved the sensitivity and precision of their assembly (S3K and S3L Fig). We expanded our analyses to all assembled transcripts in our HAP1 dataset as well as a K562 cell line dataset from the SG-Nex consortium [48]. Within the limits of imperfect annotation of the human reference genome for any given cell line, we observed that the stranding of long reads also led to significant reduction of transcripts assembled to the opposite strand of the reference transcriptome (Figs 3J and S3M). Removal of unstranded reads reduces the erroneous assembly of transcripts (S3N Fig). As exemplified by a known transcript (S3O Fig), the overall reduction in erroneous mapping can lead to assembly of a multi-exonic transcript that matches the gencode isoform. The foregoing analyses demonstrate that our stranding pipeline can be used as a general tool to infer strand information of complex transcripts from long-read cDNA data independent of short-read datasets or assembly programs to increase the accuracy of transcript assembly and any subsequent analysis. Long-read libraries are prone to reverse transcriptase cDNA synthesis artifacts As per the library preparation method, primer 1 initiates the synthesis of the first strand and primer 2 initiates the synthesis of the second strand after the reverse transcriptase switches strands (Fig 3A). Thus, the occurrence of primer 1 and its reverse complement should be mutually exclusive. Surprisingly, we discovered that many long reads have the primer 1 sequence in their first 100 bp and the reverse complement of primer 1 in their last 100 bp (Fig 4A). The occurrence of the primer 1 sequence at each end of a single read would not be anticipated from the way the library was prepared, yet this is observed in ~6% of total reads (Fig 4B). To test the prevalence of this artifact, we probed a published long-read data set [48] and detected similar levels of such reads (Fig 4B). Although Pychopper indicates the presence of such artifactual occurrences of primer configuration, the possible basis, the underlying sequence features and the downstream effects of such reads on read alignment and transcript quantitation is not known. Further examination revealed that these reads map to the same genomic locus twice in opposite directions (Fig 4C). Notably, we found that one of these two alignments is flagged as a “supplementary alignment” by the minimap2 aligner while the other is considered as the primary alignment. We reasoned that such an alignment could be possible if the reads are palindromic. Indeed, secondary-structure prediction shows that most of these reads are largely palindromic within base calling error, with one half of the read being the reverse complement of the other (Figs 4D, 4E and S4A). Surprisingly, filtering of supplementary alignments led to substantial changes in the computation of coverage and abundance of transcripts (Fig 4F). We speculate that these reads originate when the reverse transcriptase switches strands due to micro-homology [66] and continues to synthesize cDNA using the first strand as the template (Fig 4G), leading to artifactual doubling of read length via reverse complement synthesis. Consistent with this notion, we detected virtually no palindromic reads in the direct RNA long-read libraries that do not use reverse transcriptase for cDNA synthesis (Fig 4B). Hence, these analyses reveal a possible cause and consequences of a widespread artifact in ONT long-read cDNA libraries that contribute to error in read length and counting. The mapping error by minimap2 is propagated to transcript assembly and quantitation by StringTie, which could negatively impact the accuracy of gene expression analyses. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Widespread cDNA synthesis artifact in long reads. A. Plot showing enrichment of primer 1 at the 5’ end and the reverse complement of primer 1 at the 3’ end of artifactual reads. B. Prevalence of artifactual reads with primer 1 as well as its reverse complement in the libraries generated in the current as well as previously published studies. Note the lack of such artifactual reads in the library generated from direct RNA sequencing. C. Genome browser track showing the alignment of an artifactual read twice at the same locus (CNIH3) in the opposite directions. Note that this is the only read that mapped to this locus; depth of coverage by minimap2 alignment is doubled in the places where the read maps twice. D. Secondary structure prediction of the read in panel C by RNA-fold shows that it is largely palindromic. E. Meta mountain plot of the extent of palindromes in the artifactual (orange, n = 50) and the same number of randomly selected (black) reads from the whole set (not restricted to non-semi-palindromic). Base pairing score of the given base from the RNA fold structure is increased by 1 if the base is paired downstream and decreased by 1 if the base is paired upstream; no change for an unpaired base. Scores were scaled to 100 bp meta-transcript. Shaded region indicates SD. F. Measurement of the coverage (left) and abundance (TPM; right) of transcripts by StringTie before and after filtering the supplementary alignments. G. Model depicting a possible source of artifactual reads in the long-read libraries. Primer 1 initiates synthesis of the first strand by the reverse transcriptase (RT). A micro-homology region in a read may create a short hairpin which would lead to switching of the template strand from the original RNA molecule to the first strand which is being synthesized, producing a continuous palindromic read. https://doi.org/10.1371/journal.pcbi.1011576.g004 Integration of short-read data with stranded long-read data improves transcript assembly To overcome the limitations of long-read transcript assembly, we developed a hybrid transcript assembly pipeline, TASSEL (Transcript Assembly using Short and Strand Emended Long reads), that incorporates long range information of stranded long reads with high depth of short-read sequencing, with very low additional computational burden (Fig 5A). First, we compared TASSEL to other hybrid and standalone long-read transcript assembly programs for transcript assembly of ERCC spike-in standards (Figs 5B and S5B). TASSEL showed markedly higher sensitivity and precision of ERCC transcript assembly when compared to FLAIR [55], Bambu [67], StringTie Mix [29], RNA-Bloom2 [68], IsONform [69], IsoQuant [70] and RATTLE [71]. While the other pipelines failed to assemble >40 of the 92 ERCC transcripts, only 13 transcripts were not assembled via TASSEL (S5B Fig). FLAIR and StringTie Mix also assembled many incomplete ERCC transcripts where the assembled transcripts were shorter than the actual molecular standards. Most importantly, TASSEL, by far, assembled the highest number of complete ERCC transcripts (S5B Fig). When tested as a function of transcript abundance, we observed a large variation in the rate of false negative assembly of ERCC transcripts amongst the tested assembly methods (Fig 5C). TASSEL showed a substantially lower false negative rate than other methods, indicating its higher sensitivity. As alternative TSS and TTS of transcripts dictate most isoform differences across human tissues [72], a key parameter of assembling the correct and coherent transcript is determination of its ends. The ends of TASSEL-derived ERCC transcripts matched completely with the actual ends of these standard transcripts (Figs 5D and S5A). On the other hand, many transcripts assembled with StringTie Mix were artifactually truncated at their 5’ and/or 3’ ends. When tested on more complex sequin spike-in RNA standards, TASSEL showed highest sensitivity of transcript assembly when compared to TALON [73], Bambu, FLAIR and StringTie Mix (Fig 5E). To test the accuracy of de novo assembly capabilities, we tested TASSEL by randomly removing 50% of sequin annotations. TASSEL assembled the highest number of correct “total” and “hidden” transcripts when compared to the unstranded long-read, stranded long-read, short-read and StringTie Mix assemblies and TAMA pipeline [74] (Figs 5F and S5C). We observed that TAMA is highly reliant on reference annotations as it was able to assemble only one matching sequin transcript (compared to 107 for TASSEL) in the absence of reference annotations at the merge stage, and assembled only one matching "hidden” sequin transcript (compared to 48 for TASSEL) when 50% of sequin annotations were provided at the merge stage (S5C Fig). Expanding the scope of TASSEL, we observed a similar increase in the sensitivity of transcript assembly from datasets obtained from the PCR-cDNA kit (S5D Fig). Comparing the precision of transcript assembly, we observed that the baseline precision of TASSEL was lower than the short-read or StringTie Mix assembly (Fig 5G). However, the precision of TASSEL assembly can be increased to the equivalent levels of short-read assembly and StringTie Mix by increasing the stringency of minimum read counts required for transcript assembly. Importantly, TASSEL maintains higher sensitivity of assembly than other assembly methods at higher stringency of detection. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Improved transcript assembly with a strand-aware hybrid pipeline. A. Workflow of TASSEL (Transcript Assembly using Short and Strand-Emended Long reads) pipeline. Transcripts obtained from short-read RNA-seq are merged with those obtained from stranded long reads in a strand-aware manner. B. Comparison of the indicated transcript assembly methods for the sensitivity and precision of ERCC transcript assembly in the HAP1 dataset. RNA-bloom2* refers to transcript assembly by RNA-bloom2 using correction from the corresponding short reads. C. False negative rate of assembling ERCC transcripts, by each of the indicated assembly methods, as a function of their abundance in the HAP1 dataset. Each point represents only the ERCC transcripts at the given concentration. SLURP-stranded reads were used for FLAIR assembly. D. Ends of the 92 ERCC transcripts (arranged in the increasing order of length) assembled by TASSEL (magenta circle) and StringTie Mix (StMix, green diamond) in the HAP1 dataset. Gray bar indicates actual transcript. The color bar indicates the abundance of the given transcript. E. Sensitivity of the indicated assembly methods at the transcript level for the sequin spike-in standards. 100% of sequin annotations were provided to each of the methods during assembly of transcripts. FLAIR + short-read indicates integration of the corresponding short-read dataset at the FLAIR correction step. SLURP-stranded reads were used for FLAIR assembly. Assembled transcripts were then compared against the sequin reference annotation using gffcompare. F. Number of total (left) or hidden (right) matching transcripts assembled by the indicated assembly method when only 50% of the sequin annotations were provided during transcript assembly. Hidden transcripts refer to the sequin standard transcripts whose annotations were removed from the annotation file. USL: unstranded long-read, SL: stranded long-read, SR: short-read, StMix: StringTie Mix. G. Sensitivity and precision of the indicated assembly methods under increasing stringency of transcript detection using 50% of the sequin annotations. Stringency was increased by increasing the minimum read counts (-c) required for calling a transcript; indicated by size of the marker. H. Proximity of TSS (left) and TTS (right) of known genes (gencode hg38v41) to the TSS and TTS of transcripts assembled by TASSEL or StringTie Mix in the HAP1 dataset. I. Enrichment of H3K4me3 (left, normalized to input) and RNA Pol II (right, normalized to the total number of mapped reads) at the TSS of transcripts assembled by TASSEL or StringTie Mix in the HAP1 dataset. H3K4me3 and RNA Pol II occupancy were calculated from ChIP-seq data from HAP1 cells [97,103]. J. Average PRO-seq signal at TSS of transcripts assembled by TASSEL or StringTie Mix on the positive (top) and negative (bottom) strands. Normalized PRO-seq data in HAP1 cells were obtained from [99]. https://doi.org/10.1371/journal.pcbi.1011576.g005 Next, we compared the entire transcriptome assembled in our HAP1 dataset by these programs with the reference transcripts. Although Bambu performed better than StringTie Mix for ERCC assembly, it had the lowest sensitivity of assembly at the transcript level (S5E Fig). Here again, TASSEL outperformed the other tested assembly programs. As StringTie Mix was closest to TASSEL in performance, we performed further comparisons between the two. In comparison to StringTie Mix, TASSEL showed a modest increase in transcripts that match completely to reference transcripts (S5F Fig, left). Importantly, there was a substantial decrease in the assembled transcripts that are contained within the reference transcripts (S5F Fig, right). Notably, the incidence of such intron retention transcripts is much lower in the TASSEL assemblies than those derived from only long reads. Based on our earlier analysis (Fig 2H), we rationalize that this improvement in TASSEL could be due to the incorporation of higher depth of short-read sequencing. We also tested how TASSEL compares to StringTie Mix in the accurate determination of transcription start sites (TSS) and transcription termination sites (TTS) of all assembled transcripts. For this, we compared the TSS and TTS of TASSEL and StringTie Mix transcripts with those of known human genes (Fig 5H). TASSEL showed much better overlap with known TSS and TTS than StringTie Mix, indicating that TASSEL assembles more coherent transcripts than StringTie Mix. To further evaluate the accuracy of the assembled 5’-ends, agnostic of the reference assemblies that may not reflect TSS in a particular lineage [72], we sought to compare TASSEL and StringTie Mix for enrichment of other genomic features of 5’ ends from the same cell line. At the molecular level, transcription start sites are sites of high H3K4me3 deposition (epigenetic mark of active TSS [75]) and RNA Polymerase II (RNA Pol II) occupancy [76]. TSS derived from TASSEL were more enriched for these two functional entities than those derived from StringTie Mix (Fig 5I). Precision Run-On sequencing (PRO-seq), used to map the location of active RNA polymerase, can provide estimates of the 5’ end of transcripts [77]. TASSEL-derived TSS showed substantially higher PRO-seq signal than StringTie Mix-derived TSS (Fig 5J), suggesting that TASSEL TSS are better indicators of active TSS than StringTie Mix TSS. To fully characterize the capabilities of TASSEL, we then dissected the contribution of the individual components of TASSEL. First, we observed that the omission of SLURP-mediated stranding in TASSEL leads to an increase in the incorrect assembly of transcripts on the wrong strand and a decrease in the proportion of completely matching transcripts (S5G Fig), suggesting an important role of SLURP stranding in the overall efficacy of TASSEL. Then we investigated the contribution of short- and long-read transcripts to the TASSEL transcriptome (S5H Fig). We observed that both short- and long-read transcriptomes contribute equivalently to the TASSEL transcriptome. Furthermore, the consensus transcripts in the TASSEL transcriptome evince better enrichment of TSS and TTS (S5I Fig) as well as H3K4me3 occupancy, Pol II occupancy and PRO-seq signal (S5J Fig) than the transcripts obtained from either of the short- or long-read transcriptomes alone. Notably, TASSEL-mediated improvement is not limited to transcript ends. For example, TASSEL fully assembled a known spliced human transcript where the original short- or long-read dataset failed to do so (S5K Fig). As a proof of principle, we tested TASSEL on a PacBio [78] long-read dataset and observed that TASSEL leads to substantial improvement in assembly of the sequin spike-in standards (S5L Fig). We note that the HL1, HAP1, and sequin datasets tested here encompass a wide range of relative short to long read depth (S5M Fig). Efficacy of TASSEL on these datasets suggests its broad applicability on datasets of varying depths. Collectively, our data show that TASSEL outperforms contemporary transcript assembly methods for correct and complete assembly of mono- and multi-exonic transcripts as well as enrichment of transcriptionally relevant features. Thus, we felt confident moving forward to use our method to interrogate assembly of cheRNA as a challenging testbed. TASSEL improves cheRNA identification Based on the foregoing analyses, we reasoned that TASSEL can be used to improve the assembly of cheRNA which have been challenging to characterize due to their non-canonical features, low abundance, and high variance. Owing to its higher depth and dynamic range, we used only short-read RNA-seq of the chromatin and nucleoplasm fractions to define cheRNA. We defined cheRNA based on marked enrichment (>4 fold) of a transcript in the chromatin fraction relative to the nucleoplasm fraction (Fig 6A). The control ERCC spike-in standards were detected at equivalent levels in the two fractions (S6A Fig). Using differential expression analyses, we detected 6,746 and 4,969 cheRNA genes in HAP1 and HL1 cells, respectively. The detected cheRNA genes were proximal to cell line-specific protein coding genes (S6B Fig), suggesting potential biological significance. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Improved cheRNA characterization by integration of short-read assemblies with stranded long-read assemblies. A. Volcano plot for enrichment of genes in the chromatin fraction vs adjusted p value. DESeq2 on short-read counts by StringTie was used to obtain cheRNA genes (red) that are significantly (Benjamini and Hochberg adjusted p < 0.05) enriched (>4 fold) in the chromatin fraction as compared to the nucleoplasm fraction. B. Metagene plots and heatmaps of mapped read coverage (average CPM) for HAP1 (left) and HL1 (right) long-read alignments, scaled to the cheRNA genes in the corresponding samples. C. Number of cheRNA genes with or without any overlapping gene in the corresponding long-read datasets in HAP1 and HL1 samples. D. Example cheRNA locus depicting the efficacy of the optimized methodology and improved assembly of cheRNA genes. Top: coverage from short-read nucleoplasm and chromatin fractions shows high enrichment in the chromatin fraction (data presented are the sum of replicates). Bottom: assembly by indicated methods shows resolution of the segmentation problem by merging short-read with the stranded long-read assembly. E. The number of segmented cheRNA genes identified in HAP1 and HL1 short-read datasets after merging of short-read and stranded long-read transcriptomes. F. Metagene plot depicting average RNA Pol II occupancy (solid lines; shaded region indicates SE) at the TSS (± 1 kb) of segmented, unsegmented and TASSEL-refined cheRNA genes in the HAP1 dataset. RNA Pol II occupancy was calculated from ChIP-seq data from HAP1 cells [97]. G. Metagene plots of mapped read coverage (average CPM) for HAP1 (left) and HL1 (right) long-read alignments, scaled to the TASSEL-refined cheRNA genes. https://doi.org/10.1371/journal.pcbi.1011576.g006 The exact transcript structure of the detected cheRNA is not clear due to caveats in short-read RNA-seq, such as lower coverage of transcript start and end sites (Figs 1F, 1G and S1H) and apparent segmentation of transcripts due to the short length of aligned reads. When compared to the long-read dataset, we observed moderate to high enrichment of long read coverage across cheRNA genes in comparison to their upstream and downstream regions (Fig 6B), suggesting that cheRNA genes, identified through short-read RNA-seq, are represented in the long-read datasets. Then we directly compared the cheRNA genes from the short-read datasets with the transcriptome assembled from the long-read datasets. More than 75% of cheRNA genes from the short-read dataset had an overlapping gene in the corresponding long-read dataset (Fig 6C). At a finer scale, more than 40% of cheRNA genes from the HAP1 short-read dataset and more than 20% from the HL1 short-read dataset had a counterpart in the long-read datasets with more than 90% overlap (S6C Fig). We observed that cheRNA genes with minimum overlap were significantly less abundant than those with maximum overlap (S6D Fig). While this analysis shows the potential of long-read gene assembly in identifying cheRNA gene structures inferred from short-read analysis, it also reveals that long reads fail to assemble coherent cheRNA transcripts in regions of low coverage. In addition to low coverage, other inherent platform-specific variations (Fig 2C, 2G and 2H) could also contribute to incongruence of short-read cheRNA genes. An example is highlighted in Fig 6D, where a cheRNA locus is detected on the basis of high enrichment in the chromatin fraction as compared to the nucleoplasm fraction in the short-read data. Although the coverage track indicates a single gene, StringTie assembly of short reads predicts two cheRNA genes at this locus. The StringTie assembly of the original long reads resulted in assembly of three transcripts–one on the Crick strand and two on the Watson strand. StringTie Mix [29] also fails to predict the correct transcript. Strikingly, use of TASSEL led to the assembly of one transcript in the correct direction. TASSEL not only assembled the transcript on the correct strand but also resolved the segmentation problem. We computed that ~30% of the original cheRNA were segmented prior to application of TASSEL (Fig 6E). Importantly, both mono and multi-exonic segmented cheRNA are refined by TASSEL (S6E Fig). To test that this computational approach is biologically meaningful, we compared RNA Pol II occupancy at the TSS of originally segmented or unsegmented cheRNA genes with TASSEL-refined cheRNA genes. Consistent with the expectation of Pol II enrichment decorating actively transcribed promoter regions [76], we detected a substantial increase in Pol II enrichment at the TSS of refined cheRNA (Figs 6F and S6F). There were also better indications of divergent transcription–a characteristic of bona fide promoters–from the TSS of refined cheRNA genes. Additionally, there was marked improvement in coverage of long reads over refined cheRNA genes with sharpened demarcation of 5’ and 3’ ends (Fig 6G vs 6B), suggesting convergence of the two datasets for better delineation of transcripts. These analyses highlight the utility of TASSEL for improved assembly of correct transcripts and genes, even for one of the most challenging classes of molecules. Using short- and long-read RNA-seq to capture chromatin-enriched RNA To evaluate the merits of short- and long-read sequencing platforms for assembly and molecular feature characterization of transcripts, we focused our efforts on a specific but challenging class of transcripts called cheRNA (chromatin-enriched RNA). We leveraged biochemical fractionation of nuclei [37,41] to obtain cheRNA (Fig 1A) in two cell lines—HAP1, a near-haploid leukemia cell line derived from KBM-7 [42], and HL1, a cardiac muscle cell line derived from AT-1 mouse atrial cardiomyocytes [43]. Importantly, at the nuclear extraction step, mono-exonic ERCC RNA standards [44] were spiked in to allow for quality control analyses and direct comparisons between samples for both short- and long-read libraries. Fractionated RNA was then depleted of ribosomal RNA and subjected to either short-read or long-read RNA sequencing. Owing to higher quantitative depth, short-read sequencing (n = 3) was done on both fractions to identify transcripts which are significantly enriched in the chromatin fraction as compared to the nucleoplasmic fraction. We define these transcripts as cheRNA (S1A Fig). We performed long-read sequencing (n = 2) for only the chromatin fraction for comparison and conjugation with short-read transcripts from the chromatin fraction and to refine cheRNA transcript models. For long-read RNA-seq, RNA was polyadenylated for use with polyT primers, subjected to library preparation using the Oxford Nanopore direct cDNA sequencing kit, then run on a MinION sequencer. We chose the cDNA sequencing platform as it has higher yield than the direct RNA sequencing platform [45]. We obtained ~1.2M and 3M reads for HAP1 replicates, and ~1.8M and 2.5M reads for HL1 replicates and subjected them to our analysis pipeline (S1 Table). The median sizes of reads in the long-read libraries were 1.7–1.9kb for HAP1 samples and 1.2–1.4kb for HL1 samples; replicates were nearly identical in read length profiles (Figs 1B and S1B and S1 Table). Thus, our long-read sequencing captures much longer reads than typical short-read sequencing. Next, long reads were aligned using minimap2 [46] to human (hg38) and mouse (mm10) genome assemblies for HAP1 and HL1, respectively. Aligned long reads showed strong correlation for genomic coverage between the replicates (pearson r = 0.92 and 0.86 for HAP1 and HL1, respectively) (Figs 1C and S1C). Similarly, short-read alignments with hisat2 [47] also showed high correlation between replicates (S1D Fig) for both cell lines, attesting to the reproducibility of the RNA fractionation, library preparation, and alignment methodology used here. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Short- and long-read RNA-seq to capture chromatin-enriched RNA. A. Schematic illustrating the nuclear fractionation of cells to isolate chromatin-enriched RNA (cheRNA) then subjected to short- and long-read sequencing. Nuclei from HAP1 and HL1 cells were fractionated into nucleoplasm and chromatin fractions using urea/detergent buffer [37], and ERCC standard RNA [44] were spiked in before RNA extraction and sequencing. For short-read, both the nucleoplasm and chromatin fraction RNA were subjected to single end 50 bp Illumina sequencing (n = 3). For long-read, only the chromatin fraction was sequenced using Oxford Nanopore Technology (ONT; n = 2). B. Density plot of read lengths from long-read sequencing of HAP1 replicates (n = 2). C. Correlation between HAP1 replicates (n = 2) for genomic coverage of long-read alignments binned at 1 kb. Pearson correlation is shown. D. Average counts per million (average CPM, dark color; +/- SD, lighter color) of aligned reads across all ERCC transcripts, meta-scaled to 1000 nt, in the HAP1 short-read (n = 3) and long-read (n = 2) samples. E. Relationship between the average fraction of ERCC transcripts covered by short (left) or long (right) read alignments as a function of their amount (attomoles) or length (nt) for HAP1 samples. F. Metagene plots and heatmaps of mapped read coverage (average CPM) for HAP1 short- and long-read alignments, scaled to transcription start sites (TSS) and transcription termination sites (TTS) of gencode hg38v41 genes. G. Enrichment of TSS ± 50 bp as well as TTS ± 50 bp in short-read replicates (n = 3) and long-read replicates (n = 2) in HAP1 samples. Average counts in the indicated region for each of the replicates were normalized to a background of random 100 bp regions in the same library to account for variations in the depth of the libraries. * p < 0.05; ** p < 0.01; unpaired Student’s t-test. H. Metagene plots of mapped read coverage (average CPM) for HAP1 short (left) and long (right) reads, centered on curated transcription start sites (TSS; [50]) or termination sites (polyA; [51]). https://doi.org/10.1371/journal.pcbi.1011576.g001 Spike-in standards highlight similarities and limitations in short- and long-read alignment Transcript coverage is critical for correct assembly and estimation of abundance. The use of ERCC spike-in RNA enables us to directly compare coverage of short- and long-read alignments. Short-read alignments were highly variable in the gene body with clear lack of coverage at the 5’ and 3’ ends, whereas long-read alignments showed largely homogenous coverage across ERCC transcripts (Figs 1D and S1E). Next, we tested the extent to which a given transcript is covered as a function of its abundance or length. We observed that the fraction of a given transcript covered by short-read alignment showed a sigmoidal relationship with the amount added: near-linear sensitivity for low abundance transcripts, with coverage saturating at mid- to high-abundance transcripts (Figs 1E and S1F). While long-read alignments also showed a similar trend, they had less dynamic range, especially for low abundance transcripts. Of the 92 ERCC transcripts, 78 and 82 were detected for HAP1 and HL1 short-read datasets respectively (Figs 1E and S1F, left), as compared to 51 and 58 in the long-read datasets (Figs 1E and S1F, right). This is consistent with higher depth of the short-read datasets as compared to the long-read datasets (S1 Table). Importantly, the fraction coverage of ERCC transcripts was independent of the length of the transcripts for both short- and long-read alignments. Similarly, the coverage of known genes in our HAP1 and HL1 datasets is also independent of the length of the gene (S1G Fig), indicating that our libraries are free of any artifactual length biases. Thus, high correlation between replicates, sensitivity to abundance, and absence of length bias demonstrate the reliability of our library-prep and alignment methods for both sequencing platforms. Long-read alignments are better in capturing transcript termini As short-read libraries are generated using random hexamers from fragmented RNA-samples and sequenced often with read lengths of less than 100 bp, a given read covers only a fraction of the transcript. Since long reads have the potential to capture a full-length transcript (Figs 1B and S1B), we reasoned that long-read alignment might be able to capture ends of transcripts better than short-read alignment. To formally test this hypothesis, we compared the distributions of long- and short-read alignments over known genes. While the short-read alignments showed better coverage in the gene bodies, the long-read alignments evinced better demarcation of transcription start sites and transcription termination sites of annotated genes (Figs 1F, S1H and S1I). A direct comparison showed that the 5’ and 3’ ends of a gene are more prevalent in the long-read alignments than the short-read alignments (Fig 1G). The lower gene-body coverage in the long-read alignments is consistent with previous findings [20,48]. Additionally, we observed strong 3’-end coverage bias for long reads as reported by previous studies [20,49]. A limitation of the foregoing analyses is that they are restricted to reference genes. However, there are many more experimentally derived TSS and TTS independent of the consensus reference annotation. To expand our analysis, we calculated the coverage of short- and long-read alignments on transcription start sites annotated on the basis of CAGE, TSS-Seq and RAMPAGE [50] as well as highly curated polyA sites [51]. Alignments from both sequencing platforms showed enrichment over these key ends of transcripts (Figs 1H, S1J and S1K). The metagene plots of long-read alignments centered over curated TSS showed more density going into the gene body relative to upstream of the TSS for both cell lines. Conversely, long-reads centered over TTS showed more density coming from the gene body relative to downstream of the TTS for both cell lines. Corresponding short-read plots showed noisier profiles and symmetrical density distributions, indicating less biologically representative capturing of 5’ and 3’ ends (Figs 1H, S1J and S1K). Thus, we conclude that long reads are capturing transcript molecule ends more effectively than short reads. Enhanced capture of intact transcripts in long-read sequencing can improve the detection of rare isoforms and gene-fusion events (see S1 Note). Taken together, these analyses suggest that short- and long-read datasets can complement each other for better coverage of transcriptomic elements. Benchmarking long-read transcript assembly One of the crucial steps of RNA-seq analysis is assembly of transcripts from the reads as it dictates the identification of transcript structure, isoform variants, gene abundance and differential gene expression. Therefore, we compared three different transcript assembly programs: Cufflinks [52], StringTie2 [53,54] and FLAIR [55]. First, we compared these programs for the assembly of the ERCC spike-in standards as we know the exact transcript structure of these transcripts. We observed that guided assembly with StringTie yielded highest sensitivity and precision across short- and long-read assembly (S2A Fig). A major limitation of the ERCC spike-in standards is that they are unspliced transcripts with no alternative isoforms; as such they may fail to capture the complexity of a typical mammalian transcriptome. To overcome this caveat, we also tested the assembly methods on the sequin spike-in standards in the existing short- and long-read datasets from the SG-Nex consortium [48]. Sequins are multi-exonic spike-in RNA standards with alternative isoforms [56]. Similar to ERCC standards, guided assembly with StringTie outperformed other short- or long-read methods (S2A Fig). Furthermore, StringTie also performed better than other methods for transcript assembly in our HAP1 and HL1 datasets. In general, we observed that short-read transcript assembly has higher accuracy as well as precision than long-read transcript assembly. This could arise due to multiple factors such as lower depth, higher error rate and suboptimal assembly of long-read sequencing [57]. As estimation of transcript and gene abundance is one of the desired outcomes of most RNA-seq analyses, we assessed whether our short- and long-read alignment and transcript assembly are sensitive to variations in abundance and lengths of transcripts in the transcriptome. To do so, we computed abundance of mono-exonic ERCC and multi-exonic sequin spike-in standards, as their identity, abundance, and lengths are known a priori. We observed that the abundance of each spike-in standard in the short- and long-read libraries via StringTie assembly and quantitation showed a near-linear relationship with the concentration of each standard transcript added during library preparation (Figs 2A and S2B). A similar relationship was also observed using BedTools [58] which computes coverage based on alignments and is independent of transcript assembly (S2C Fig). Short-read library preparation involves PCR amplification whereas the long-read direct cDNA libraries were made without PCR. The linear range of transcript quantitation by short-read assembly and its similarity with the long-read assembly suggests that it is largely free of PCR biases typically thought to be associated with short-read RNA-seq analyses [7,8]. We also observed that quantitation of short- and long-read data are independent of the length of spike-in transcripts (Figs 2A and S2B). This is particularly informative for short-read RNA-seq as it demonstrates that although individual RNA is fragmented during short-read library prep, StringTie can efficiently assemble and quantitate a wide range (0.2–7 kb) of well-annotated transcripts from short reads. However, in the absence of an adequate ground truth knowledge of the precise transcriptome in these cell lines, we caution the extrapolation of this finding to more complex and diverse mammalian transcriptomes. Assembled transcript counts from long-read replicates were well correlated for medium and high abundance transcripts (Fig 2B, HAP1 R2 = 0.83; S2D Fig, HL1 R2 = 0.74), indicating reproducibility of the extraction, sequencing, and analysis pipeline. Consistent with higher counting error at the lower ends of the distribution, we detected increased divergence in quantitation of low abundance transcripts. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Optimization of short- and long-read transcript assembly. A. Average abundance (Tags Per Million; TPM) of ERCC transcripts determined by StringTie versus amount added (attomoles) or length (nt) of the transcript in HAP1 samples (n = 3 for short reads, n = 2 for long reads). Short-read plots are shown in red (left), long-read plots in blue (right). R2, correlation coefficient for linear regression. B. Correlation between HAP1 replicates for long-read transcriptome assembly by StringTie. Shown are the scatter plots of abundance (TPM) of each transcript in the two replicates. Colors correspond to kernel density estimations of scatter plot distribution. R2, correlation coefficient for linear regression. C. Comparison of structure of transcripts assembled by StringTie for HAP1 and HL1 long-read samples with structure of reference genome transcripts. The class codes for relationship between the assembled transcript and the closest reference transcript were deduced from gffcompare. D. Correlation plot for the average abundance (TPM) of ERCC spike-in transcripts in the short-read datasets (n = 3) with the average abundance in the long-read datasets (n = 2) of HAP1 samples, determined by StringTie. Shown are the transcripts detected by both sequencing platforms. E. Scatter plot comparing the average abundance (TPM) of all transcripts in the short-read datasets (n = 3) and the long-read datasets (n = 2) in HAP1 samples. Transcripts assembled by short- and long-read assembly were merged using StringTie merge to obtain a non-redundant pool of transcripts across samples. Abundance in each sample was then calculated using the re-estimation function of StringTie using this merged transcriptome as the template. Color bar corresponds to kernel density estimations of scatter plot distribution. F. Number of long-read transcripts with or without any overlapping transcript in the short-read datasets (left); number of short-read transcripts with or without any overlapping transcript in the long-read datasets (right). G. Structural comparison of short-read assembled transcripts with the long-read assembled transcripts for HAP1 and HL1 samples. The long-read transcriptome assembly was used as the reference and short-read as the query in gffcompare. H. Percent of transcripts in the down-sampled short-read dataset for several key transcript structures (class codes in panel 2G) using long-read transcripts as a reference in HL1 samples. Note that the depth of 10% of HL1 short-read dataset (dotted line) is approximately equivalent to the depth of long-read dataset in terms of number of mapped bases. https://doi.org/10.1371/journal.pcbi.1011576.g002 Having established the efficacy of StringTie in the assembly of transcripts from long reads, we compared the assembled long-read transcriptomes with reference transcriptomes (Fig 2C). We observed that more than 80% of long-read transcripts overlapped with the reference transcriptome: 5–14% of transcripts matched exactly with the reference transcriptome, while 6–8% showed a match of at least one intron-exon junction. In contrast, 45–60% of transcripts assembled from short-read samples showed an exact match with the reference transcriptome (S2E Fig). Similar to previous analyses [54], we observed that a large percentage of the long-read assembled transcripts (>40%) were fully contained in reference gene introns. These truncated transcripts could originate from inherent limitations of long-read sequencing, such as incomplete assembly due to low coverage and internal priming of oligo-dT primers [reviewed in 20]. The prevalence of such transcripts is higher in our dataset, possibly because of the enrichment of nascent transcripts in the chromatin-bound transcripts [59–61]. Notably, many transcripts in our long-read datasets (~7%; Fig 2C class code “s”), in contrast to the short-read datasets (S2E Fig, class code “s”), were assembled to the opposite strand of the reference transcriptome. This is because strand information is lost during the preparation of cDNA for ONT direct cDNA long-read RNA-seq. In contrast, the short-read libraries were prepared using a method that preserves the strand information. Comparison of short- and long-read data After establishing that our long-read datasets pass initial quality control measures, we assessed long-read sequencing performance relative to the short-read sequencing. To do so, we leveraged matching short-read datasets for each cell line, which also contained spiked in ERCC RNA. Analysis of averaged TPMs of ERCC standards from StringTie for long reads (n = 2) against short reads (n = 3) showed strong correlation between the HAP1 and HL1 datasets (R2 = 0.88 and 0.91, respectively) (Figs 2D and S2F). Expanding this analysis to all transcripts assembled by StringTie showed high correlation of quantitation for highly abundant transcripts (Figs 2E and S2G). However, we observed increased divergence for low abundance transcripts (log10(TPM) < 0), as short-read sequencing showed more dynamic range than long-read sequencing for these transcripts. This is consistent with higher sensitivity of short-read sequencing for low abundance transcripts (Figs 2A and S2B). While both transcript assembly platforms peaked at similar lengths in their distributions, short-read assemblies had a higher number of long transcripts (S2H Fig). We observed more than 70% of long-read transcripts in the HAP1 dataset and more than 60% in the HL1 dataset had an overlapping transcript in the corresponding short-read dataset (Fig 2F). Conversely, more than 90% of short-read transcripts had an overlap in the corresponding long-read dataset (Fig 2F). Then, we compared the fine-scale structure of short- and long-read assembled transcripts. More than 20% of short-read transcripts in the HAP1 dataset and more than 25% of short-read transcripts in the HL1 dataset showed an exact match with their long-read counterparts (Fig 2G). Additionally, ~30% of multi-exon short-read transcripts had at least one intron-exon junction matched with the corresponding long-read transcripts. The differences in transcript structure between the two platforms could arise due to differences in the depth of sequencing or the inherent differences in the sequencing and assembly methodologies. To test the effect of depth differences, we compared the long-read transcriptome structure with that derived from down-sampled short reads (Fig 2H). The percentage of short-read transcripts matching exactly with a long-read transcript increases concomitantly with the depth of short-read sequencing (Fig 2H, class code “=“) until the short-read coverage equals that of long-read coverage. We detected minimal change thereafter as it denotes the maximum limit of coherent assembly for long-read assemblies and saturation of the total number of transcripts assembled in the short-read assembly (S2I Fig). The inverse trend is seen for short-read transcripts that are contained within the long-read transcripts (Fig 2H, class code “c”), emphasizing the fact that assembly from low-depth short-read RNA-seq leads to artifactual segmentation of transcripts. Conversely, we detected that a higher proportion of short-read transcripts contained a long-read transcript with increase in the depth of short-read sequencing (Fig 2H, class code “k”). This suggests that lower depth of long-read sequencing can cause incomplete assembly of transcripts, consistent with our observation that short-read assemblies had a higher number of long transcripts (S2H). The proportion of short-read transcripts with no counterpart in the long-read dataset was typically higher in the high-depth short-read dataset (Fig 2H, class code “u”). These analyses indicate that sequencing depth is an important but not the only factor that limits concordant assembly of transcripts between the short- and long-read platforms. Our analyses highlight two main caveats of transcript assembly by long reads: assembly of long-read transcripts on the incorrect strand due to the lack of strand of origin information, and containment of long-read transcripts in the reference genes due to the lack of sequencing depth. We undertook to develop a computational pipeline to overcome these limitations. Development of computational stranding of long reads to improve transcript assembly There are many methods to prepare and sequence stranded libraries for short-read RNA-seq [62] which culminate in accurate assembly and analyses of transcripts [63]. However, libraries prepared from ONT direct and PCR-cDNA methods inherently lack strand of origin information. Consequently, these methods suffer from alignment of reads to the incorrect strand of the locus of origin, leading to assembly of transcripts opposite to their true strand (Fig 2C). As the existing software for stranding of long reads discarded >70% of the reads in our datasets (S3A Fig), we sought to develop a computational “stranding” pipeline, SLURP (Stranding Long Unstranded Reads using Primer sequences). Our method exploits the sequences of primers used during preparation of cDNA long-read libraries to infer strand of origin (Fig 3A). Specifically, primer 1 (with a polyT 3’ end) is used to synthesize the first strand of cDNA by reverse transcriptase, which subsequently switches its template strand using primer 2 (with a GGG 3’ end) to synthesize the second cDNA strand. Therefore, we reasoned that the presence of primer 1 or the reverse complement of primer 2 in a read would indicate that it originates from the first strand, whereas the presence of primer 2 or the reverse complement of primer 1 would denote that the read is derived from the second strand. As the primers are incorporated in the majority of the reads, we took advantage of a motif-enrichment tool, MEME [64], to predict the sequences of primer 1 and primer 2 (S3B Fig). We checked the location and prevalence of the two primer sequences in the long reads. Consistent with priming from the transcript termini, primer 1 and primer 2 were highly enriched within the first 100 bp of the reads while their reverse complements were enriched in the last 100 bp of the reads (Fig 3B). To our surprise, we observed that the prevalence of primer 1 (>53% of reads) is considerably higher than primer 2 (<9%) in the direct-cDNA kit, suggesting that the first strand synthesis step is more efficient than strand-switching and second strand synthesis. This skew may account for the coverage bias at the 3’-end of long-read RNA-seq reported in many previous studies [reviewed in 20]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Primer-based elucidation of the strand of origin of long reads. A. Schematic of ONT long-read cDNA library preparation. Primer 1 with 3’ polyT initiates first strand synthesis by reverse transcriptase that adds non-templated -CCC- at the end. Primer 2 with -GGG- end anneals to the first strand and mediates strand switching for synthesis of the second strand. Sequencing adapters are subsequently ligated to the double stranded cDNA molecule. Either of the two strands (F: first, S: second) can be loaded in a given sequencing nanopore. B. Location and enrichment of sequencing primers (primer 1 and primer 2) and their reverse complements in the long reads. C. Change in the correct strand mapping of unstranded and stranded long-reads to ERCC transcripts. D. Assembly of ERCC-00092 transcript by unstranded and stranded long reads. E. Comparison of Pychopper and SLURP for the sensitivity of assembling the ERCC (left) or human (right) transcripts in the HAP1 dataset. F. Comparison of user time required by Pychopper and SLURP to strand reads in the HAP1 dataset. G. Comparison of percentage of sequin spike-in standard transcripts mapping to the opposite strand of the reference annotation in the unstranded and stranded long-read assemblies using direct cDNA and PCR-cDNA datasets with StringTie as well as direct cDNA dataset with FLAIR (n = 2–3; mean ±SD; * p < 0.05; ** p < 0.01; *** p < 0.001 unpaired Student’s t-test). H. Comparison of SLURP-mediated change in the sensitivity and precision of the sequin spike-in standards transcript assembly. I. Example of a sequin spike-in standard transcript correctly assembled upon SLURP-mediated stranding of long reads. J. Comparison of percentage of transcripts mapping to the opposite strand of reference transcripts in the unstranded and stranded long-read transcriptome of HAP1 cells (n = 2; mean ±SD; ** p < 0.01 unpaired Student’s t-test). https://doi.org/10.1371/journal.pcbi.1011576.g003 We compared different combinations of primer searches as well as permissible number of mismatches to strand our long-read libraries. Guided by their prevalence (Fig 3B), we restricted our search of primer sequences to the first or last 100 bp of reads to minimize artifactual stranding due to coincidental occurrence of these sequences in read interiors. We observed that a 3-criteria (primer 1, primer 2 and reverse complement of primer 2) search combined with 2 mismatches provided an appropriate balance between the number of assembled transcripts and extent of correct stranding with respect to the reference transcriptome (S3C and S3D Fig). SLURP yielded significantly more stranded reads than UNAGI [65] and Pychopper (https://github.com/epi2me-labs/pychopper) (S3A Fig). We checked the efficacy of SLURP on ERCC spike-in standards, as their precise strand of origin is known a priori. SLURP successfully reassigned the majority (>90%) of the long reads originated from ERCC loci to their correct strand (Fig 3C). Importantly, while the alignment of unstranded long reads led to ambiguous assembly of two ERCC-00092 transcripts in opposite directions, alignment of stranded long reads led to assembly of one correct transcript (Fig 3D). Notably, SLURP demonstrated higher sensitivity of ERCC transcript assembly than Pychopper, with equivalent or better precision (Figs 3E and S3E) in the HAP1 dataset. Similar increase in sensitivity of detection was also observed for human transcripts in the HAP1 datasets (Figs 3E and S3E). As a result, use of SLURP led to assembly of a substantially greater number of matching transcripts than Pychopper (S3F Fig). We reason that the increase in the yield of stranded reads by SLURP results in increased sensitivity of detection as well as precise end-to-end assembly. For example, use of Pychopper led to a truncated assembly of ERCC-00108 transcript whereas use of SLURP led to assembly of the full transcript (S3G Fig). Furthermore, use of SLURP had markedly lower computational load than Pychopper (Fig 3F). SLURP-mediated improvement is not limited to StringTie as similar increase in sensitivity and precision was also detected for transcripts assembled by FLAIR (S3H Fig). When tested on HL1 (mouse transcripts) and HCT116 datasets from the SG-Nex consortium, SLURP outperformed Pychopper in terms of sensitivity and precision of assembly as well as the computational overhead of stranding (S3I and S3J Fig). Then we tested SLURP on multi-exonic sequin spike-in standards. Use of SLURP markedly reduced the proportion of transcripts assembled on the wrong strand of the reference annotation for both direct cDNA and PCR-cDNA kits of the ONT platform (Fig 3G). In addition to the transcript assembly by StringTie, SLURP also rectified wrong-strand assembly by FLAIR. SLURP not only corrected the strand of transcript assembly but also improved the overall sensitivity and precision of the sequin transcriptome in the direct- and PCR-cDNA datasets with StringTie and FLAIR (Fig 3H). For example, neither of the two isoforms of the multi-exonic sequin transcript R1_22 were correctly assembled using the uncorrected long reads (Fig 3I). SLURP correction led to assembly of both the isoforms in the correct direction. When tested on sequin spike-in standards with alternate TSS/TTS, SLURP improved the sensitivity and precision of their assembly (S3K and S3L Fig). We expanded our analyses to all assembled transcripts in our HAP1 dataset as well as a K562 cell line dataset from the SG-Nex consortium [48]. Within the limits of imperfect annotation of the human reference genome for any given cell line, we observed that the stranding of long reads also led to significant reduction of transcripts assembled to the opposite strand of the reference transcriptome (Figs 3J and S3M). Removal of unstranded reads reduces the erroneous assembly of transcripts (S3N Fig). As exemplified by a known transcript (S3O Fig), the overall reduction in erroneous mapping can lead to assembly of a multi-exonic transcript that matches the gencode isoform. The foregoing analyses demonstrate that our stranding pipeline can be used as a general tool to infer strand information of complex transcripts from long-read cDNA data independent of short-read datasets or assembly programs to increase the accuracy of transcript assembly and any subsequent analysis. Long-read libraries are prone to reverse transcriptase cDNA synthesis artifacts As per the library preparation method, primer 1 initiates the synthesis of the first strand and primer 2 initiates the synthesis of the second strand after the reverse transcriptase switches strands (Fig 3A). Thus, the occurrence of primer 1 and its reverse complement should be mutually exclusive. Surprisingly, we discovered that many long reads have the primer 1 sequence in their first 100 bp and the reverse complement of primer 1 in their last 100 bp (Fig 4A). The occurrence of the primer 1 sequence at each end of a single read would not be anticipated from the way the library was prepared, yet this is observed in ~6% of total reads (Fig 4B). To test the prevalence of this artifact, we probed a published long-read data set [48] and detected similar levels of such reads (Fig 4B). Although Pychopper indicates the presence of such artifactual occurrences of primer configuration, the possible basis, the underlying sequence features and the downstream effects of such reads on read alignment and transcript quantitation is not known. Further examination revealed that these reads map to the same genomic locus twice in opposite directions (Fig 4C). Notably, we found that one of these two alignments is flagged as a “supplementary alignment” by the minimap2 aligner while the other is considered as the primary alignment. We reasoned that such an alignment could be possible if the reads are palindromic. Indeed, secondary-structure prediction shows that most of these reads are largely palindromic within base calling error, with one half of the read being the reverse complement of the other (Figs 4D, 4E and S4A). Surprisingly, filtering of supplementary alignments led to substantial changes in the computation of coverage and abundance of transcripts (Fig 4F). We speculate that these reads originate when the reverse transcriptase switches strands due to micro-homology [66] and continues to synthesize cDNA using the first strand as the template (Fig 4G), leading to artifactual doubling of read length via reverse complement synthesis. Consistent with this notion, we detected virtually no palindromic reads in the direct RNA long-read libraries that do not use reverse transcriptase for cDNA synthesis (Fig 4B). Hence, these analyses reveal a possible cause and consequences of a widespread artifact in ONT long-read cDNA libraries that contribute to error in read length and counting. The mapping error by minimap2 is propagated to transcript assembly and quantitation by StringTie, which could negatively impact the accuracy of gene expression analyses. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Widespread cDNA synthesis artifact in long reads. A. Plot showing enrichment of primer 1 at the 5’ end and the reverse complement of primer 1 at the 3’ end of artifactual reads. B. Prevalence of artifactual reads with primer 1 as well as its reverse complement in the libraries generated in the current as well as previously published studies. Note the lack of such artifactual reads in the library generated from direct RNA sequencing. C. Genome browser track showing the alignment of an artifactual read twice at the same locus (CNIH3) in the opposite directions. Note that this is the only read that mapped to this locus; depth of coverage by minimap2 alignment is doubled in the places where the read maps twice. D. Secondary structure prediction of the read in panel C by RNA-fold shows that it is largely palindromic. E. Meta mountain plot of the extent of palindromes in the artifactual (orange, n = 50) and the same number of randomly selected (black) reads from the whole set (not restricted to non-semi-palindromic). Base pairing score of the given base from the RNA fold structure is increased by 1 if the base is paired downstream and decreased by 1 if the base is paired upstream; no change for an unpaired base. Scores were scaled to 100 bp meta-transcript. Shaded region indicates SD. F. Measurement of the coverage (left) and abundance (TPM; right) of transcripts by StringTie before and after filtering the supplementary alignments. G. Model depicting a possible source of artifactual reads in the long-read libraries. Primer 1 initiates synthesis of the first strand by the reverse transcriptase (RT). A micro-homology region in a read may create a short hairpin which would lead to switching of the template strand from the original RNA molecule to the first strand which is being synthesized, producing a continuous palindromic read. https://doi.org/10.1371/journal.pcbi.1011576.g004 Integration of short-read data with stranded long-read data improves transcript assembly To overcome the limitations of long-read transcript assembly, we developed a hybrid transcript assembly pipeline, TASSEL (Transcript Assembly using Short and Strand Emended Long reads), that incorporates long range information of stranded long reads with high depth of short-read sequencing, with very low additional computational burden (Fig 5A). First, we compared TASSEL to other hybrid and standalone long-read transcript assembly programs for transcript assembly of ERCC spike-in standards (Figs 5B and S5B). TASSEL showed markedly higher sensitivity and precision of ERCC transcript assembly when compared to FLAIR [55], Bambu [67], StringTie Mix [29], RNA-Bloom2 [68], IsONform [69], IsoQuant [70] and RATTLE [71]. While the other pipelines failed to assemble >40 of the 92 ERCC transcripts, only 13 transcripts were not assembled via TASSEL (S5B Fig). FLAIR and StringTie Mix also assembled many incomplete ERCC transcripts where the assembled transcripts were shorter than the actual molecular standards. Most importantly, TASSEL, by far, assembled the highest number of complete ERCC transcripts (S5B Fig). When tested as a function of transcript abundance, we observed a large variation in the rate of false negative assembly of ERCC transcripts amongst the tested assembly methods (Fig 5C). TASSEL showed a substantially lower false negative rate than other methods, indicating its higher sensitivity. As alternative TSS and TTS of transcripts dictate most isoform differences across human tissues [72], a key parameter of assembling the correct and coherent transcript is determination of its ends. The ends of TASSEL-derived ERCC transcripts matched completely with the actual ends of these standard transcripts (Figs 5D and S5A). On the other hand, many transcripts assembled with StringTie Mix were artifactually truncated at their 5’ and/or 3’ ends. When tested on more complex sequin spike-in RNA standards, TASSEL showed highest sensitivity of transcript assembly when compared to TALON [73], Bambu, FLAIR and StringTie Mix (Fig 5E). To test the accuracy of de novo assembly capabilities, we tested TASSEL by randomly removing 50% of sequin annotations. TASSEL assembled the highest number of correct “total” and “hidden” transcripts when compared to the unstranded long-read, stranded long-read, short-read and StringTie Mix assemblies and TAMA pipeline [74] (Figs 5F and S5C). We observed that TAMA is highly reliant on reference annotations as it was able to assemble only one matching sequin transcript (compared to 107 for TASSEL) in the absence of reference annotations at the merge stage, and assembled only one matching "hidden” sequin transcript (compared to 48 for TASSEL) when 50% of sequin annotations were provided at the merge stage (S5C Fig). Expanding the scope of TASSEL, we observed a similar increase in the sensitivity of transcript assembly from datasets obtained from the PCR-cDNA kit (S5D Fig). Comparing the precision of transcript assembly, we observed that the baseline precision of TASSEL was lower than the short-read or StringTie Mix assembly (Fig 5G). However, the precision of TASSEL assembly can be increased to the equivalent levels of short-read assembly and StringTie Mix by increasing the stringency of minimum read counts required for transcript assembly. Importantly, TASSEL maintains higher sensitivity of assembly than other assembly methods at higher stringency of detection. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Improved transcript assembly with a strand-aware hybrid pipeline. A. Workflow of TASSEL (Transcript Assembly using Short and Strand-Emended Long reads) pipeline. Transcripts obtained from short-read RNA-seq are merged with those obtained from stranded long reads in a strand-aware manner. B. Comparison of the indicated transcript assembly methods for the sensitivity and precision of ERCC transcript assembly in the HAP1 dataset. RNA-bloom2* refers to transcript assembly by RNA-bloom2 using correction from the corresponding short reads. C. False negative rate of assembling ERCC transcripts, by each of the indicated assembly methods, as a function of their abundance in the HAP1 dataset. Each point represents only the ERCC transcripts at the given concentration. SLURP-stranded reads were used for FLAIR assembly. D. Ends of the 92 ERCC transcripts (arranged in the increasing order of length) assembled by TASSEL (magenta circle) and StringTie Mix (StMix, green diamond) in the HAP1 dataset. Gray bar indicates actual transcript. The color bar indicates the abundance of the given transcript. E. Sensitivity of the indicated assembly methods at the transcript level for the sequin spike-in standards. 100% of sequin annotations were provided to each of the methods during assembly of transcripts. FLAIR + short-read indicates integration of the corresponding short-read dataset at the FLAIR correction step. SLURP-stranded reads were used for FLAIR assembly. Assembled transcripts were then compared against the sequin reference annotation using gffcompare. F. Number of total (left) or hidden (right) matching transcripts assembled by the indicated assembly method when only 50% of the sequin annotations were provided during transcript assembly. Hidden transcripts refer to the sequin standard transcripts whose annotations were removed from the annotation file. USL: unstranded long-read, SL: stranded long-read, SR: short-read, StMix: StringTie Mix. G. Sensitivity and precision of the indicated assembly methods under increasing stringency of transcript detection using 50% of the sequin annotations. Stringency was increased by increasing the minimum read counts (-c) required for calling a transcript; indicated by size of the marker. H. Proximity of TSS (left) and TTS (right) of known genes (gencode hg38v41) to the TSS and TTS of transcripts assembled by TASSEL or StringTie Mix in the HAP1 dataset. I. Enrichment of H3K4me3 (left, normalized to input) and RNA Pol II (right, normalized to the total number of mapped reads) at the TSS of transcripts assembled by TASSEL or StringTie Mix in the HAP1 dataset. H3K4me3 and RNA Pol II occupancy were calculated from ChIP-seq data from HAP1 cells [97,103]. J. Average PRO-seq signal at TSS of transcripts assembled by TASSEL or StringTie Mix on the positive (top) and negative (bottom) strands. Normalized PRO-seq data in HAP1 cells were obtained from [99]. https://doi.org/10.1371/journal.pcbi.1011576.g005 Next, we compared the entire transcriptome assembled in our HAP1 dataset by these programs with the reference transcripts. Although Bambu performed better than StringTie Mix for ERCC assembly, it had the lowest sensitivity of assembly at the transcript level (S5E Fig). Here again, TASSEL outperformed the other tested assembly programs. As StringTie Mix was closest to TASSEL in performance, we performed further comparisons between the two. In comparison to StringTie Mix, TASSEL showed a modest increase in transcripts that match completely to reference transcripts (S5F Fig, left). Importantly, there was a substantial decrease in the assembled transcripts that are contained within the reference transcripts (S5F Fig, right). Notably, the incidence of such intron retention transcripts is much lower in the TASSEL assemblies than those derived from only long reads. Based on our earlier analysis (Fig 2H), we rationalize that this improvement in TASSEL could be due to the incorporation of higher depth of short-read sequencing. We also tested how TASSEL compares to StringTie Mix in the accurate determination of transcription start sites (TSS) and transcription termination sites (TTS) of all assembled transcripts. For this, we compared the TSS and TTS of TASSEL and StringTie Mix transcripts with those of known human genes (Fig 5H). TASSEL showed much better overlap with known TSS and TTS than StringTie Mix, indicating that TASSEL assembles more coherent transcripts than StringTie Mix. To further evaluate the accuracy of the assembled 5’-ends, agnostic of the reference assemblies that may not reflect TSS in a particular lineage [72], we sought to compare TASSEL and StringTie Mix for enrichment of other genomic features of 5’ ends from the same cell line. At the molecular level, transcription start sites are sites of high H3K4me3 deposition (epigenetic mark of active TSS [75]) and RNA Polymerase II (RNA Pol II) occupancy [76]. TSS derived from TASSEL were more enriched for these two functional entities than those derived from StringTie Mix (Fig 5I). Precision Run-On sequencing (PRO-seq), used to map the location of active RNA polymerase, can provide estimates of the 5’ end of transcripts [77]. TASSEL-derived TSS showed substantially higher PRO-seq signal than StringTie Mix-derived TSS (Fig 5J), suggesting that TASSEL TSS are better indicators of active TSS than StringTie Mix TSS. To fully characterize the capabilities of TASSEL, we then dissected the contribution of the individual components of TASSEL. First, we observed that the omission of SLURP-mediated stranding in TASSEL leads to an increase in the incorrect assembly of transcripts on the wrong strand and a decrease in the proportion of completely matching transcripts (S5G Fig), suggesting an important role of SLURP stranding in the overall efficacy of TASSEL. Then we investigated the contribution of short- and long-read transcripts to the TASSEL transcriptome (S5H Fig). We observed that both short- and long-read transcriptomes contribute equivalently to the TASSEL transcriptome. Furthermore, the consensus transcripts in the TASSEL transcriptome evince better enrichment of TSS and TTS (S5I Fig) as well as H3K4me3 occupancy, Pol II occupancy and PRO-seq signal (S5J Fig) than the transcripts obtained from either of the short- or long-read transcriptomes alone. Notably, TASSEL-mediated improvement is not limited to transcript ends. For example, TASSEL fully assembled a known spliced human transcript where the original short- or long-read dataset failed to do so (S5K Fig). As a proof of principle, we tested TASSEL on a PacBio [78] long-read dataset and observed that TASSEL leads to substantial improvement in assembly of the sequin spike-in standards (S5L Fig). We note that the HL1, HAP1, and sequin datasets tested here encompass a wide range of relative short to long read depth (S5M Fig). Efficacy of TASSEL on these datasets suggests its broad applicability on datasets of varying depths. Collectively, our data show that TASSEL outperforms contemporary transcript assembly methods for correct and complete assembly of mono- and multi-exonic transcripts as well as enrichment of transcriptionally relevant features. Thus, we felt confident moving forward to use our method to interrogate assembly of cheRNA as a challenging testbed. TASSEL improves cheRNA identification Based on the foregoing analyses, we reasoned that TASSEL can be used to improve the assembly of cheRNA which have been challenging to characterize due to their non-canonical features, low abundance, and high variance. Owing to its higher depth and dynamic range, we used only short-read RNA-seq of the chromatin and nucleoplasm fractions to define cheRNA. We defined cheRNA based on marked enrichment (>4 fold) of a transcript in the chromatin fraction relative to the nucleoplasm fraction (Fig 6A). The control ERCC spike-in standards were detected at equivalent levels in the two fractions (S6A Fig). Using differential expression analyses, we detected 6,746 and 4,969 cheRNA genes in HAP1 and HL1 cells, respectively. The detected cheRNA genes were proximal to cell line-specific protein coding genes (S6B Fig), suggesting potential biological significance. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Improved cheRNA characterization by integration of short-read assemblies with stranded long-read assemblies. A. Volcano plot for enrichment of genes in the chromatin fraction vs adjusted p value. DESeq2 on short-read counts by StringTie was used to obtain cheRNA genes (red) that are significantly (Benjamini and Hochberg adjusted p < 0.05) enriched (>4 fold) in the chromatin fraction as compared to the nucleoplasm fraction. B. Metagene plots and heatmaps of mapped read coverage (average CPM) for HAP1 (left) and HL1 (right) long-read alignments, scaled to the cheRNA genes in the corresponding samples. C. Number of cheRNA genes with or without any overlapping gene in the corresponding long-read datasets in HAP1 and HL1 samples. D. Example cheRNA locus depicting the efficacy of the optimized methodology and improved assembly of cheRNA genes. Top: coverage from short-read nucleoplasm and chromatin fractions shows high enrichment in the chromatin fraction (data presented are the sum of replicates). Bottom: assembly by indicated methods shows resolution of the segmentation problem by merging short-read with the stranded long-read assembly. E. The number of segmented cheRNA genes identified in HAP1 and HL1 short-read datasets after merging of short-read and stranded long-read transcriptomes. F. Metagene plot depicting average RNA Pol II occupancy (solid lines; shaded region indicates SE) at the TSS (± 1 kb) of segmented, unsegmented and TASSEL-refined cheRNA genes in the HAP1 dataset. RNA Pol II occupancy was calculated from ChIP-seq data from HAP1 cells [97]. G. Metagene plots of mapped read coverage (average CPM) for HAP1 (left) and HL1 (right) long-read alignments, scaled to the TASSEL-refined cheRNA genes. https://doi.org/10.1371/journal.pcbi.1011576.g006 The exact transcript structure of the detected cheRNA is not clear due to caveats in short-read RNA-seq, such as lower coverage of transcript start and end sites (Figs 1F, 1G and S1H) and apparent segmentation of transcripts due to the short length of aligned reads. When compared to the long-read dataset, we observed moderate to high enrichment of long read coverage across cheRNA genes in comparison to their upstream and downstream regions (Fig 6B), suggesting that cheRNA genes, identified through short-read RNA-seq, are represented in the long-read datasets. Then we directly compared the cheRNA genes from the short-read datasets with the transcriptome assembled from the long-read datasets. More than 75% of cheRNA genes from the short-read dataset had an overlapping gene in the corresponding long-read dataset (Fig 6C). At a finer scale, more than 40% of cheRNA genes from the HAP1 short-read dataset and more than 20% from the HL1 short-read dataset had a counterpart in the long-read datasets with more than 90% overlap (S6C Fig). We observed that cheRNA genes with minimum overlap were significantly less abundant than those with maximum overlap (S6D Fig). While this analysis shows the potential of long-read gene assembly in identifying cheRNA gene structures inferred from short-read analysis, it also reveals that long reads fail to assemble coherent cheRNA transcripts in regions of low coverage. In addition to low coverage, other inherent platform-specific variations (Fig 2C, 2G and 2H) could also contribute to incongruence of short-read cheRNA genes. An example is highlighted in Fig 6D, where a cheRNA locus is detected on the basis of high enrichment in the chromatin fraction as compared to the nucleoplasm fraction in the short-read data. Although the coverage track indicates a single gene, StringTie assembly of short reads predicts two cheRNA genes at this locus. The StringTie assembly of the original long reads resulted in assembly of three transcripts–one on the Crick strand and two on the Watson strand. StringTie Mix [29] also fails to predict the correct transcript. Strikingly, use of TASSEL led to the assembly of one transcript in the correct direction. TASSEL not only assembled the transcript on the correct strand but also resolved the segmentation problem. We computed that ~30% of the original cheRNA were segmented prior to application of TASSEL (Fig 6E). Importantly, both mono and multi-exonic segmented cheRNA are refined by TASSEL (S6E Fig). To test that this computational approach is biologically meaningful, we compared RNA Pol II occupancy at the TSS of originally segmented or unsegmented cheRNA genes with TASSEL-refined cheRNA genes. Consistent with the expectation of Pol II enrichment decorating actively transcribed promoter regions [76], we detected a substantial increase in Pol II enrichment at the TSS of refined cheRNA (Figs 6F and S6F). There were also better indications of divergent transcription–a characteristic of bona fide promoters–from the TSS of refined cheRNA genes. Additionally, there was marked improvement in coverage of long reads over refined cheRNA genes with sharpened demarcation of 5’ and 3’ ends (Fig 6G vs 6B), suggesting convergence of the two datasets for better delineation of transcripts. These analyses highlight the utility of TASSEL for improved assembly of correct transcripts and genes, even for one of the most challenging classes of molecules. Discussion With the increased interest in long-read RNA-seq, current efforts are directed towards weighing the merits of long-read RNA-seq over its short-read counterpart as well as development of new tools for improved coalescence of the two platforms [20,29,48,79,80]. To examine whether short- or long-read RNA-seq data or a combination of both would permit robust transcript characterization, we compared key features of read alignment, read coverage, transcript assembly, and capture of biologically relevant attributes of transcripts from short- and long-read RNA-seq. To sharpen the challenge, we focused on chromatin-enriched RNA, a class of largely unannotated, low abundance transcripts that have presented problems for previous short read analyses [37,38]. We performed Illumina short-read and ONT direct cDNA long-read RNA-seq in parallel on multiple biological replicates in cell lines of mouse and human origin. Consistent with the performance of the two platforms, we obtained 2-6X coverage/bp for short-read sequencing and 0.5–0.7X for long-read sequencing. Alignments of reads from both platforms to the corresponding genomes were highly correlated between replicates, attesting to their reproducibility. Use of mono-exonic spike-in ERCC [44] and multi-exonic sequin [48,56] RNA standards enabled us to objectively compare the performance of the two platforms. Both showed a near-linear relationship for observed versus expected transcript abundance, validating their use in transcript quantitation and differential gene expression analyses. Owing to its higher sequencing depth, the short-read platform had a greater dynamic range for transcript detection, fraction of transcript covered as well as quantitation. In contrast, long-read coverage was more homogenous across ERCC transcripts and provided better coverage of 5’ and 3’ ends than short-read alignments. Collectively, our data suggest that short-read RNA-seq has quantitative advantages whereas the long-read RNA-seq has enhanced qualitative attributes. A hybrid approach that retains the best aspects of each can result in more definitive transcriptome characterization. Strengths and weaknesses of short- and long-read RNA-seq As transcript assembly is critical for most of the downstream analyses, we systematically compared multiple transcript assembly approaches. Consistent with previous studies [53,54], reference-guided assembly with StringTie2 showed higher sensitivity as well as precision for transcript assembly for both short- and long-read RNA-seq, relative to other tested programs. A large fraction of long-read transcripts was fully contained within the reference intron, perhaps largely due to the enrichment of nascent transcripts in chromatin fractions of nuclear RNA [59–61] or limitations of long-read sequencing such as low coverage and internal priming events of oligo-dT based primers. We observed a high overlap between transcripts assembled in the short- and long-read datasets. A direct comparison of the structure of transcripts showed that many in the long-read datasets were “contained” in the short-read transcripts, largely due to the higher depth of the short-read sequencing. Therefore, multiple limitations of long-read sequencing can cause truncation of assembled transcripts at low coverage regions. The lower depth of long-read assembly also impacted the quantitation of low abundance transcripts when compared to the short-read transcriptome, cautioning its use for pan-transcriptome quantitation on its own. SLURP strands and improves transcript assembly The lack of strand of origin information in the ONT direct and PCR-cDNA long-read sequencing hinders direct comparison to reference transcriptomes and short-read assemblies. Consistent with this limitation, we report clear evidence of mapping of ~40% of long reads to the incorrect strand and consequent ambiguous assembly of transcripts. The existing stranding methods discard a majority of reads leading to loss of information critical for high fidelity transcript assembly. To address this problem, we developed and benchmarked an accessible computational stranding pipeline, SLURP. Using SLURP, we corrected the erroneous mapping of long reads, resulting in a striking improvement in the assembly of spiked-in ERCC and sequin transcripts as well as assembly of transcriptomes from human and mouse cell line datasets. SLURP outperforms other stranding pipelines such as UNAGI and Pychopper, perhaps due to the way the sequences are searched within the reads and/or tolerance of mismatches. Importantly, SLURP stranding is effective in improving the assembly of mono-exonic as well as complex multi-exonic transcripts with spliced isoforms. Although SLURP strands 60–75% reads, the loss of remaining unstranded reads does not seem to hamper the sensitivity of transcript detection or assembly. On the contrary, SLURP stranding leads to an overall increase in the sensitivity and precision of transcript assembly (Figs 3H and S3L) due to the removal of the underlying noise caused by misaligned reads that otherwise reduces the confidence of mapping in the correct orientation and creates discontinuity in intact transcript assembly (Figs 3I and S3O). StringTie Mix [29], a hybrid assembly program, permitted comparable stranding efficiency but resulted in truncated transcripts, seemingly due to overweighting of short reads in the hybrid assembly. Unlike StringTie Mix, SLURP can be used as a general-purpose standalone pipeline without the corresponding short-read data and independent of the assembly program. Importantly, our stranding pipeline can be broadly applied to preexisting long-read RNA-seq studies for improved transcript assembly, comparison and integration with short-read RNA-seq data, and high-confidence detection of bona fide antisense transcripts shown to play a critical role in gene regulation [81]. Prevalent artifacts in direct ONT cDNA long-read sequencing Our in-depth stranding analysis led to the serendipitous discovery that long-read ONT cDNA libraries are plagued with a cDNA synthesis artifact. Surprisingly, ~6% of long-read cDNA libraries are palindromes (within base-calling error). As we did not detect such reads in the ONT direct RNA library, we attribute these reads to an artifact of first strand synthesis, wherein reverse transcriptase switches from its initial RNA template to its product cDNA strand for further elongation. This finding is distinct from previously reported reverse transcriptase artifacts such as artificial splicing, in which the reverse transcriptase jumps from one direct repeat to another direct repeat of the same RNA or different RNA molecule, causing the deletion of the intervening sequence [82,83]. There is less chance of formation and detection of such palindromes in the short-read datasets as short-read libraries are synthesized from fragmented RNA and sequenced with shorter read lengths, often with enzymes that lack template switching activity. Alignment using standard long-read aligners such as minimap2 leads to double mapping of these reads to the same genomic region and flagging of one of the two alignments as a supplementary alignment. However, such an alignment not only inflates coverage statistics at the alignment stage but also leads to imprecise assembly and quantitation of all transcripts. It is important for the field to be aware of this likely prevalent artifact in long-read cDNA libraries which, to our knowledge, has not yet been reported. In principle, alternate cDNA library preparations with non-template switching reverse transcriptases or future aligners and assembly algorithms adapted to resolve these palindromes can be employed to accommodate these pitfalls. Enhanced detection of molecular features promises to improve functional analyses Alternative transcription start and termination sites are shown to be the major drivers of isoform variance of transcripts [72]. Knowledge of transcript ends is also critical for downstream experimental manipulations to evaluate function. For example, efficiency of CRISPRi [84] targeting drops off sharply as a function of distance from the TSS [85]. Moreover, effective interpretation of experiments involving insertion of a strong polyadenylation sequence to make an RNA-level knockout [33,86] relies on accurate knowledge of the TSS to minimize residual RNA fragment size. In our study, direct comparison of short- and long-read alignments over known genes and curated transcription start and termination sites exposed informative differences between the two platforms. The ends of the transcripts are much better represented in the long-read alignments than in the short-read alignments. We reason that this difference may arise due to multiple factors such as incomplete cDNA synthesis from random hexamer primers on fragmented RNA and heterogenous mapping of segmented short reads. In contrast, full-length cDNA synthesis on unfragmented RNA using a 3’-based oligoT primer and potential sequencing of the entire cDNA culminate in end-to-end coverage of transcripts in the long-read data set. This provides a qualitative advantage to the long-read RNA-seq for full-length characterization of transcripts. TASSEL improves integration of short- and long-read transcript assembly Informed by the strengths and caveats of each of the two platforms, we developed a hybrid pipeline, TASSEL, to merge the short-read transcriptome with the stranded long-read transcriptome. When tested on ERCC standards against commonly used long-read transcript assembly methods such as FLAIR [55], Bambu [67], StringTie Mix [29], RNA-Bloom2 [68], IsONform [69], IsoQuant [70] and RATTLE [71], TASSEL outperformed other methods by assembling the highest number of matching spike-in transcripts. We further tested TASSEL on complex multi-exonic sequin standards and the human transcriptome in our HAP1 dataset. TASSEL had the lowest false negative rate of transcript assembly and unlike other methods, TASSEL led to assembly of complete transcripts on the correct strand. FLAIR was limited by low sensitivity and incomplete assembly. Although StringTie Mix was able to assemble transcripts on the correct strand, it had a substantially higher false negative rate than TASSEL and assembled many transcripts with truncated ends. We note that with lower stringency of detection, TASSEL may suffer from reduced precision. However, users can enhance the precision of TASSEL by increasing the stringency of detection while still achieving higher sensitivity than other methods tested here. Analyses based on spliced sequin spike-in standards as well as on HAP1 dataset indicate that TASSEL improves transcript assembly of complex multi-exonic transcripts with spliced isoforms. TASSEL also led to a substantial reduction in the transcripts that are assembled within an intron of the reference gene. We reason that this happens due to the amelioration of long-read artifacts such as low coverage and internal priming events of oligo-dT primers by short-read transcripts which are assembled from high depth libraries prepared without oligo-dT primers. Another reason for the higher efficacy of TASSEL is the use of stranded long reads which not only resolves the conflicting orientation of neighboring reads but also enhances the coverage of correctly assembled transcripts. Additionally, as TASSEL is employed after alignment and assembly of short- and long-read transcripts, it is not biased towards short-read transcript depth and maintains long-read information. As a test of its advantage at the functional level, TASSEL had higher enrichment of known TSS, TTS, histone modification of active TSS, and RNA Pol II than StringTie Mix. Together, TASSEL outperforms contemporary assembly methods for assembly on the correct strand, complete end-to-end assembly highest sensitivity of assembly, and unbiased integration of high depth short-read sequencing with the qualitative enrichment of long-read sequencing, which collectively culminates to superior manifestation of biologically relevant transcriptomic features. In addition to the ONT long-read sequencing, another prevalent long-read sequencing platform is PacBio Iso-seq [78]. We note that the inherent read correction module of Iso-seq data analysis correctly strands the long reads by orienting them in the 5’ to 3’ direction. We show that TASSEL-based merge of a transcriptome derived from stranded PacBio long reads with that derived from corresponding short reads improves transcript detection and assembly. We tested TASSEL on cheRNA transcripts which have been challenging to characterize due to their low abundance in cells, high variation from one cell type to another, and lack of canonical attributes of coding transcripts [37,38]. In comparison to short- or long-read data alone or combined analysis with StringTie Mix, the use of TASSEL led to a marked improvement in the assembly of cheRNA. It corrected the segmentation of many mono- as well as multi-exonic cheRNA genes, resulting in enhanced definition of cheRNA transcription start sites as defined by RNA Pol II enrichment [76,87]. As the majority of cheRNA are lncRNA, our TASSEL-based assembly marks the first successful use of a hybrid assembly method in enhanced detection as well as assembly of lncRNA. It can therefore be used to resolve open questions about lncRNA such as their true molecular identity, convergence with other genomic as well transcriptomic features, and the role of antisense lncRNA. Collectively, our analyses of short- and long-read RNA-seq attributes and the development of stranding and merging pipelines can inform and improve future long-read transcriptome analyses beyond the cheRNA transcripts tested here. Strengths and weaknesses of short- and long-read RNA-seq As transcript assembly is critical for most of the downstream analyses, we systematically compared multiple transcript assembly approaches. Consistent with previous studies [53,54], reference-guided assembly with StringTie2 showed higher sensitivity as well as precision for transcript assembly for both short- and long-read RNA-seq, relative to other tested programs. A large fraction of long-read transcripts was fully contained within the reference intron, perhaps largely due to the enrichment of nascent transcripts in chromatin fractions of nuclear RNA [59–61] or limitations of long-read sequencing such as low coverage and internal priming events of oligo-dT based primers. We observed a high overlap between transcripts assembled in the short- and long-read datasets. A direct comparison of the structure of transcripts showed that many in the long-read datasets were “contained” in the short-read transcripts, largely due to the higher depth of the short-read sequencing. Therefore, multiple limitations of long-read sequencing can cause truncation of assembled transcripts at low coverage regions. The lower depth of long-read assembly also impacted the quantitation of low abundance transcripts when compared to the short-read transcriptome, cautioning its use for pan-transcriptome quantitation on its own. SLURP strands and improves transcript assembly The lack of strand of origin information in the ONT direct and PCR-cDNA long-read sequencing hinders direct comparison to reference transcriptomes and short-read assemblies. Consistent with this limitation, we report clear evidence of mapping of ~40% of long reads to the incorrect strand and consequent ambiguous assembly of transcripts. The existing stranding methods discard a majority of reads leading to loss of information critical for high fidelity transcript assembly. To address this problem, we developed and benchmarked an accessible computational stranding pipeline, SLURP. Using SLURP, we corrected the erroneous mapping of long reads, resulting in a striking improvement in the assembly of spiked-in ERCC and sequin transcripts as well as assembly of transcriptomes from human and mouse cell line datasets. SLURP outperforms other stranding pipelines such as UNAGI and Pychopper, perhaps due to the way the sequences are searched within the reads and/or tolerance of mismatches. Importantly, SLURP stranding is effective in improving the assembly of mono-exonic as well as complex multi-exonic transcripts with spliced isoforms. Although SLURP strands 60–75% reads, the loss of remaining unstranded reads does not seem to hamper the sensitivity of transcript detection or assembly. On the contrary, SLURP stranding leads to an overall increase in the sensitivity and precision of transcript assembly (Figs 3H and S3L) due to the removal of the underlying noise caused by misaligned reads that otherwise reduces the confidence of mapping in the correct orientation and creates discontinuity in intact transcript assembly (Figs 3I and S3O). StringTie Mix [29], a hybrid assembly program, permitted comparable stranding efficiency but resulted in truncated transcripts, seemingly due to overweighting of short reads in the hybrid assembly. Unlike StringTie Mix, SLURP can be used as a general-purpose standalone pipeline without the corresponding short-read data and independent of the assembly program. Importantly, our stranding pipeline can be broadly applied to preexisting long-read RNA-seq studies for improved transcript assembly, comparison and integration with short-read RNA-seq data, and high-confidence detection of bona fide antisense transcripts shown to play a critical role in gene regulation [81]. Prevalent artifacts in direct ONT cDNA long-read sequencing Our in-depth stranding analysis led to the serendipitous discovery that long-read ONT cDNA libraries are plagued with a cDNA synthesis artifact. Surprisingly, ~6% of long-read cDNA libraries are palindromes (within base-calling error). As we did not detect such reads in the ONT direct RNA library, we attribute these reads to an artifact of first strand synthesis, wherein reverse transcriptase switches from its initial RNA template to its product cDNA strand for further elongation. This finding is distinct from previously reported reverse transcriptase artifacts such as artificial splicing, in which the reverse transcriptase jumps from one direct repeat to another direct repeat of the same RNA or different RNA molecule, causing the deletion of the intervening sequence [82,83]. There is less chance of formation and detection of such palindromes in the short-read datasets as short-read libraries are synthesized from fragmented RNA and sequenced with shorter read lengths, often with enzymes that lack template switching activity. Alignment using standard long-read aligners such as minimap2 leads to double mapping of these reads to the same genomic region and flagging of one of the two alignments as a supplementary alignment. However, such an alignment not only inflates coverage statistics at the alignment stage but also leads to imprecise assembly and quantitation of all transcripts. It is important for the field to be aware of this likely prevalent artifact in long-read cDNA libraries which, to our knowledge, has not yet been reported. In principle, alternate cDNA library preparations with non-template switching reverse transcriptases or future aligners and assembly algorithms adapted to resolve these palindromes can be employed to accommodate these pitfalls. Enhanced detection of molecular features promises to improve functional analyses Alternative transcription start and termination sites are shown to be the major drivers of isoform variance of transcripts [72]. Knowledge of transcript ends is also critical for downstream experimental manipulations to evaluate function. For example, efficiency of CRISPRi [84] targeting drops off sharply as a function of distance from the TSS [85]. Moreover, effective interpretation of experiments involving insertion of a strong polyadenylation sequence to make an RNA-level knockout [33,86] relies on accurate knowledge of the TSS to minimize residual RNA fragment size. In our study, direct comparison of short- and long-read alignments over known genes and curated transcription start and termination sites exposed informative differences between the two platforms. The ends of the transcripts are much better represented in the long-read alignments than in the short-read alignments. We reason that this difference may arise due to multiple factors such as incomplete cDNA synthesis from random hexamer primers on fragmented RNA and heterogenous mapping of segmented short reads. In contrast, full-length cDNA synthesis on unfragmented RNA using a 3’-based oligoT primer and potential sequencing of the entire cDNA culminate in end-to-end coverage of transcripts in the long-read data set. This provides a qualitative advantage to the long-read RNA-seq for full-length characterization of transcripts. TASSEL improves integration of short- and long-read transcript assembly Informed by the strengths and caveats of each of the two platforms, we developed a hybrid pipeline, TASSEL, to merge the short-read transcriptome with the stranded long-read transcriptome. When tested on ERCC standards against commonly used long-read transcript assembly methods such as FLAIR [55], Bambu [67], StringTie Mix [29], RNA-Bloom2 [68], IsONform [69], IsoQuant [70] and RATTLE [71], TASSEL outperformed other methods by assembling the highest number of matching spike-in transcripts. We further tested TASSEL on complex multi-exonic sequin standards and the human transcriptome in our HAP1 dataset. TASSEL had the lowest false negative rate of transcript assembly and unlike other methods, TASSEL led to assembly of complete transcripts on the correct strand. FLAIR was limited by low sensitivity and incomplete assembly. Although StringTie Mix was able to assemble transcripts on the correct strand, it had a substantially higher false negative rate than TASSEL and assembled many transcripts with truncated ends. We note that with lower stringency of detection, TASSEL may suffer from reduced precision. However, users can enhance the precision of TASSEL by increasing the stringency of detection while still achieving higher sensitivity than other methods tested here. Analyses based on spliced sequin spike-in standards as well as on HAP1 dataset indicate that TASSEL improves transcript assembly of complex multi-exonic transcripts with spliced isoforms. TASSEL also led to a substantial reduction in the transcripts that are assembled within an intron of the reference gene. We reason that this happens due to the amelioration of long-read artifacts such as low coverage and internal priming events of oligo-dT primers by short-read transcripts which are assembled from high depth libraries prepared without oligo-dT primers. Another reason for the higher efficacy of TASSEL is the use of stranded long reads which not only resolves the conflicting orientation of neighboring reads but also enhances the coverage of correctly assembled transcripts. Additionally, as TASSEL is employed after alignment and assembly of short- and long-read transcripts, it is not biased towards short-read transcript depth and maintains long-read information. As a test of its advantage at the functional level, TASSEL had higher enrichment of known TSS, TTS, histone modification of active TSS, and RNA Pol II than StringTie Mix. Together, TASSEL outperforms contemporary assembly methods for assembly on the correct strand, complete end-to-end assembly highest sensitivity of assembly, and unbiased integration of high depth short-read sequencing with the qualitative enrichment of long-read sequencing, which collectively culminates to superior manifestation of biologically relevant transcriptomic features. In addition to the ONT long-read sequencing, another prevalent long-read sequencing platform is PacBio Iso-seq [78]. We note that the inherent read correction module of Iso-seq data analysis correctly strands the long reads by orienting them in the 5’ to 3’ direction. We show that TASSEL-based merge of a transcriptome derived from stranded PacBio long reads with that derived from corresponding short reads improves transcript detection and assembly. We tested TASSEL on cheRNA transcripts which have been challenging to characterize due to their low abundance in cells, high variation from one cell type to another, and lack of canonical attributes of coding transcripts [37,38]. In comparison to short- or long-read data alone or combined analysis with StringTie Mix, the use of TASSEL led to a marked improvement in the assembly of cheRNA. It corrected the segmentation of many mono- as well as multi-exonic cheRNA genes, resulting in enhanced definition of cheRNA transcription start sites as defined by RNA Pol II enrichment [76,87]. As the majority of cheRNA are lncRNA, our TASSEL-based assembly marks the first successful use of a hybrid assembly method in enhanced detection as well as assembly of lncRNA. It can therefore be used to resolve open questions about lncRNA such as their true molecular identity, convergence with other genomic as well transcriptomic features, and the role of antisense lncRNA. Collectively, our analyses of short- and long-read RNA-seq attributes and the development of stranding and merging pipelines can inform and improve future long-read transcriptome analyses beyond the cheRNA transcripts tested here. Materials and methods Cell culture HAP1 cells were grown in IMDM media (Gibco, 12440–053), supplemented with 10% FBE (Seradigm, 3100–500) and 1% Pen/Strep (ThermoFisher 15140–122) to 80–90% confluency in T75 flasks at 37°C. HL1 mouse cardiomyocytes (#SCC065) were grown per manufacturer instructions in supplemented Claycomb Medium (51800C-500ML) + 10% FBS (TMS-016) + 0.1mM Norepinephrine with 30mM L-ascorbic acid (A0937, A7506) + 5% L-Glutamine (G7513) in 0.1% gelatin-coated (SF008) T75 flasks. To prepare the cells for RNA harvesting, they were washed once with 1X PBS, detached with TrypLE Express (Gibco #12605010), quenched with FBE-containing media, then washed with PBS, and pelleted by centrifugation for 5 min at 250 x g. Cell pellets were either snap frozen with liquid nitrogen or processed immediately. Nuclear fractionation and RNA isolation Cell pellets were resuspended in Buffer A (10 mM HEPES•KOH pH 7.5, 10 mM KCl, 10% glycerol, 340 mM sucrose, 4mM MgCl2, 1 mM DTT, 1 x Protease Inhibitor Cocktail (PIC) [1mM AEBSF, 0.8 μM aprotinin, 20 μM leupeptin, 15 μM pepstatin A, 40 μM bestatin, 15 μM E-64; from 200x DMSO stock]). Nuclear fractions were separated as described previously [37,38]. 1 μL of a 1:10 diluted ERCC RNA Spike-in standards aliquot (Life Technologies 4456740) was added to each nuclear fraction prior to addition of TRIzol reagent (Life Technologies 15596026). RNA from each fraction was extracted as described previously [37,38]. RNA was processed using the Zymo RNA Clean & Concentrator kit (Zymo Research R1017) with on-column DNase digestion. Ribosomal RNA was removed using the Ribo-zero Gold rRNA depletion kit (Illumina MRZG12324) for HL1 short-read RNA-seq samples and the RiboMinus kit (Invitrogen A15026) for HL1 long-read RNA-seq as well as HAP1 short- and long-read samples. Library preparation for short-read RNA-seq 100 ng of rRNA-depleted RNA from each nuclear fraction was used with the NEBNext Ultra II Directional library kit (NEB E7765S) to prepare RNA-seq libraries. RNA-seq libraries of nuclear fractions (chromatin pellet extract and soluble nuclear extract) from three independent cultures were sequenced through the University of Chicago Genomics Core Facility on the Illumina HiSeq 4000 to obtain 50 bp single end reads. Library prep for long-read RNA-seq Two HAP1 and two HL1 long-read sequencing libraries were generated as follows. RNA from each fraction was extracted with Zymo RNA Clean & Concentrator (R1019) and subjected to qPCR analysis (primer sequences in S2 Table) for confirmation of enrichment for respective fractions (see S1A Fig). Next, the chromatin fraction RNA was depleted for ribosomal RNA with the RiboMinus Eukaryote Kit v2 (A15020) and precipitated with ethanol, then polyadenylated with E. coli Poly(A) Polymerase (M0276S) (for later priming with long read sequencing primers) and precipitated with ethanol. Successful rRNA depletion was assessed using BioAnalyzer. Finally, rRNA-depleted and polyadenylated chromatin RNA was subjected to library preparation according to the Oxford Nanopore direct cDNA sequencing kit protocol (SQK:DCS109). Libraries were checked for purity and size on a TapeStation with genomic DNA reagents (5067–5366) and tape (5067–5365). 10–50 fmol of library was loaded onto a MinION (MIN-101B) flow cell (R9.4.1; FLO-MIN106D) and run for 48 hours. Real-time basecalling information and FASTQ files were generated with the MinKNOW software using default settings (HAP1 rep1: minQ 7; HAP1 rep2: minQ 7; HL1 rep1: minQ 7; HL1 rep2: minQ 9). Alignment of short reads Read quality and adapter content was checked using FastQC [88]. All short-read RNA seq libraries passed the quality check. Reads were then aligned to reference genomes (hg38 for HAP1 and mm10 for HL1 reads, catenated with ERCC sequences) using hisat2 version 2.1.0 [47] with—rna-strandness R—dta options. Sam files were converted to sorted bam files using samtools [89]. Long read processing and ERCC analysis FASTQ files generated by the MinKNOW software for each cell line were concatenated and mapped either to the hg38+ERCC (HAP1) or mm10+ERCC (HL1) genomes using minimap2 [46] without canonical splice sites (-ax splice -un). For stranding analyses, FASTQ files were subjected to our stranding pipeline prior to mapping (see Stranding methodology section below). For coverage analyses between replicates, HL1 replicate 1 was further filtered post-run to have a minimum phred score of 9 to match the minimum phred score of 9 used for replicate 2 using NanoFilt (-q 9). After sorting and indexing the BAM files (samtools), ERCC counts were computed using BEDTools coverage [58] and averaged between replicates; transcripts with count averages above 0 were kept. Then, coverage analyses were performed using Stringtie2 [54] with options -A to generate gene coverage in addition to transcripts and -L to specify long reads. Replicates were then analyzed in tandem using the Stringtie—merge function with expression estimation mode on (-e) to create a reference GTF with the union of transcripts between both replicates and subsequently re-estimate coverage for both genes and transcripts in individual replicates. Tags per million (TPM) was used for analysis of coverage. For the ERCC graphs, coverage was plotted against amount of ERCC added, length, and fraction of nucleotides mapped. Coverage of ERCCs was analyzed between long and short read datasets by first averaging ERCC coverage between replicates (excluding any transcripts with coverage of 0), and then plotting the averaged replicate coverages between long and short read datasets. Coverage analysis for all transcripts between long and short read datasets was performed by re-estimating coverage between the replicates with Stringtie2, then re-estimating coverage between long and short read re-estimated datasets and plotting the TPMs. As a supplementary comparison analysis, BEDTools counts vs StringTie TPMs were also plotted against each other for ERCCs. Datasets with sequin spike-in RNA standards The short- and long-read dataset containing sequin spike-in RNA data were obtained from SG-Nex consortium [48]. We used data from three replicates each for direct cDNA, PCR-cDNA and Illumina short-read sequencing of HCT116 samples that were spiked with sequin RNA standards. The sequin dataset for PacBio platform was obtained from the GEO database (accession # GSE172421). Correlation of alignment of reads For testing the correlation of alignment, bam files were converted to counts per million (CPM)-normalized bigwig files using the bamCoverage function of deepTools (-bs 1—normalizeUsing CPM) [90]. Then, the multiBigwigSummary function of deepTools was used to compute correlation of coverages of aligned reads at all genomic regions binned at 1 kb region (-bs 1000). Correlation between the replicates was then plotted using plotCorrelation function (-c pearson -p scatterplot—log1p). Metagene plots To plot the aligned short and long reads on the known genes: Coordinates of the known genes (mm10vm23 assembly for mouse genome and hg38v41 assembly for human genome) were obtained from the UCSC table browser [91]. Counts per Million (CPM)-normalized bigwig files (binned at 1 base pair) of aligned reads were then used to score mean coverage over the known genes using the computeMatrix function of deepTools in scale-regions mode (-b 1000 -a 1000—missingDataAsZero) and plotted using plotHeatmap. To plot the aligned short and long reads on transcription start sites and polyA sites: Coordinates of annotated transcription start sites were obtained from refTSS [50] and polyA sites from polyASite [51]. Mean coverage over these coordinates were then calculated using the reference-point mode of computeMatrix (-b 1000 -a 1000—missingDataAsZero) and plotted using plotHeatmap. Cell culture HAP1 cells were grown in IMDM media (Gibco, 12440–053), supplemented with 10% FBE (Seradigm, 3100–500) and 1% Pen/Strep (ThermoFisher 15140–122) to 80–90% confluency in T75 flasks at 37°C. HL1 mouse cardiomyocytes (#SCC065) were grown per manufacturer instructions in supplemented Claycomb Medium (51800C-500ML) + 10% FBS (TMS-016) + 0.1mM Norepinephrine with 30mM L-ascorbic acid (A0937, A7506) + 5% L-Glutamine (G7513) in 0.1% gelatin-coated (SF008) T75 flasks. To prepare the cells for RNA harvesting, they were washed once with 1X PBS, detached with TrypLE Express (Gibco #12605010), quenched with FBE-containing media, then washed with PBS, and pelleted by centrifugation for 5 min at 250 x g. Cell pellets were either snap frozen with liquid nitrogen or processed immediately. Nuclear fractionation and RNA isolation Cell pellets were resuspended in Buffer A (10 mM HEPES•KOH pH 7.5, 10 mM KCl, 10% glycerol, 340 mM sucrose, 4mM MgCl2, 1 mM DTT, 1 x Protease Inhibitor Cocktail (PIC) [1mM AEBSF, 0.8 μM aprotinin, 20 μM leupeptin, 15 μM pepstatin A, 40 μM bestatin, 15 μM E-64; from 200x DMSO stock]). Nuclear fractions were separated as described previously [37,38]. 1 μL of a 1:10 diluted ERCC RNA Spike-in standards aliquot (Life Technologies 4456740) was added to each nuclear fraction prior to addition of TRIzol reagent (Life Technologies 15596026). RNA from each fraction was extracted as described previously [37,38]. RNA was processed using the Zymo RNA Clean & Concentrator kit (Zymo Research R1017) with on-column DNase digestion. Ribosomal RNA was removed using the Ribo-zero Gold rRNA depletion kit (Illumina MRZG12324) for HL1 short-read RNA-seq samples and the RiboMinus kit (Invitrogen A15026) for HL1 long-read RNA-seq as well as HAP1 short- and long-read samples. Library preparation for short-read RNA-seq 100 ng of rRNA-depleted RNA from each nuclear fraction was used with the NEBNext Ultra II Directional library kit (NEB E7765S) to prepare RNA-seq libraries. RNA-seq libraries of nuclear fractions (chromatin pellet extract and soluble nuclear extract) from three independent cultures were sequenced through the University of Chicago Genomics Core Facility on the Illumina HiSeq 4000 to obtain 50 bp single end reads. Library prep for long-read RNA-seq Two HAP1 and two HL1 long-read sequencing libraries were generated as follows. RNA from each fraction was extracted with Zymo RNA Clean & Concentrator (R1019) and subjected to qPCR analysis (primer sequences in S2 Table) for confirmation of enrichment for respective fractions (see S1A Fig). Next, the chromatin fraction RNA was depleted for ribosomal RNA with the RiboMinus Eukaryote Kit v2 (A15020) and precipitated with ethanol, then polyadenylated with E. coli Poly(A) Polymerase (M0276S) (for later priming with long read sequencing primers) and precipitated with ethanol. Successful rRNA depletion was assessed using BioAnalyzer. Finally, rRNA-depleted and polyadenylated chromatin RNA was subjected to library preparation according to the Oxford Nanopore direct cDNA sequencing kit protocol (SQK:DCS109). Libraries were checked for purity and size on a TapeStation with genomic DNA reagents (5067–5366) and tape (5067–5365). 10–50 fmol of library was loaded onto a MinION (MIN-101B) flow cell (R9.4.1; FLO-MIN106D) and run for 48 hours. Real-time basecalling information and FASTQ files were generated with the MinKNOW software using default settings (HAP1 rep1: minQ 7; HAP1 rep2: minQ 7; HL1 rep1: minQ 7; HL1 rep2: minQ 9). Alignment of short reads Read quality and adapter content was checked using FastQC [88]. All short-read RNA seq libraries passed the quality check. Reads were then aligned to reference genomes (hg38 for HAP1 and mm10 for HL1 reads, catenated with ERCC sequences) using hisat2 version 2.1.0 [47] with—rna-strandness R—dta options. Sam files were converted to sorted bam files using samtools [89]. Long read processing and ERCC analysis FASTQ files generated by the MinKNOW software for each cell line were concatenated and mapped either to the hg38+ERCC (HAP1) or mm10+ERCC (HL1) genomes using minimap2 [46] without canonical splice sites (-ax splice -un). For stranding analyses, FASTQ files were subjected to our stranding pipeline prior to mapping (see Stranding methodology section below). For coverage analyses between replicates, HL1 replicate 1 was further filtered post-run to have a minimum phred score of 9 to match the minimum phred score of 9 used for replicate 2 using NanoFilt (-q 9). After sorting and indexing the BAM files (samtools), ERCC counts were computed using BEDTools coverage [58] and averaged between replicates; transcripts with count averages above 0 were kept. Then, coverage analyses were performed using Stringtie2 [54] with options -A to generate gene coverage in addition to transcripts and -L to specify long reads. Replicates were then analyzed in tandem using the Stringtie—merge function with expression estimation mode on (-e) to create a reference GTF with the union of transcripts between both replicates and subsequently re-estimate coverage for both genes and transcripts in individual replicates. Tags per million (TPM) was used for analysis of coverage. For the ERCC graphs, coverage was plotted against amount of ERCC added, length, and fraction of nucleotides mapped. Coverage of ERCCs was analyzed between long and short read datasets by first averaging ERCC coverage between replicates (excluding any transcripts with coverage of 0), and then plotting the averaged replicate coverages between long and short read datasets. Coverage analysis for all transcripts between long and short read datasets was performed by re-estimating coverage between the replicates with Stringtie2, then re-estimating coverage between long and short read re-estimated datasets and plotting the TPMs. As a supplementary comparison analysis, BEDTools counts vs StringTie TPMs were also plotted against each other for ERCCs. Datasets with sequin spike-in RNA standards The short- and long-read dataset containing sequin spike-in RNA data were obtained from SG-Nex consortium [48]. We used data from three replicates each for direct cDNA, PCR-cDNA and Illumina short-read sequencing of HCT116 samples that were spiked with sequin RNA standards. The sequin dataset for PacBio platform was obtained from the GEO database (accession # GSE172421). Correlation of alignment of reads For testing the correlation of alignment, bam files were converted to counts per million (CPM)-normalized bigwig files using the bamCoverage function of deepTools (-bs 1—normalizeUsing CPM) [90]. Then, the multiBigwigSummary function of deepTools was used to compute correlation of coverages of aligned reads at all genomic regions binned at 1 kb region (-bs 1000). Correlation between the replicates was then plotted using plotCorrelation function (-c pearson -p scatterplot—log1p). Metagene plots To plot the aligned short and long reads on the known genes: Coordinates of the known genes (mm10vm23 assembly for mouse genome and hg38v41 assembly for human genome) were obtained from the UCSC table browser [91]. Counts per Million (CPM)-normalized bigwig files (binned at 1 base pair) of aligned reads were then used to score mean coverage over the known genes using the computeMatrix function of deepTools in scale-regions mode (-b 1000 -a 1000—missingDataAsZero) and plotted using plotHeatmap. To plot the aligned short and long reads on transcription start sites and polyA sites: Coordinates of annotated transcription start sites were obtained from refTSS [50] and polyA sites from polyASite [51]. Mean coverage over these coordinates were then calculated using the reference-point mode of computeMatrix (-b 1000 -a 1000—missingDataAsZero) and plotted using plotHeatmap. Comparison of transcript assembly programs For long-read transcript assembly, we compared FLAIR, StringTie denovo and StringTie guided approaches. For sequin transcript assembly, either the complete reference annotation or random 50% annotations were provided during the guided mode of transcript assembly for each of the programs tested. To assemble transcripts with FLAIR, sorted bam files were converted to bed12 format using the bam2Bed12.py script of FLAIR. FLAIR correct was used with reference genome fasta and annotation files (Figs 5E and S2A) or short-read gtf files (Fig 5E) (flair correct—nvrna -q input.bed -g refgenome.fa -f refgenome.gtf/short-read.gtf -o FLAIRcorrect_output.bed). The corrected bed file was then used to obtain a gtf file using FLAIR collapse (flair collapse -g refgenome.fa -q FLAIR_corrected.bed -f refgenome.gtf/short-read.gtf -r sample.fq -o flair_collapse_output.gtf). For StringTie, sorted bam files for each sample were used to assemble transcripts using StringTie version 2.1.1 [54] with the -L option in either de novo or guided assembly mode (-G reference_annotation.gtf). For guided assembly, reference annotation gtf files of the gencode hg38v35 assembly for HAP1 samples and the mm10 assembly for HL1 samples were concatenated with gtf files for the ERCC spike-in RNA mix. For short-read transcript assembly, sorted bam files for each sample were used to assemble transcripts using cufflinks version 2.2.1 (-u -N—library-type fr-firststrand) [92] or StringTie version 2.1.1 (—rf) [54] in either de novo or guided assembly mode. In cufflinks guided assembly mode, reference genome transcripts that are not detected in the sample are appended to the sample gft files with their coverage marked as “0.0000”. As it would artifactually inflate the sensitivity of assembly, we removed all transcripts with zero coverage in the sample gtf files. To compare the efficacy of transcript assembly programs, gtf files of transcripts assembled by each of the programs was compared with corresponding reference genome gtf files using gffcompare [93]. Sensitivity (True positive / (True positive + False negative)) and Precision (True positive / (True positive + False positive)) of transcript assembly were compared to gauge the efficacy of assemblers. Testing stranding with UNAGI and Pychopper UNAGI [65] was used in the default mode (-i long-read.fastq -g ref_genome.fa). It yields a fastq file with stranded reads. Number of reads in this stranded fastq file was used to calculate percent stranding. Pychopper (https://github.com/epi2me-labs/pychopper) was run using the kit-specific primer configuration. Full length stranded reads yielded by Pychopper were then used to map to the reference genome using minimap2 as described above. SLURP (Stranding Long Unstranded Reads using Primers) stranding methodology We used MEME analysis [64] to predict the sequences of primers used in the cDNA library prep. These primer sequences were then used as guides to ascertain the strand of origin of long reads. For this, the following stranding methodology was adapted: 1. Reads with partial primer 1 sequence in the first 100 bp of reads, permitting 2 mismatches, were extracted using: seqkit grep -s -i -P -R 1:100 -m 2 -p GCTCTATCTTCTTT 2. Reads with the reverse complement of partial primer 2 in the last 100 bp, permitting 2 mismatches, were extracted using: seqkit grep -s -i -P -R -100:-1 -m 2 -p CCCAGCAATATCAG. 3. Reads from 1 and 2 were combined and deduplicated. 4. Reads with partial primer 2 sequence in the first 100 bp of reads, permitting 2 mismatches, were extracted using: seqkit grep -s -i -P -R 1:100 -m 2 -p CTGATATTGCTGGG. 5. Reverse complement of reads from (4) were made using: seqkit seq -t dna -r -p. 6. Reads from (3) and (5) were combined and deduplicated to obtain stranded reads. A bash script of this stranding pipeline is made available on at the GitHub repository (https://github.com/kainth-amoldeep/SLURP). We note that this default strategy will create a second strand library. If required, a first strand library can be created by obtaining the reverse complement of the SLURP output by using seqkit seq -t dna -r -p. Detection and analysis of the palindrome artifact in long-read sequencing We extracted common reads which had primer 1 in their first 100 bp (seqkit grep -s -i -P -R 1:100 -p GCTCTATCTTCTTT) and the reverse complement of primer 1 in their last 100 bp (seqkit grep -s -i -P -R -100:-1 -p AAAGAAGATAGAGC). These reads were then analyzed using RNAfold [94], as we reasoned that the secondary structure prediction of RNAfold could be used to detect potential palindromes in long reads. From the RNA-fold dot and bracket output (dot denotes unpaired residue and bracket denotes paired residue), we calculated the percentage of brackets in the given read to determine extent of residues base-paired in the long-reads. In addition, RNA-fold also provides minimum free energy (MFE) scores of residues in the reads. We binned and scaled MFE scores of multiple reads to 100 bp to obtain a meta-mountain plot of MFE scores. For mapping and counting statistics, the palindromic reads were aligned to the reference genome using minimap2 (-ax splice -un—MD). As palindromes were tagged with supplementary alignment to the same locus, we filtered out reads with the supplementary alignment sam tag using samtools (view -F 2048). The filtered and non-filtered bam files were then used to assemble and quantify transcripts using StringTie to compare coverage and abundance (TPM) of transcripts. Comparison of short- and long-read transcriptomes For comparing abundance of transcripts and genes in short- and long-read samples, StringTie-assembled gtf files of long-read samples (2 replicates) and short-read samples (3 replicates) were merged using stringtie—merge to obtain a non-redundant pool of a unified set of transcripts across samples. This merged transcriptome gtf file was then used as a reference for re-estimation of transcripts in each of the samples using stringtie -e -B -G stringtie_merge_file.gtf -C coverage_file -A gene_abundance_file. Then, average transcript and gene abundance (Tags per million, TPM) of every transcript and gene was calculated across the replicates for short-read and long-read samples. These average TPMs were compared to test the correlation between short-read sequencing and long-read sequencing. To test the overlap between short- and long-read transcripts, bedtools [58] intersect was used in default mode to obtain the number of long-read transcript with any overlap in the short-read data and vice versa. To compare the transcript structure assembled by short-read and long-read assembly, gffcompare was used with the gtf file of long-read assembly as a reference transcriptome and the gtf file of short-read assembly as the query transcriptome. The relationship between the reference and query transcripts was ascertained by the “class code” output of gffcompare. Enrichment of transcript ends in short and long reads CPM-normalized bedgraph files were made from sorted bam files of each sample using the bamCoverage function of deepTools (-bs 1—normalizeUsing CPM -of bedgraph). Then, sorted bedgraph files were used to calculate mean signal at the 100 bp region around the TSS or TTS of known genes using bedtools [58] (map -a TSS_100bp.bed -b CPMnormlized.bg -c 4 -o mean > bedtoolsmapped_TSS100bp_mean). The average signal thus obtained was then normalized to the average signal obtained from mapping reads to a similar number of random regions (100 bp wide) in the genome. Long-read gene fusion Name-sorted bam files of each long-read sample were used as input to detect fusion events using LongGF [95] with <min-overlap-len> 100 <bin_size> 30 <min-map-len> 100. Then, grep "SumGF" LongGF.log > FusionList was used to get a list of detected fusions. The list of fusion events was then used to make a circos plot using BioCircos [96]. TASSEL (Transcript Assembly using Short and Strand-Emended Long reads) Long reads were stranded using SLURP (described above). Stranded long reads and short reads were aligned to the reference genome using minimap2 and hisat2, respectively. StringTie2 was used to assemble transcripts in guided assembly mode (with -L option for long reads). Assembled transcripts were merged in a strand-aware manner using stringtie—merge—rf. Comparison of TASSEL with other assembly programs As recommended for RATTLE [71], Pychopper-processed reads were clustered in isoform mode (./rattle cluster -i processed_reads.fq—iso -o rattle_cluster/), followed by RATTLE correction (./rattle correct -i processed_reads.fq -c rattle_cluster /clusters.out -o./rattle_correct/) and RATTLE polish (./rattle polish -i./ rattle_correct /consensi.fq -o rattle_polish/—summary). The output “transcriptome” reads were mapped using minimap to obtain a paf file. The transcript models were then extracted from the aligned paf file to make a gtf file, subsequently used with gffcompare for the evaluation of transcript assembly. For IsoQuant [70], read alignments from minimap were used to assemble transcripts in the guided mode (isoquant.py—reference reference.fasta -g reference_annotat.gtf—complete_genedb—bam aligned_sorted.bam—data_type nanopore—report_novel_unspliced true -o isoquant_out/). The assembled transcripts in the output gtf file were then evaluated using gffcompare. As recommended for For RNA-Bloom2 [68], Pychopper-processed reads were used in the default mode (java -jar RNA-Bloom.jar -long processed.fq -outdir RNA-bloom_out/). We also tested short read-based correction of RNA-bloom2 assembly (java -jar RNA-Bloom.jar -long processed.fq -sef short-read.fastq -outdir RNA-bloom_out/). The output “rnabloom.transcripts.fa” reads were mapped using minimap to obtain a paf file. The transcript models were then extracted from the aligned paf file to make a gtf file, subsequently used with gffcompare for the evaluation of transcript assembly. As recommended for IsONform [69], Pychopper-processed reads were clustered in the ONT mode (isONclust—ont—fastq processed_reads.fq—outfolder./isONclust). The clustered fastq files were then made (isONclust write_fastq—N 1—clusters isONclust/final_clusters.tsv—fastq processed_reads.fq—outfolder isONclust/fastq_files), followed by correction using isONcorrect (run_isoncorrect—fastq_folder isONclust/fastq_files/—outfolder isONcorrect/correction/). The transcript isoforms were then obtained using isONform (isONform_parallel.py—fastq_folder isONcorrect_output/—outfolder isONform_output/—split_wrt_batches). The output “transcriptome.fq” was mapped using minimap to obtain a paf file. The paf file was consolidated to make a gtf file, subsequently used with gffcompare for the evaluation of transcript assembly. When comparing TASSEL to TALON [73], TAMA [74], FLAIR [55], Bambu [67] and StringTie Mix [29], we used 100% sequin annotations, 50% sequin annotations or the corresponding mammalian annotations. For TALON, we initialized the database with the reference annotations (talon_initialize_database—f sequin_annotations.gtf—g talon_sequinannotation—a talon_sequinannotation—o talon_sequinannotation_database). Then, we labeled the reads to flag for internal priming (talon_label_reads—f sample_input.sam—g reference_fasta.fa—o talonLabelreads_sampleoutput). Then we did replicate-based filtering of transcripts (talon_filter_transcripts—db talon_sequinannotation_database.db -a talon_sequinannotation—o transcript_clean) followed by extraction of a gtf file for transcript models observed by TALON (talon_create_GTF—db talon_sequinannotation_database.db -a talon_sequinannotation -b talon_sequinannotation—whitelist transcript_clean—observed -d dataset.csv—o talon_creategtf.gtf). For TAMA, the minimap2 output of long read alignment was used for the TAMA collapse module using the recommended TAMA high methodology (python tama_collapse.py -d merge_dup -x no_cap -a 300 -m 20 -z 300 -sj sj_priority -lde 3 -sjt 10 -s minimap2aligned_sampleinput.sam -f reference.fasta -p tama_high_sample). The output transcript models of “TAMA collapse” for all replicates were then merged using the “TAMA merge” module (tama_merge.py -f list_tamacollapseoutput.txt -p tamamerged). The merged bed file was converted to a gtf file for comparison to the reference annotation. For assembly with FLAIR, sorted bam files of combined replicates (stranded with SLURP in Figs 5B, 5C, 5E, S5B and S5E) were converted to bed12 format using the bam2Bed12.py script of FLAIR. FLAIR correct was used with reference genome fasta and annotation files or short-read gtf files (flair correct—nvrna -q input.bed -g refgenome.fa -f refgenome.gtf/short-read.gtf -o FLAIRcorrect_output.bed). The corrected bed file was then used to obtain a gtf file using FLAIR collapse (flair collapse -g refgenome.fa -q FLAIR_corrected.bed -f refgenome.gtf/short-read.gtf -r sample.fq -o flair_collapse_output.gtf). Bambu (version 3.1.1) was obtained from github/goekelab. An annotation file for Bambu was prepared using BambuAnnotations < prepareAnnotations(ref_annotations.gtf). Long-read alignments of combined replicates were obtained from minimap2 and were used to make a summarized experiment object using longread_Bambu_se <- Bambu(reads = longread.bam, annotations = BambuAnnotations, genome = ref.fa). Transcripts assembled by Bambu were extracted using Bambu_constructedAnnotations = longread_Bambu_se [assays(longread_Bambu_se)$fullLengthCounts > 0]. For StringTie Mix, bam files of combined short-read replicates and long reads replicates were obtained using hisat2 and minimap2, respectively. Reference annotation gtf files of the gencode hg38v35 assembly for HAP1 samples and the mm10 assembly for HL1 samples were concatenated with gtf files for the ERCC spike-in RNA mix. These files were then used with StringTie Mix (stringtie—mix short-read.bam long-read.bam -o stringtiemix.gtf -C stringtiemix -G reference.gtf). Stringency of transcript detection was increased by increasing the minimum number of reads required to assemble the transcripts using the -c option of StringTie (2, 4, 6, 8, 10, 15, 20, 25, 50, 75, 100, 150, 200, 250, 300). 50% reduced sequin annotations were provided during transcript assembly for each replicate of short- and long-read dataset. Assembled transcripts for each replicate were merged for the given program using the StringTie merge function as described above for StringTie, StringTie Mix and TASSEL. These merged transcriptomes, at each setting of -c option, were then used to test the sensitivity and precision of transcript assembly by comparing against the full set of sequin annotations using gffcompare. For PacBio data, subreads of a PacBio long-read run and short reads of an Illumina NextSeq 500 run for H1975 cell line, spiked-in sequin standards, were obtained from GSE172421. Long-reads were aligned with pbmm2 and subjected to assembly using the StringTie -L option. Short-reads were aligned and assembled as described above using hisat2 and StringTie. The transcriptome obtained from each platform was TASSEL merged and compared to sequin annotations using gffcompare. Enrichment of H3K4me3, RNA Pol II and PRO-seq at TSS Fastq files for H3K4me3 ChIP-seq in HAP1 cells (Input: SRR6671609, H3K4me3: SRR6671589) and RNA Pol II ChIP-seq for HAP1 cells (SRR2301045) [97] were downloaded from the GEO database. Reads were aligned to genome (hg38) using bowtie2 [98]. For H3K4me3, an input normalized bigwig file was made using the bamCompare function of deepTools (bamCompare -bs 1 -b1 H3K4me3_IP.bam -b2 Input.bam–operation ratio). For Pol II ChIP-seq, sorted bam files were directly converted to counts-normalized bigwig files using the bamCoverage function of deepTools (-normalizeUsing CPM). The bigwig files were then used to plot metagene profiles on TSSs of transcripts from StringTie Mix and TASSEL genes using the computeMatrix and plotProfile functions of deepTools. For PRO-seq, bigwig files of normalized PRO-seq signal in HAP1 cells were obtained from GSM5829596 [99]. Forward PRO-seq signals were contoured over plus strand TSS and reverse PRO-seq signals were contoured over negative strand TSS of TASSEL or StringTie Mix transcripts using the computeMatrix function and plotted using the plotProfile function of deepTools. Estimation of cheRNA StringTie and DESeq2 [100] were used to detect transcripts and genes which were significantly enriched in the chromatin fraction as compared to the nucleoplasm fraction in short-read sequencing. For this, gtf files of chromatin fraction reads and nucleoplasm fraction reads (three biological replicates each) were obtained by StringTie guided assembly of aligned reads. The transcripts from chromatin and nuclear fractions (as assembled by StringTie on the respective alignment files) are merged with default settings of StringTie—merge. Therefore, the use of a unified set of transcripts created by StringTie—merge to calculate ratiometric enrichment avoids the influence of sample-to-sample background heterogeneity as all the samples are re-evaluated against a common template. These gtf files were merged using default settings of stringtie—merge—rf to create a pool of high-confidence unified non-redundant transcripts across the samples for differential gene expression of known as well as de novo transcripts across experimental conditions. StringTie—merge also rectifies any incomplete transcript in a given sample if it detects a corresponding complete transcript in another sample. This unified and consolidated pool of transcripts across chromatin and nuclear fractions was then used as a reference transcriptome to re-estimate transcript structure and abundance in each of the replicates of both fractions using stringtie -e -B -G stringtie_merge.gtf. Raw counts of the re-estimated transcript and gene abundance were obtained using the prepDE.py script from StringTie. The raw counts were then used to calculate differentially expressed transcripts and genes in the chromatin vs nucleoplasm fractions using DESeq2. Transcripts and genes which were enriched more than four-fold in the chromatin fraction at adjusted p < 0.05 and had a length of more than 200 bp were deemed as cheRNA. Gene ontology for cheRNA proximal genes Protein coding genes closest to cheRNA genes were computed using the closestBed function of bedtools (-a cheRNA_genes -b refgenome_proteincodinggene). The identified genes were then uploaded to the DAVID gene ontology portal [101] to calculate enriched biological processes categories. Comparison and merge of cheRNA with long-read sequencing To plot long-read alignments on cheRNA genes: CPM-normalized bigwig files of long-read alignments were used to calculate mean coverage over cheRNA genes using the computeMatrix function of deepTools in scale-regions mode (-b 1000 -a 1000—missingDataAsZero) and plotted using plotHeatmap. To calculate the extent of segmentation of cheRNA genes, genes in the TASSEL-merged gtf file were further consolidated using bedtools merge (-s -c 4,6 -o distinct). Then we intersected these consolidated genes with cheRNA genes using intersectBed (-wa -wb -s) to obtain the number of consolidated genes that intersected more than once with different cheRNA genes, essentially counting the segmented cheRNA genes. Pol II occupancy at cheRNA Fastq files for RNA Pol II ChIP-seq for HAP1 cells (SRR2301045) [97] and eight-week old mouse heart (RNA Pol II IP: SRR489670; Input: SRR489681) [102] were downloaded from the GEO database. Reads were aligned to respective genomes (hg38/mm10) using bowtie2 [98]. For HAP1 Pol II ChIP-seq, sorted bam files were directly converted to bigwig files using the bamCoverage function of deepTools. For mouse Pol II, an input normalized bigwig file was made using the bamCompare function of deepTools (bamCompare -bs 1 -b1 Pol_IP.bam -b2 Input.bam—operation ratio). The bigwig files were then used to plot metagene profiles on TSSs of cheRNA genes using the plotProfile function. Testing stranding with UNAGI and Pychopper UNAGI [65] was used in the default mode (-i long-read.fastq -g ref_genome.fa). It yields a fastq file with stranded reads. Number of reads in this stranded fastq file was used to calculate percent stranding. Pychopper (https://github.com/epi2me-labs/pychopper) was run using the kit-specific primer configuration. Full length stranded reads yielded by Pychopper were then used to map to the reference genome using minimap2 as described above. SLURP (Stranding Long Unstranded Reads using Primers) stranding methodology We used MEME analysis [64] to predict the sequences of primers used in the cDNA library prep. These primer sequences were then used as guides to ascertain the strand of origin of long reads. For this, the following stranding methodology was adapted: 1. Reads with partial primer 1 sequence in the first 100 bp of reads, permitting 2 mismatches, were extracted using: seqkit grep -s -i -P -R 1:100 -m 2 -p GCTCTATCTTCTTT 2. Reads with the reverse complement of partial primer 2 in the last 100 bp, permitting 2 mismatches, were extracted using: seqkit grep -s -i -P -R -100:-1 -m 2 -p CCCAGCAATATCAG. 3. Reads from 1 and 2 were combined and deduplicated. 4. Reads with partial primer 2 sequence in the first 100 bp of reads, permitting 2 mismatches, were extracted using: seqkit grep -s -i -P -R 1:100 -m 2 -p CTGATATTGCTGGG. 5. Reverse complement of reads from (4) were made using: seqkit seq -t dna -r -p. 6. Reads from (3) and (5) were combined and deduplicated to obtain stranded reads. A bash script of this stranding pipeline is made available on at the GitHub repository (https://github.com/kainth-amoldeep/SLURP). We note that this default strategy will create a second strand library. If required, a first strand library can be created by obtaining the reverse complement of the SLURP output by using seqkit seq -t dna -r -p. Detection and analysis of the palindrome artifact in long-read sequencing We extracted common reads which had primer 1 in their first 100 bp (seqkit grep -s -i -P -R 1:100 -p GCTCTATCTTCTTT) and the reverse complement of primer 1 in their last 100 bp (seqkit grep -s -i -P -R -100:-1 -p AAAGAAGATAGAGC). These reads were then analyzed using RNAfold [94], as we reasoned that the secondary structure prediction of RNAfold could be used to detect potential palindromes in long reads. From the RNA-fold dot and bracket output (dot denotes unpaired residue and bracket denotes paired residue), we calculated the percentage of brackets in the given read to determine extent of residues base-paired in the long-reads. In addition, RNA-fold also provides minimum free energy (MFE) scores of residues in the reads. We binned and scaled MFE scores of multiple reads to 100 bp to obtain a meta-mountain plot of MFE scores. For mapping and counting statistics, the palindromic reads were aligned to the reference genome using minimap2 (-ax splice -un—MD). As palindromes were tagged with supplementary alignment to the same locus, we filtered out reads with the supplementary alignment sam tag using samtools (view -F 2048). The filtered and non-filtered bam files were then used to assemble and quantify transcripts using StringTie to compare coverage and abundance (TPM) of transcripts. Comparison of short- and long-read transcriptomes For comparing abundance of transcripts and genes in short- and long-read samples, StringTie-assembled gtf files of long-read samples (2 replicates) and short-read samples (3 replicates) were merged using stringtie—merge to obtain a non-redundant pool of a unified set of transcripts across samples. This merged transcriptome gtf file was then used as a reference for re-estimation of transcripts in each of the samples using stringtie -e -B -G stringtie_merge_file.gtf -C coverage_file -A gene_abundance_file. Then, average transcript and gene abundance (Tags per million, TPM) of every transcript and gene was calculated across the replicates for short-read and long-read samples. These average TPMs were compared to test the correlation between short-read sequencing and long-read sequencing. To test the overlap between short- and long-read transcripts, bedtools [58] intersect was used in default mode to obtain the number of long-read transcript with any overlap in the short-read data and vice versa. To compare the transcript structure assembled by short-read and long-read assembly, gffcompare was used with the gtf file of long-read assembly as a reference transcriptome and the gtf file of short-read assembly as the query transcriptome. The relationship between the reference and query transcripts was ascertained by the “class code” output of gffcompare. Enrichment of transcript ends in short and long reads CPM-normalized bedgraph files were made from sorted bam files of each sample using the bamCoverage function of deepTools (-bs 1—normalizeUsing CPM -of bedgraph). Then, sorted bedgraph files were used to calculate mean signal at the 100 bp region around the TSS or TTS of known genes using bedtools [58] (map -a TSS_100bp.bed -b CPMnormlized.bg -c 4 -o mean > bedtoolsmapped_TSS100bp_mean). The average signal thus obtained was then normalized to the average signal obtained from mapping reads to a similar number of random regions (100 bp wide) in the genome. Long-read gene fusion Name-sorted bam files of each long-read sample were used as input to detect fusion events using LongGF [95] with <min-overlap-len> 100 <bin_size> 30 <min-map-len> 100. Then, grep "SumGF" LongGF.log > FusionList was used to get a list of detected fusions. The list of fusion events was then used to make a circos plot using BioCircos [96]. TASSEL (Transcript Assembly using Short and Strand-Emended Long reads) Long reads were stranded using SLURP (described above). Stranded long reads and short reads were aligned to the reference genome using minimap2 and hisat2, respectively. StringTie2 was used to assemble transcripts in guided assembly mode (with -L option for long reads). Assembled transcripts were merged in a strand-aware manner using stringtie—merge—rf. Comparison of TASSEL with other assembly programs As recommended for RATTLE [71], Pychopper-processed reads were clustered in isoform mode (./rattle cluster -i processed_reads.fq—iso -o rattle_cluster/), followed by RATTLE correction (./rattle correct -i processed_reads.fq -c rattle_cluster /clusters.out -o./rattle_correct/) and RATTLE polish (./rattle polish -i./ rattle_correct /consensi.fq -o rattle_polish/—summary). The output “transcriptome” reads were mapped using minimap to obtain a paf file. The transcript models were then extracted from the aligned paf file to make a gtf file, subsequently used with gffcompare for the evaluation of transcript assembly. For IsoQuant [70], read alignments from minimap were used to assemble transcripts in the guided mode (isoquant.py—reference reference.fasta -g reference_annotat.gtf—complete_genedb—bam aligned_sorted.bam—data_type nanopore—report_novel_unspliced true -o isoquant_out/). The assembled transcripts in the output gtf file were then evaluated using gffcompare. As recommended for For RNA-Bloom2 [68], Pychopper-processed reads were used in the default mode (java -jar RNA-Bloom.jar -long processed.fq -outdir RNA-bloom_out/). We also tested short read-based correction of RNA-bloom2 assembly (java -jar RNA-Bloom.jar -long processed.fq -sef short-read.fastq -outdir RNA-bloom_out/). The output “rnabloom.transcripts.fa” reads were mapped using minimap to obtain a paf file. The transcript models were then extracted from the aligned paf file to make a gtf file, subsequently used with gffcompare for the evaluation of transcript assembly. As recommended for IsONform [69], Pychopper-processed reads were clustered in the ONT mode (isONclust—ont—fastq processed_reads.fq—outfolder./isONclust). The clustered fastq files were then made (isONclust write_fastq—N 1—clusters isONclust/final_clusters.tsv—fastq processed_reads.fq—outfolder isONclust/fastq_files), followed by correction using isONcorrect (run_isoncorrect—fastq_folder isONclust/fastq_files/—outfolder isONcorrect/correction/). The transcript isoforms were then obtained using isONform (isONform_parallel.py—fastq_folder isONcorrect_output/—outfolder isONform_output/—split_wrt_batches). The output “transcriptome.fq” was mapped using minimap to obtain a paf file. The paf file was consolidated to make a gtf file, subsequently used with gffcompare for the evaluation of transcript assembly. When comparing TASSEL to TALON [73], TAMA [74], FLAIR [55], Bambu [67] and StringTie Mix [29], we used 100% sequin annotations, 50% sequin annotations or the corresponding mammalian annotations. For TALON, we initialized the database with the reference annotations (talon_initialize_database—f sequin_annotations.gtf—g talon_sequinannotation—a talon_sequinannotation—o talon_sequinannotation_database). Then, we labeled the reads to flag for internal priming (talon_label_reads—f sample_input.sam—g reference_fasta.fa—o talonLabelreads_sampleoutput). Then we did replicate-based filtering of transcripts (talon_filter_transcripts—db talon_sequinannotation_database.db -a talon_sequinannotation—o transcript_clean) followed by extraction of a gtf file for transcript models observed by TALON (talon_create_GTF—db talon_sequinannotation_database.db -a talon_sequinannotation -b talon_sequinannotation—whitelist transcript_clean—observed -d dataset.csv—o talon_creategtf.gtf). For TAMA, the minimap2 output of long read alignment was used for the TAMA collapse module using the recommended TAMA high methodology (python tama_collapse.py -d merge_dup -x no_cap -a 300 -m 20 -z 300 -sj sj_priority -lde 3 -sjt 10 -s minimap2aligned_sampleinput.sam -f reference.fasta -p tama_high_sample). The output transcript models of “TAMA collapse” for all replicates were then merged using the “TAMA merge” module (tama_merge.py -f list_tamacollapseoutput.txt -p tamamerged). The merged bed file was converted to a gtf file for comparison to the reference annotation. For assembly with FLAIR, sorted bam files of combined replicates (stranded with SLURP in Figs 5B, 5C, 5E, S5B and S5E) were converted to bed12 format using the bam2Bed12.py script of FLAIR. FLAIR correct was used with reference genome fasta and annotation files or short-read gtf files (flair correct—nvrna -q input.bed -g refgenome.fa -f refgenome.gtf/short-read.gtf -o FLAIRcorrect_output.bed). The corrected bed file was then used to obtain a gtf file using FLAIR collapse (flair collapse -g refgenome.fa -q FLAIR_corrected.bed -f refgenome.gtf/short-read.gtf -r sample.fq -o flair_collapse_output.gtf). Bambu (version 3.1.1) was obtained from github/goekelab. An annotation file for Bambu was prepared using BambuAnnotations < prepareAnnotations(ref_annotations.gtf). Long-read alignments of combined replicates were obtained from minimap2 and were used to make a summarized experiment object using longread_Bambu_se <- Bambu(reads = longread.bam, annotations = BambuAnnotations, genome = ref.fa). Transcripts assembled by Bambu were extracted using Bambu_constructedAnnotations = longread_Bambu_se [assays(longread_Bambu_se)$fullLengthCounts > 0]. For StringTie Mix, bam files of combined short-read replicates and long reads replicates were obtained using hisat2 and minimap2, respectively. Reference annotation gtf files of the gencode hg38v35 assembly for HAP1 samples and the mm10 assembly for HL1 samples were concatenated with gtf files for the ERCC spike-in RNA mix. These files were then used with StringTie Mix (stringtie—mix short-read.bam long-read.bam -o stringtiemix.gtf -C stringtiemix -G reference.gtf). Stringency of transcript detection was increased by increasing the minimum number of reads required to assemble the transcripts using the -c option of StringTie (2, 4, 6, 8, 10, 15, 20, 25, 50, 75, 100, 150, 200, 250, 300). 50% reduced sequin annotations were provided during transcript assembly for each replicate of short- and long-read dataset. Assembled transcripts for each replicate were merged for the given program using the StringTie merge function as described above for StringTie, StringTie Mix and TASSEL. These merged transcriptomes, at each setting of -c option, were then used to test the sensitivity and precision of transcript assembly by comparing against the full set of sequin annotations using gffcompare. For PacBio data, subreads of a PacBio long-read run and short reads of an Illumina NextSeq 500 run for H1975 cell line, spiked-in sequin standards, were obtained from GSE172421. Long-reads were aligned with pbmm2 and subjected to assembly using the StringTie -L option. Short-reads were aligned and assembled as described above using hisat2 and StringTie. The transcriptome obtained from each platform was TASSEL merged and compared to sequin annotations using gffcompare. Enrichment of H3K4me3, RNA Pol II and PRO-seq at TSS Fastq files for H3K4me3 ChIP-seq in HAP1 cells (Input: SRR6671609, H3K4me3: SRR6671589) and RNA Pol II ChIP-seq for HAP1 cells (SRR2301045) [97] were downloaded from the GEO database. Reads were aligned to genome (hg38) using bowtie2 [98]. For H3K4me3, an input normalized bigwig file was made using the bamCompare function of deepTools (bamCompare -bs 1 -b1 H3K4me3_IP.bam -b2 Input.bam–operation ratio). For Pol II ChIP-seq, sorted bam files were directly converted to counts-normalized bigwig files using the bamCoverage function of deepTools (-normalizeUsing CPM). The bigwig files were then used to plot metagene profiles on TSSs of transcripts from StringTie Mix and TASSEL genes using the computeMatrix and plotProfile functions of deepTools. For PRO-seq, bigwig files of normalized PRO-seq signal in HAP1 cells were obtained from GSM5829596 [99]. Forward PRO-seq signals were contoured over plus strand TSS and reverse PRO-seq signals were contoured over negative strand TSS of TASSEL or StringTie Mix transcripts using the computeMatrix function and plotted using the plotProfile function of deepTools. Estimation of cheRNA StringTie and DESeq2 [100] were used to detect transcripts and genes which were significantly enriched in the chromatin fraction as compared to the nucleoplasm fraction in short-read sequencing. For this, gtf files of chromatin fraction reads and nucleoplasm fraction reads (three biological replicates each) were obtained by StringTie guided assembly of aligned reads. The transcripts from chromatin and nuclear fractions (as assembled by StringTie on the respective alignment files) are merged with default settings of StringTie—merge. Therefore, the use of a unified set of transcripts created by StringTie—merge to calculate ratiometric enrichment avoids the influence of sample-to-sample background heterogeneity as all the samples are re-evaluated against a common template. These gtf files were merged using default settings of stringtie—merge—rf to create a pool of high-confidence unified non-redundant transcripts across the samples for differential gene expression of known as well as de novo transcripts across experimental conditions. StringTie—merge also rectifies any incomplete transcript in a given sample if it detects a corresponding complete transcript in another sample. This unified and consolidated pool of transcripts across chromatin and nuclear fractions was then used as a reference transcriptome to re-estimate transcript structure and abundance in each of the replicates of both fractions using stringtie -e -B -G stringtie_merge.gtf. Raw counts of the re-estimated transcript and gene abundance were obtained using the prepDE.py script from StringTie. The raw counts were then used to calculate differentially expressed transcripts and genes in the chromatin vs nucleoplasm fractions using DESeq2. Transcripts and genes which were enriched more than four-fold in the chromatin fraction at adjusted p < 0.05 and had a length of more than 200 bp were deemed as cheRNA. Gene ontology for cheRNA proximal genes Protein coding genes closest to cheRNA genes were computed using the closestBed function of bedtools (-a cheRNA_genes -b refgenome_proteincodinggene). The identified genes were then uploaded to the DAVID gene ontology portal [101] to calculate enriched biological processes categories. Comparison and merge of cheRNA with long-read sequencing To plot long-read alignments on cheRNA genes: CPM-normalized bigwig files of long-read alignments were used to calculate mean coverage over cheRNA genes using the computeMatrix function of deepTools in scale-regions mode (-b 1000 -a 1000—missingDataAsZero) and plotted using plotHeatmap. To calculate the extent of segmentation of cheRNA genes, genes in the TASSEL-merged gtf file were further consolidated using bedtools merge (-s -c 4,6 -o distinct). Then we intersected these consolidated genes with cheRNA genes using intersectBed (-wa -wb -s) to obtain the number of consolidated genes that intersected more than once with different cheRNA genes, essentially counting the segmented cheRNA genes. Pol II occupancy at cheRNA Fastq files for RNA Pol II ChIP-seq for HAP1 cells (SRR2301045) [97] and eight-week old mouse heart (RNA Pol II IP: SRR489670; Input: SRR489681) [102] were downloaded from the GEO database. Reads were aligned to respective genomes (hg38/mm10) using bowtie2 [98]. For HAP1 Pol II ChIP-seq, sorted bam files were directly converted to bigwig files using the bamCoverage function of deepTools. For mouse Pol II, an input normalized bigwig file was made using the bamCompare function of deepTools (bamCompare -bs 1 -b1 Pol_IP.bam -b2 Input.bam—operation ratio). The bigwig files were then used to plot metagene profiles on TSSs of cheRNA genes using the plotProfile function. Supporting information S1 Fig. Supporting data for Fig 1. A. RT-qPCR confirmation of RNA fractionation into chromatin (purple) and nucleoplasm (green) extracts in the indicated samples. Protein coding transcripts (GAPDH and B2M) were used as representative of nucleoplasm enrichment while lncRNA (MALAT1 and PVT1) were used as controls for chromatin enrichment. Error bar indicates SD for replicates (n = 3 for short-read; n = 2 for long-read). B. Density plot of read lengths from long-read sequencing of HL1 replicates (n = 2). C. Correlation between HL1 replicates (n = 2) for genomic coverage of long-read alignments binned at 1 kb. Pearson correlation is shown. D. Correlation between HAP1 (left) and HL1 (right) replicates (n = 3) for genomic coverage of short-read alignments (top nucleoplasm fraction; bottom chromatin fraction) binned at 1 kb. Pearson correlations are shown for each plot. E. Average counts per million (average CPM, dark color; +/- SD, lighter color) of aligned reads across all ERCC transcripts, meta-scaled to 1000 nt, in the HL1 short-read (n = 3) and long-read (n = 2) samples. F. Relationship between the average fraction of ERCC transcripts covered by short (left) or long (right) read alignments as a function of their amount (attomoles) or length (nt) for HL1 samples. G. Relationship between length and CPKM (coverage per kb per million mapped reads) of known genes in HAP1 short- and long-read datasets. Average CPKM for replicates are plotted against length of genes on a log10 scale. Colors correspond to kernel density estimations of scatter plot distribution. H. Metagene plots of mapped read coverage (average CPM) for HL1 short- and long-read alignments, scaled to transcription start sites (TSS) and transcription termination sites (TTS) of known genes. I. Metagene plot of mapped read coverage (average CPM) for HAP1 long-read alignments, scaled to transcription start sites (TSS) and transcription termination sites (TTS) of a subset of known genes which are similar in length distribution (0.2–2 kb) to ERCC genes. Note that the similarity of profile of this plot with that of ERCC genes (main Fig 1D, right) suggests that apparent gene body coverage decay in Fig 1F right is due to meta scaling of coverage on genes longer than the median long read. J. Metagene plots of mapped read coverage (average CPM) for HL1 short (top) and long (bottom) reads, centered on curated transcription start sites (TSS; [50]) or termination sites (polyA; [51]). K. Zoomed-in metagene plots of mapped read coverage (average CPM) for HAP1 short reads, centered on curated transcription start sites (TSS; [50]) or termination sites (polyA; [51]). Note the similarity of decay in signal in this plot to that observed for ERCC coverage (main Fig 1D, left). L. Circos plot depicting gene fusions supported by at least 2 reads in HAP1 (top) and HL1 (bottom) long-read datasets. Dark blue arcs indicate fusion events detected in both replicates; light blue arcs indicate fusion events detected in only one of the two replicates. The BCR::ABL fusion, a key genomic fusion, is highlighted in red. https://doi.org/10.1371/journal.pcbi.1011576.s001 (PDF) S2 Fig. Supporting data for Fig 2. A. Comparison of the indicated short-read (left) and long-read (right) transcript assembly methods for ERCC, sequin and HAP1 transcripts. Shown are the sensitivity and precision of assembly at the transcript level as computed by gffcompare. Note that apparent (app.) sensitivity and precision indicate comparison to reference human annotations which does not reflect ground truth for a given cell line. Error bar indicates SD (n = 2–3). B. Top: Average abundance (Tags Per Million; TPM) of ERCC transcripts determined by StringTie versus amount added (attomoles) or length (nt) of the transcript in HL1 samples (n = 3 for short reads, n = 2 for long reads). Bottom: Average abundance (Tags Per Million; TPM) of sequin spike-in transcripts determined by StringTie versus amount added or length (nt) of the transcript in SGNex direct-cDNA samples of HCT116 samples (n = 3 for short and long reads). Short-read plots are shown in red (left), long-read plots in blue (right). R2, correlation coefficient for linear regression. C. Comparison of abundance (TPM) of ERCC transcripts obtained from StringTie with BEDTools counts for HL1 long-read replicate 1. Both methods performed nearly identically (R2 = 0.99, correlation coefficient for linear regression). D. Correlation between HL1 replicates for transcriptome assembly by StringTie. Shown are the scatter plots of abundance (TPM) of each transcript in the two replicates. Colors correspond to kernel density estimations of scatter plot distribution. R2, correlation coefficient for linear regression. E. Comparison of structure of transcripts assembled by StringTie for HAP1 and HL1 short-read samples with structure of reference genome transcripts. The class codes for relationship between the assembled transcript and the closest reference transcript were deduced from gffcompare. F. Correlation plot for the average abundance (TPM) of ERCC spike-in transcripts in the short-read datasets (n = 3) with the average abundance in the long-read datasets (n = 2) of HL1 samples, determined by StringTie. Shown are the transcripts detected by both sequencing platforms. G. Scatter plot comparing the average abundance (TPM) of all transcripts in the short-read datasets (n = 3) and the long-read datasets (n = 2) in HL1 samples. Abundance was calculated using the re-estimation function of StringTie from the merged transcriptome. Color bars correspond to kernel density estimations of scatter plot distribution. H. Histograms of length of transcripts assembled in the short- and long-read transcriptome. The distribution is shown for transcripts up to 10 kb in length. I. Total number of transcripts assembled in the HL1 short-read dataset as a function of percent of short reads sampled. Dotted line indicates equivalent coverage compared to the corresponding long-read dataset in terms of total mapped bases. https://doi.org/10.1371/journal.pcbi.1011576.s002 (PDF) S3 Fig. Supporting data for Fig 3. A. Percent of long reads stranded by UNAGI [65], Pychopper, and SLURP (Stranding Long Unstranded Reads using Primers) in the HAP1 and HL1 datasets. B. Detection of primer 1 and primer 2 sequences in the long reads using MEME [64] de novo motif discovery. C. Measurement of indicated stranding methods in terms of the number of transcripts assembled and extent of transcripts mapping to the wrong strand of the reference genome. 3crit indicates using primer 1, primer 2 and reverse complement of primer 2; 4crit indicates primer 1, reverse complement of primer 1, primer 2 and reverse complement of primer 2; m2 indicates allowing 2 mismatches and m3 indicates allowing 3 mismatches. D. Workflow of SLURP pipeline. Reads that contain the first-strand synthesis primer (primer 1) or reverse complement (rc) of the strand switching primer (primer 2) are non-redundantly merged with the reverse complement of reads that contain the strand switching primer. E. Comparison of Pychopper and SLURP for the precision of assembling the ERCC (left) or all (right) transcripts in the HAP1 dataset. F. Comparison of Pychopper and SLURP for the total number of matching ERCC (left) or all (right) transcripts in the HAP1 dataset. G. Example of an ERCC transcript correctly assembled using SLURP where Pychopper failed to do so. H. Comparison of Pychopper and SLURP for the sensitivity and precision of transcript assembly by FLAIR in the HAP1 dataset. I. Comparison of Pychopper and SLURP for the sensitivity and precision of transcript assembly, total number of matching transcripts, and the computational time required for the stranding of reads in the HL1 dataset. J. Same as I, for the HCT116 dataset from the SG-Nex consortium. K. Example of a sequin spike-in transcript annotation with two spliced isoforms that vary in TSS and TTS, that is correctly assembled by SLURP-stranded long reads where the original unstranded reads failed to do so. L. Comparison of SLURP-mediated change in the sensitivity and precision of the sequin spike-in standards for the subset of splice variants with alternative TSS or TTS, e.g., the two transcript isoforms of R2_20 in panel K. M. Efficacy of the stranding pipeline in a previously published dataset [48]. Shown are the percentage of transcripts mapping to the incorrect strand before and after SLURP-stranding. N. Efficacy of the SLURP-stranding pipeline upon retention or removal of unstranded long reads in the HAP1 dataset. Shown are the percentage of transcripts mapping to the incorrect strand. O. Example of a known gencode transcript assembled by SLURP-stranded long reads where the original unstranded reads failed to do so in the HAP1 dataset. https://doi.org/10.1371/journal.pcbi.1011576.s003 (PDF) S4 Fig. Supporting data for Fig 4. A. Bar plot of percentage of paired bases in the artifactual reads as a test for palindromes in the reads. RNAfold was used to determine paired bases in the secondary structure where a palindrome outweighs other structures. https://doi.org/10.1371/journal.pcbi.1011576.s004 (PDF) S5 Fig. Supporting data for Fig 5. A. Ends of the 92 ERCC transcripts (arranged in increasing order of length) assembled by TASSEL (magenta circle) and StringTie Mix (StMix, green diamond) in the HL1 dataset. Gray bar indicates actual transcript. The color bar indicates the abundance of the given transcript. B. Number and extent of ERCC standard transcripts assembled by the indicated assembly method in the HAP1 dataset. StMix: StringTie Mix. SLURP-stranded reads were used for FLAIR assembly. C. Number of matching sequin transcripts assembled by the indicated method when no reference sequin annotation (left) or 50% of the reference annotations (middle and right) were provided to TAMA at the merge stage. Note that no sequin annotation was provided to StringTie Mix or TASSEL at the merge stage for any of the comparison points here. D. Number of total matching sequin transcripts assembled by the indicated assembly method with ONT PCR-cDNA sequencing kit. E. Sensitivity of the indicated assembly methods at the locus level for the HAP1 dataset. Transcriptome assembled by the given method in the HAP1 dataset was compared against reference annotation (gencode hg38v35) using gffcompare. SLURP-stranded reads were used for FLAIR assembly. Apparent sensitivity indicates use of gencode annotation as ground truth. F. Percent of assembled transcripts that match completely with a transcript (left) or are contained within an intron (right) of the reference transcript, using only long-read assembly, StringTie Mix or TASSEL in the HAP1 and HL1 datasets. G. Comparison of inclusion or omission of SLURP stranding in TASSEL. Shown are percentage of transcripts mapped to the opposite strand (top) or completely matching (bottom) the reference transcripts in the HAP1 dataset. H. Number of transcripts contributed by short- and long-read assemblies to the TASSEL transcriptome in the sequin and HAP1 datasets. Long or short-read transcripts were used as a query against the TASSEL merged transcriptome as a reference in gffcompare. I. Proximity of TSS (left) and TTS (right) of known genes (gencode hg38v41) to the TSS and TTS of consensus transcripts in TASSEL or those of short or long only transcripts in the HAP1 dataset. J. Enrichment of H3K4me3 (left, normalized to input) and RNA Pol II (middle, normalized to the total number of mapped reads) and average PRO-seq signal (right) at the TSS of consensus transcripts in TASSEL or those of short or long only transcripts in the HAP1 dataset. H3K4me3 and RNA Pol II occupancy was calculated from ChIP-seq data from HAP1 cells [97,103]. Normalized PRO-seq data in HAP1 cells were obtained from [99]. K. Example of a gencode hg38v41 transcript assembled by TASSEL where the short- or long-read assembly failed to do so. L. Sensitivity and precision of sequin transcript assembly using short-read, long-read or TASSEL assembly with long read data from the PacBio sequencing platform. M. Relative depth of short to long read coverage (mapped bases) in the indicated datasets. https://doi.org/10.1371/journal.pcbi.1011576.s005 (PDF) S6 Fig. Supporting data for Fig 6. A. Average count of genes in the chromatin and nucleoplasm fractions in HAP1 (left) and HL1 (right). Normalized gene counts obtained through DESeq2 were averaged for replicates (n = 3). B. Gene ontology enrichment analyses of protein-coding genes closest to cheRNA genes detected in HAP1 and HL1 datasets. Top 15 categories detected by DAVID [101] and associated number of genes are shown. Color bars correspond to false discovery rate (FDR). C. Histogram depicting the extent of overlap between cheRNA genes from short-read assembly and genes assembled from long-read assembly in HAP1 and HL1 samples. D. Comparison of the abundance (normalized DESeq2 gene count) of cheRNA genes showing minimum (0–10%) and maximum (90–100%) overlap with a corresponding long-read gene. Data points outside 1.5x of Inter-quartile range were removed as outliers. **** p < 0.0001, Two-tailed Mann-Whitney test. E. Prevalence of mono and multi-exonic cheRNA genes in segmented or unsegmented cheRNA genes. F. Metagene plot depicting average RNA Pol II occupancy (solid lines; shaded regions indicate SE) at the TSS (±1Kb) of segmented, unsegmented and TASSEL-refined cheRNA genes in the HL1 dataset. RNA Pol II occupancy was calculated from ChIP-seq data from heart of an eight-week-old mouse [102]. https://doi.org/10.1371/journal.pcbi.1011576.s006 (PDF) S1 Note. Long-read alignments capture novel cell line-specific gene fusion events. https://doi.org/10.1371/journal.pcbi.1011576.s007 (DOCX) S1 Table. Sequencing output of short- and long-read RNA-seq. https://doi.org/10.1371/journal.pcbi.1011576.s008 (PDF) S2 Table. Primers used for RT-qPCR analyses. https://doi.org/10.1371/journal.pcbi.1011576.s009 (PDF) Acknowledgments We thank Peter Faber, Lindsay Scarpitta, and Mikayla Marchuk in the University of Chicago Functional Genomics Facility for Illumina sequencing. We thank Dr. Jonathan Göke and other members of the SG-Nex consortium for providing relevant files of sequin annotations. We thank members of the Ruthenburg and Ivan Moskowitz labs at the University of Chicago for helpful suggestions to improve the manuscript.