Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Using deep learning to predict the outcome of live birth from more than 10,000 embryo data

Using deep learning to predict the outcome of live birth from more than 10,000 embryo data Background: Recently, the combination of deep learning and time-lapse imaging provides an objective, standard and scientific solution for embryo selection. However, the reported studies were based on blastocyst formation or clinical pregnancy as the end point. To the best of our knowledge, there is no predictive model that uses the outcome of live birth as the predictive end point. Can a deep learning model predict the probability of live birth from time- lapse system? Methods: This study retrospectively analyzed the time-lapse data and live birth outcomes of embryos samples from January 2018 to November 2019. We used the SGD optimizer with an initial learning rate of 0.025 and cosine learn- ing rate reduction strategy. The network is randomly initialized and trained for 200 epochs from scratch. The model is quantitively evaluated over a hold-out test and a 5-fold cross-validation by the average area under the curve (AUC) of the receiver operating characteristic (ROC) curve. Results: The deep learning model was able to predict live birth outcomes from time-lapse images with an AUC of 0.968 in 5-fold stratified cross-validation. Conclusions: This research reported a deep learning model that predicts the live birth outcome of a single blasto- cyst transfer. This efficient model for predicting the outcome of live births can automatically analyze the time-lapse images of the patient’s embryos without the need for manual embryo annotation and evaluation, and then give a live birth prediction score for each embryo, and sort the embryos by the predicted value. Keywords: Time-lapse microscopy, Embryo development, Embryo quality, Pregnancy Introduction higher risk of complications [3–6]. Therefore, with the Since Louis Brown was born, the first test tube baby [1], development of assisted reproductive technology, single more than seven million babies have been born around embryo transfer has gradually become the first choice of the world attribute to assisted reproduction technology IVF. However, single embryo transfer still faces an urgent (ART) [2]. In the early stage of IVF technology develop- problem: how to choose the best embryo to transfer to ment, multiple embryo transfer was the main transfer maintain the ideal success rate [7]. The trend of choosing method. However, multiple pregnancy was often accom- single embryo transfer is closely related to the improve- panied by premature delivery, more expenditure and ment and progress of embryo selection technology. Therefore, embryo identification and selection technol - ogy are particularly important and significant. In order *Correspondence: [email protected]; [email protected] to solve this problem, scholars have developed several Reproductive Medicine Center, Tongji Hospital, Tongji Medicine College, methods for identifying and selecting the best embryos Huazhong University of Science and Technology, Wuhan, People’s for transfer, such as: blastocyst culture, time-lapse pho- Republic of China School of Computer Science and Technology, Harbin Institute tography imaging system and pre-transfer genetic testing of Technology, Weihai 264209, China [8–10]. © The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Huang et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 2 of 7 Embryologists evaluated and observed the embryos large sample of single blastocyst transfer to obtain an effi - used optical microscope, which was taken out from the cient predictive model. conventional incubator at a specific time point during the first 5 days of life before the time-lapse imaging system Materials and methods was applied to the clinic [11]. Because of this disadvan- Patients tage, many events in the embryonic development process This was a noninterventional, retrospective, single-center have been missed [12]. And the emergence of time- cohort study of patients undergoing routine practice. lapse photography technology had just made up for this In order to reflect the broad range of patients typically shortcoming. encountered in clinical practice, no inclusion/exclu- Embryologists use the time-lapse photography sys- sion criteria were applied on baseline characteristics. tem to observe and evaluate the embryo that in a stable The time-lapse embryo data used in our work are col - environment, rather than exposed in a variable condition lected from Reproductive Medicine Center of Tongji (such as changing gas composition, unstable humidity, Hospital, Huazhong University of Science and Technol- insecure temperature and movement conditions), and ogy, Wuhan, China. The whole dataset contains 33,738 can obtain a lot of information between embryo develop- embryo samples captured by Embryoscope Plus time- ment, time and embryo potential [13, 14]. lapse microscope system. The fertilization time of these Scholars have introduced the mathematical technol- embryos were from January 2018 to November 2019, and ogy of artificial intelligence into ART, in order to acquire we continuously pay return visits until January 2021 to more information from the pictures obtained by the TL confirm whether these IVF treatments lead to live birth system, which may trigger a revolution. AI is a term that outcomes. All patients signed written informed consent can be divided into many areas, such as: artificial neu - and underwent the routine clinical treatment performed ral network (ANN), fuzzy logic, genetic algorithm (GA), in our center. No additional intervention was performed. machine learning and deep learning [15, 16]. The emergence of time-lapse incubation makes it pos - sible to record the complete cycle of an embryo from a Ethical approval blastomere to a blastocyst, when all morphokinetic fea- The study conformed to the Declaration of Helsinki for tures centralized [17]. Meanwhile, owing to its abundant Medical Research involving Human Subjects. It was time-lapse data, time-lapse incubation emerges up many approved by the Ethical Committee of Reproductive new research ideas combined with deep learning tech- Medicine Center, Tongji Hospital, Tongji Medicine Col- nology which is known as a data driven method. Deep lege, Huazhong University of Science and Technology. learning can uncover numerous subtle features which may not be paid attention to manually but do help the corresponding classification or prediction. When fed Dataset with enough well labeled data, deep learning model have The classification of the outcome of each embryo was the ability to find an optimal representation of the given shown in Table  1. And the final indicator was live birth. dataset by continuously conducting back-propagation. The whole dataset contained 33,738 embryos with labels u Th s, we can explore the general pattern which lead to a of positive, negative, and pending, as shown in Fig.  1. specific mapping from data to our desired tasks. The pending embryos referred to the unthawed embryos The deep learning literature that has been reported on which could be exploited in our future work, but were embryo selection is a design study with blastocyst forma- excluded in the experiments of this paper. Meanwhile, tion or clinical pregnancy as the end point. To the best only the single blastocyst transfer embryos were col- of our knowledge, there is no research on deep learning lected, including fresh cycle and frozen-thaw cycle. models designed with the end of live birth outcome. In u Th s, the engaged dataset in this paper contained 15,434 this study, we want to analyze the data of single-center, embryos with positive and negative labels. Table 1 Classification of the outcome of each embryo involved Classification Outcome Positive Live birth after a complete pregnancy cycle Negative Fail to live birth or embryo discarded because of a failed or abnormal fertilization, grossly abnormal morphology or aneuploidy from preimplan- tation genetic testing Pending Embryo in storage and not yet used Huang  et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 3 of 7 Fig. 1 The outcomes of the embryos being studied Embryo culture and frozen‑embryo transfer (FET) Serum hCG was measured to diagnosis a pregnancy The methods used for sperm preparation, for IVF and 2 weeks after embryo transfer and then was tested serially embryo culture, have been described previously [18]. to monitor rising titers. A clinical pregnancy was defined Briefly, semen was collected in sterile containers by as the presence of a gestational sac with fetal heart activ- masturbation after 3–5 d of sexual abstinence and then ity observed on ultrasound examination 5 weeks after maintained at 37 °C for 30 min. After liquefaction, sam- oocyte retrieval [19]. The live birth outcome data were ples were analyzed for sperm concentration, motility obtained by telephone interview of the parents after and morphology according to the World Health Organ- delivery. ization criteria. The oocytes were incubated in G-IVF medium (Vitrolife) and fertilized 3 to 4 h after retrieval. Deep learning model Normal fertilization was defined as zygotes with two In this work, we designed an end-to-end deep learn- pronuclei (2PN) and fertilized oocytes were continu- ing model to predict live birth probability. We label our ously cultured in G1 medium for 2 more days. Then, embryo samples by 0 and 1 according to real live birth the embryos were transferred to G2 medium and con- outcomes, where 1 represents live birth whereas 0 rep- tinued to be cultured for 3 more days. The additional resents not. The designed supervised network regresses good-quality blastocysts were cryopreserved for sub- the discrete prediction value between 0 and 1 under the sequent frozen-embryo transfer (FET) cycles. For the guidance of ground truth labels. FET cycles, oral estradiol (Progynova, Bayer) was pro- The network structure consists of seven convolution vided, 2 mg/d from cycle day 1–4, 4 mg/d from day 5–8 modules and two fully connected layers. The first mod - and 6 mg/d from day 9–12. Transvaginal ultrasound ule contains three convolution blocks which represents scanning was performed to assess the endometrial a combination of a convolution layer, a batch normali- thickness and ovulation from day 13; the estradiol dos- zation layer and a following ReLU (Rectified Linear age was adjusted based on the endometrial thickness. Unit) as an activation function. As is widely known that Administration of 40 mg progesterone intramuscularly the residual block proposed in ResNet [20] is demon- was given when the endometrium reached a thickness strated effective in numerous classification tasks, the of 8 mm or maximum. Administration of 60–80 mg of subsequent six convolution modules who share the progesterone was provided for the following 5 days. same architecture are composed of three basic residual Blastocysts transfer was performed on day 6, after blocks and a convolution block. Feature maps are down 5 days of progesterone administration. sampled at the last convolution block of each module. Huang et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 4 of 7 Training strategies The whole network in this work can be described a Aimed at the extremely imbalance of the positive and ResNet like network, as shown in Table 2. but the num- negative samples, we implement the following measures ber of modules differs from that in benchmark struc - during the training term. In the cross-validation experi- ture. Also, the complexity of our model is much higher ment, we perform data augmentation after splitting the than the benchmark model, specifically reflected on the dataset according to Table  3. The specific method is as number of convolution kernels. follows: Firstly, we conduct abundant data augmenta- We utilize BCE-Loss (binary cross entropy loss) as tion measures, including affine transformations and ran - a loss function to guide the backpropagation during domly coarse dropout. Affine transformations refer to training term when the model constantly optimizes flip, translation, rotation, scaling, each operation occurs itself. Since the loss function calculate the distance randomly at a probability of 50 %. Coarse dropout means between output predictions and target labels, our pur- randomly drop some local pixels, the selected local pix- pose is to minimize the loss value. els are painted in solid black, we set the probability rang- ing from 2 to 5%. Secondly, we over sample the positive samples at a certain multiple, which equals to the ratio of positive and negative samples, i.e., sixteen in our experi- Table 2 Network structure of the proposed method. The basic ments. The original images captured by time-lapse incu - block is engaged from ResNet18 [20] bation are 800 pixels, which should be further resized to Layer Filter Size Output Size 224 for network training after data augmentation. We used the SGD optimizer with an initial learning Conv1_x 7 × 7, 64 224 × 224 3 × 3, 64 224 × 224 rate of 0.025 and cosine learning rate reduction strategy. 3 × 3, 128, stride 2 112 × 112 The network is randomly initialized and trained for 200 Conv2_x 112 × 112 3 × 3, 128 epochs from scratch. × 3 56 × 56 3 × 3, 128 3 × 3, 256, stride 2 Performance testing Conv3_x 56 × 56 3 × 3, 256 × 3 The model is quantitively evaluated over a 5-fold cross- 3 × 3, 256 28 × 28 validation by the average area under the curve (AUC) of 3 × 3, 512, stride 2 the receiver operating characteristic (ROC) curve. Conv4_x 28 × 28 3 × 3, 512 × 3 ROC curve connects all points described by true posi- 14 × 14 3 × 3, 512 tive rate and false positive rate under all possible thresh- 3 × 3, 1024, stride 2 olds, which is a boundary value between positive and Conv5_x 14 × 14 3 × 3, 1024 × 3 7 × 7 3 × 3, 1024 negative samples. Considering that true positive rate 3 × 3, 2048, stride 2 and false positive rate are in a trade-off relationship cor - Conv6_x 7 × 7 responding to thresholds, we can quantify the discrimi- 3 × 3, 2048 × 2 5 × 5 3 × 3, 2048 nating power by calculating the area under the curve, this 3 × 3, 2048 is so-called AUC. A binary classifier who has incompa - Conv7_x 5 × 5 3 × 3, 2048 rable discriminating power can possess an AUC value × 2 3 × 3 3 × 3, 2048 of 1, whereas the weakest who almost emerge the judge- 3 × 3, 2048 ment randomly possess an AUC value of 0.5, and a higher Fc1 Max pool 3 × 3 1 × 1 AUC value implies a better performance. AUC is more 2048-d fc Table 3 Result of the 5-fold cross-validation analysis Fold 1 Fold 2 Fold 3 Fold 4 Fold 5 AUC (n = 3812) (n = 3812) (n = 3811) (n = 3811) (n = 3811) 1 Test Train Train Train Train 0.970 2 Train Test Train Train Train 0.964 3 Train Train Test Train Train 0.968 4 Train Train Train Test Train 0.976 5 Train Train Train Train Test 0.960 Average 0.968 Average AUC, The mean area under the curve across 5 cross-validation steps Huang  et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 5 of 7 reasonable than accuracy especially in classification tasks with imbalance data. In order to comprehensively evaluate the performance of our model, we perform a hold-out test and a 5-fold cross-validation simultaneously [21]. In the hold-out test or so-called train-val-test approach, we randomly split the dataset in a ratio of 5:1:1 for training set, vali- dation set, and test set, respectively. In the latter evalu- ation method, we randomly divide our data into five parts with equal size, where the proportion of positive and negative samples in each separate is same. Then, five models should be trained. In each case, a specific subset is selected for validation while the remaining four subsets serve as a training set. Finally, we can figure out the mean AUC of the five folds to evaluate the performance on the whole dataset. Compared with hold-out test, cross- validation can eliminate the possible overestimating or underestimating caused by undesired sample division. Results Fig. 2 ROC curve for prediction of live birth. ROC, Receiver operating characteristic. AUC, area under the curve From January 2018 to November 2019, a total of 5913 cycles used the time-lapse culture system. Among them, some patients have not been transferred in fresh Discussion cycle, and their embryos have not yet been thawed. In This study is a preliminary study of deep learning with the end, 3382 fresh cycles and 3270 frozen-thaw trans- live birth data as the end point during the IVF cycle. Our fer (FET) cycles were included in the study and 33,738 results show that Timelapse images can be combined embryos samples were analyzed. Basic information of the with deep learning technology for clinical applications. patients included in this study was shown in Table 4. Morales et al. [22], Xu et al. [23] and Santos Filho et al. [24] used static images to assess embryo quality or select Roc the best embryos to be transferred in the absence of early Analysis of the ROC was shown in Fig.  2. The resulting embryo development data. These methods lack support AUC of this research to predict live birth on the testing of more comprehensive data. dataset was 0.968. Dirvanauskas et al. [25] used convolutional neural net- work (CNN) to predict the developmental stage of the 5‑fold cross‑validation embryo analyze by analyze embryo images obtained from Table 3 showed the results of 5-fold cross-validation. The the time-lapse photography system, with a success rate average value of AUC was 0.968. The AUC was reproduc - of 97.62%. However, this method does not have the abil- ible in individual train-validation runs. ity to predict pregnancy. Khosravi et al. developed a new framework (STORK) based on the inception of Google’s Hold‑out test model to predict the quality of embryos with an AUC as The AUC value of the conducted hold-out test was 0.957, high as 0.98. The study has a large sample size, complex which was evaluated on the test set. The result was com - model, and high accuracy, but it cannot be used to pre- parable with the 5-fold cross-validation. dict live births [26]. It is demonstrated that our model has a better performance when compared with existing Table 4 Basic information of the patients included in this study benchmark model, but it still deserves to be optimized Age (y) 30.4 ± 3.9 since it’s high complexity. Such a complex network Duration of infertility (y) 3.3 ± 2.3 requires considerable computing resources, so it depends Duration of stimulation (d) 10.4 ± 1.8 highly on hardware device. Basal FSH (IU/L) 7.4 ± 1.9 There is also a latest report that creates the predictive No. of oocytes retrieved 12.9 ± 4.2 model of blastocyst transfer [27]. The author analyzed the No. of mature oocytes 11.0 ± 3.8 data of more than 10,000 embryos and obtained a predic- No. of embryo cultured 7.9 ± 3.5 tive model with an AUC of 0.93. However, the predictive Huang et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 6 of 7 endpoint of this study is the clinical pregnancy, which systems. On the other hand, we directly used the live is the most prominent difference from our study. In this birth outcome as the deep learning model label. The study, we hope to get the best predictive effect, so we false positive data of aborted embryos can be excluded. chose to predict the blastocyst transfer based on the final There is another flaw in this study, that is, the samples live birth outcome. are all from blastocyst transfer, and there is no model Obviously, there is no single method that can solve all design for cleavage embryo transfer. In fact, we have the problems in the field of assisted reproduction, and tried deep learning for the evolution of the cleavage different methods have their own key research direc - stage, but the effect is not satisfactory. This may be one tions. The model we developed was very complex and has of the reasons why there is no model reported for pre- a high accuracy rate. That includes a large sample size, dicting the outcome of the cleavage stage embryo [27, and the sample database covers patients and clinical pro- 31, 32]. grams with various conditions. The results are repeatable In conclusion, this model has good predictive value and have high clinical guidance significance. However, for embryos selection by deep learning. It can help we have to admit that our data come from the embryo embryologists choose the best embryos for transfer, images obtained by the time-lapse photography system freezing and thaw, and can shorten the time for patients after fertilization, ranging from 105 h to 125 h, instead of from embryo transfer to becoming a parent. video data which lacks early embryo development data. If we generalize this model into the task of prediction from Supplementary Information 3-day embryos, more refined works need to be done. As The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s12884- 021- 04373-5. we all known, more spatiotemporal features can be cap- tured if we use the entire video as an input. But we find Additional file 1. A brief explanatory diagram of this research. the predictive power will not progress obviously if we Additional file 2. A brief description video of this research. use the whole video as input rather than the blastocyst frames, considering the parameters of a model are greatly restricted due to the capacity of machines when faced Acknowledgements The author would like to thank the nurses and doctors who helped to collect with video data. and organize the data. Thanks to Dr. Zhou Li for his suggestions on the experi- There is no clear evidence that AI applied to IVF can mental design of this article, and Dr. Xinling Ren, Dr. Li Wu, and Dr. Lixia Zhu for increase the cumulative success rate [28, 29]. Whether a their suggestions on writing this article. patient can finally give birth to a healthy baby is not only Authors’ contributions related to the embryo itself, but also to the patient’s own B.H. was responsible for experimental design, data analysis and manuscript health, age, reproductive history, clinical plan and many writing. S.Z. was responsible for carrying out the deep learning analysis. B.M. and Y.Y. were responsible for coordinating the study and assembling the time- other factors. Our deep learning model does not include lapse data. S.Z. and L.J. contributed to the analysis of the project. The author(s) these variables in the database, which is also the direc- read and approved the final manuscript. tion we need to work hard in the future. It is worth not- Funding ing that the live birth rate in this study showed a high This work was supported by the National Natural Science Foundation of China level (45.6%). As we all know, age and ovarian reserve are (81801531). very important factors that determine the clinical preg- Availability of data and materials nancy rate and live birth rate of IVF [30]. This higher live The datasets used and/or analysed during the current study are available from birth rate may be related to the younger population in the corresponding author on reasonable request. this study (average age is 30.4 years) and better ovarian reserve (average number of oocytes retrieved is 12.9). Declarations In 2019, an important paper was reported in AI- Ethics approval and consent to participate assisted embryo selection, the author retrospectively The study conformed to the Declaration of Helsinki for Medical Research analyzed time-lapse videos and clinical outcomes of involving Human Subjects. All patients signed written informed consent and 10,638 embryos from eight different IVF clinics [31]. underwent the routine clinical treatment performed in our center. No addi- tional intervention was performed. It was approved by the Ethical Committee The deep learning model they reported was able to pre - of Reproductive Medicine Center, Tongji Hospital, Tongji Medicine College, dict fetal heart pregnancy from time-lapse videos with Huazhong University of Science and Technology (No. S097). an AUC of 0.93. We think our research is different. This Consent for publication article is a single-center research. The advantage of Not applicable. this lies in the data analysis of large samples in a single center, which avoids the influence of different embryo Competing interests The authors report no financial or commercial conflicts of interest. operation procedures and different embryo culture Huang  et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 7 of 7 Received: 10 June 2021 Accepted: 29 December 2021 23. Xu L, Wei X, Yin Y, Wang W, Zhou M. Automatic classification of human embryo microscope images based on LBP feature. In: Chinese Confer- ence on Image & Graphics Technologies, vol. 2014; 2014. 24. Santos Filho E, Noble JA, Poli M, Griffiths T, Emerson G, Wells D. A method for semi-automatic grading of human blastocyst microscope images. Human Reprod (Oxford, England). 2012;27(9):2641–8. References 25. Dirvanauskas D, Maskeliunas R, Raudonis V, Damasevicius R. Embryo 1. Steptoe PC, Edwards RG. Birth after the reimplantation of a human development stage prediction algorithm for automated time lapse embryo. Lancet. 1978;2(8085):366. incubators. Comput Methods Prog Biomed. 2019;177:161–74. 2. Tiitinen A. Single embryo transfer: why and how to identify the embryo 26. Khosravi P, Kazemi E, Zhan Q, Malmsten JE, Toschi M, Zisimopoulos P, with the best developmental potential. Best Pract Res Clin Endocrinol et al. Deep learning enables robust assessment and selection of human Metab. 2019;33(1):77–88. blastocysts after in vitro fertilization. NPJ digital Med. 2019;2:21. 3. Pinborg A, Wennerholm UB, Romundstad LB, Loft A, Aittomaki K, 27. Chavez-Badiola A, Mendizabal-Ruiz G, Flores-Saiffe FA, Garcia-Sanchez R, Söderström-Anttila V, et al. Why do singletons conceived after assisted Drakeley AJ. Deep learning as a predictive tool for fetal heart pregnancy reproduction technology have adverse perinatal outcome? Systematic following time-lapse incubation and blastocyst transfer. Hum Reprod. review and meta-analysis. Hum Reprod Update. 2013;19(2):87–104. 2020;2:2. 4. Tiitinen A. Prevention of multiple pregnancies in infertility treatment. Best 28. Ahlstrom A, Park H, Bergh C, Selleskog U, Lundin K. Conventional mor- Pract Res Clin Obstetr Gynaecol. 2012;26(6):829–40. phology performs better than morphokinetics for prediction of live birth 5. Bergh C, Wennerholm UB. Obstetric outcome and long-term follow up after day 2 transfer. Reprod BioMed Online. 2016;33(1):61–70. of children conceived through assisted reproduction. Best Pract Res Clin 29. Goodman LR, Goldberg J, Falcone T, Austin C, Desai N. Does the addition Obstetr Gynaecol. 2012;26(6):841–52. of time-lapse morphokinetics in the selection of embryos for transfer 6. Ledger WL, Anumba D, Marlow N, Thomas CM, Wilson EC. The costs improve pregnancy rates? A randomized controlled trial. Fertil Sterility. to the NHS of multiple births after IVF treatment in the UK. BJOG. 2016;105(2):275–285.e210. 2006;113(1):21–5. 30. Aboulghar MM, El-Faissal Y, Kamel A, Mansour R, Serour G, Aboulghar 7. Aparicio B, Cruz M, Meseguer M. Is morphokinetic analysis the answer? M, et al. The effect of early administration of rectal progesterone in IVF/ Reprod BioMed Online. 2013;27(6):654–63. ICSI twin pregnancies on the preterm birth rate: a randomized trial. BMC 8. Gallego RD, Remohi J, Meseguer M. Time-lapse imaging: the state of the Pregnancy Childbirth. 2020;20(1):351. artdagger. Biol Reprod. 2019;101(6):1146–54. 31. Tran D, Cooke S, Illingworth PJ, Gardner DK. Deep learning as a predic- 9. Sutherland K, Leitch J, Lyall H, Woodward BJ. Time-lapse imaging of tive tool for fetal heart pregnancy following time-lapse incubation and inner cell mass splitting with monochorionic triamniotic triplets after blastocyst transfer. Hum Reprod. 2019;34(6):1011–8. elective single embryo transfer: a case report. Reprod BioMed Online. 32. Babayev E, Feinberg EC. Embryo through the lens: from time-lapse 2019;38(4):491–6. cinematography to artificial intelligence. Fertil Steril. 2020;113(2):342–3. 10. Ebner T, Sesli O, Kresic S, Enengl S, Stoiber B, Reiter E, et al. Time-lapse imaging of cytoplasmic strings at the blastocyst stage suggests their association with spontaneous blastocoel collapse. Reprod BioMed Publisher’s Note Online. 2020;40(2):191–9. Springer Nature remains neutral with regard to jurisdictional claims in pub- 11. Bormann CL, Thirumalaraju P, Kanakasabapathy MK, Kandula H, Souter lished maps and institutional affiliations. I, Dimitriadis I, et al. Consistency and objectivity of automated embryo assessments using deep neural networks. Fertil Steril. 2020;113(4):781–7 e781. 12. Rubio I, Kuhlmann R, Agerholm I, Kirk J, Herrero J, Escribá MJ, et al. Limited implantation success of direct-cleaved human zygotes: a time-lapse study. Fertil Steril. 2012;98(6):1458–63. 13. Meseguer M. Time-lapse: the remaining questions to be answered. Fertil Steril. 2016;105(2):295–6. 14. Kirkegaard K, Ahlström A, Ingerslev HJ, Hardarson T. Choosing the best embryo by time lapse versus standard morphology. Fertil Steril. 2015;103(2):323–32. 15. Fernandez EI, Ferreira AS, Cecilio MHM, Cheles DS, de Souza RCM, Nogueira MFG, et al. Artificial intelligence in the IVF laboratory: overview through the application of different types of algorithms for the classifica- tion of reproductive data. J Assist Reprod Genet. 2020;37:2359–76. 16. Curchoe CL, Bormann CL. Artificial intelligence and machine learning for human reproduction and embryology presented at ASRM and ESHRE 2018. J Assist Reprod Genet. 2019;36(4):591–600. 17. Bori L, Paya E, Alegre L, Viloria TA, Remohi JA, Naranjo V, et al. Novel and conventional embryo parameters as input data for artificial neural networks: an artificial intelligence model applied for prediction of the Re Read ady y to to submit y submit your our re researc search h ? Choose BMC and benefit fr ? Choose BMC and benefit from om: : implantation potential. Fertil Steril. 2020;114(6):1232–41. 18. Huang B, Qian K, Li Z, Yue J, Yang W, Zhu G, et al. Neonatal outcomes fast, convenient online submission after early rescue intracytoplasmic sperm injection: an analysis of a 5-year thorough peer review by experienced researchers in your field period. Fertil Steril. 2015;103(6):1432–7 e1431. 19. Zhu L, Xi Q, Zhang H, Li Y, Ai J, Jin L. Blastocyst culture and cryopreserva- rapid publication on acceptance tion to optimize clinical outcomes of warming cycles. Reprod BioMed support for research data, including large and complex data types Online. 2013;27(2):154–60. • gold Open Access which fosters wider collaboration and increased citations 20. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition; maximum visibility for your research: over 100M website views per year 2016. • 21. Vabalas A, Gowen E, Poliakoff E, Casson AJ. Machine learning algorithm validation with a limited sample size. PLoS One. 2019;14(11):e0224365. At BMC, research is always in progress. 22. Morales DA, Bengoetxea E, Larrañaga P. Selection of human Learn more biomedcentral.com/submissions embryos for transfer by Bayesian classifiers. Comput Biol Med. 2008;38(11–12):1177–86. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Pregnancy and Childbirth Springer Journals

Using deep learning to predict the outcome of live birth from more than 10,000 embryo data

Download PDF
7 pages

Loading next page...
 
/lp/springer-journals/using-deep-learning-to-predict-the-outcome-of-live-birth-from-more-FlMsy93Cbu

References (54)

Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2022
eISSN
1471-2393
DOI
10.1186/s12884-021-04373-5
Publisher site
See Article on Publisher Site

Abstract

Background: Recently, the combination of deep learning and time-lapse imaging provides an objective, standard and scientific solution for embryo selection. However, the reported studies were based on blastocyst formation or clinical pregnancy as the end point. To the best of our knowledge, there is no predictive model that uses the outcome of live birth as the predictive end point. Can a deep learning model predict the probability of live birth from time- lapse system? Methods: This study retrospectively analyzed the time-lapse data and live birth outcomes of embryos samples from January 2018 to November 2019. We used the SGD optimizer with an initial learning rate of 0.025 and cosine learn- ing rate reduction strategy. The network is randomly initialized and trained for 200 epochs from scratch. The model is quantitively evaluated over a hold-out test and a 5-fold cross-validation by the average area under the curve (AUC) of the receiver operating characteristic (ROC) curve. Results: The deep learning model was able to predict live birth outcomes from time-lapse images with an AUC of 0.968 in 5-fold stratified cross-validation. Conclusions: This research reported a deep learning model that predicts the live birth outcome of a single blasto- cyst transfer. This efficient model for predicting the outcome of live births can automatically analyze the time-lapse images of the patient’s embryos without the need for manual embryo annotation and evaluation, and then give a live birth prediction score for each embryo, and sort the embryos by the predicted value. Keywords: Time-lapse microscopy, Embryo development, Embryo quality, Pregnancy Introduction higher risk of complications [3–6]. Therefore, with the Since Louis Brown was born, the first test tube baby [1], development of assisted reproductive technology, single more than seven million babies have been born around embryo transfer has gradually become the first choice of the world attribute to assisted reproduction technology IVF. However, single embryo transfer still faces an urgent (ART) [2]. In the early stage of IVF technology develop- problem: how to choose the best embryo to transfer to ment, multiple embryo transfer was the main transfer maintain the ideal success rate [7]. The trend of choosing method. However, multiple pregnancy was often accom- single embryo transfer is closely related to the improve- panied by premature delivery, more expenditure and ment and progress of embryo selection technology. Therefore, embryo identification and selection technol - ogy are particularly important and significant. In order *Correspondence: [email protected]; [email protected] to solve this problem, scholars have developed several Reproductive Medicine Center, Tongji Hospital, Tongji Medicine College, methods for identifying and selecting the best embryos Huazhong University of Science and Technology, Wuhan, People’s for transfer, such as: blastocyst culture, time-lapse pho- Republic of China School of Computer Science and Technology, Harbin Institute tography imaging system and pre-transfer genetic testing of Technology, Weihai 264209, China [8–10]. © The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Huang et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 2 of 7 Embryologists evaluated and observed the embryos large sample of single blastocyst transfer to obtain an effi - used optical microscope, which was taken out from the cient predictive model. conventional incubator at a specific time point during the first 5 days of life before the time-lapse imaging system Materials and methods was applied to the clinic [11]. Because of this disadvan- Patients tage, many events in the embryonic development process This was a noninterventional, retrospective, single-center have been missed [12]. And the emergence of time- cohort study of patients undergoing routine practice. lapse photography technology had just made up for this In order to reflect the broad range of patients typically shortcoming. encountered in clinical practice, no inclusion/exclu- Embryologists use the time-lapse photography sys- sion criteria were applied on baseline characteristics. tem to observe and evaluate the embryo that in a stable The time-lapse embryo data used in our work are col - environment, rather than exposed in a variable condition lected from Reproductive Medicine Center of Tongji (such as changing gas composition, unstable humidity, Hospital, Huazhong University of Science and Technol- insecure temperature and movement conditions), and ogy, Wuhan, China. The whole dataset contains 33,738 can obtain a lot of information between embryo develop- embryo samples captured by Embryoscope Plus time- ment, time and embryo potential [13, 14]. lapse microscope system. The fertilization time of these Scholars have introduced the mathematical technol- embryos were from January 2018 to November 2019, and ogy of artificial intelligence into ART, in order to acquire we continuously pay return visits until January 2021 to more information from the pictures obtained by the TL confirm whether these IVF treatments lead to live birth system, which may trigger a revolution. AI is a term that outcomes. All patients signed written informed consent can be divided into many areas, such as: artificial neu - and underwent the routine clinical treatment performed ral network (ANN), fuzzy logic, genetic algorithm (GA), in our center. No additional intervention was performed. machine learning and deep learning [15, 16]. The emergence of time-lapse incubation makes it pos - sible to record the complete cycle of an embryo from a Ethical approval blastomere to a blastocyst, when all morphokinetic fea- The study conformed to the Declaration of Helsinki for tures centralized [17]. Meanwhile, owing to its abundant Medical Research involving Human Subjects. It was time-lapse data, time-lapse incubation emerges up many approved by the Ethical Committee of Reproductive new research ideas combined with deep learning tech- Medicine Center, Tongji Hospital, Tongji Medicine Col- nology which is known as a data driven method. Deep lege, Huazhong University of Science and Technology. learning can uncover numerous subtle features which may not be paid attention to manually but do help the corresponding classification or prediction. When fed Dataset with enough well labeled data, deep learning model have The classification of the outcome of each embryo was the ability to find an optimal representation of the given shown in Table  1. And the final indicator was live birth. dataset by continuously conducting back-propagation. The whole dataset contained 33,738 embryos with labels u Th s, we can explore the general pattern which lead to a of positive, negative, and pending, as shown in Fig.  1. specific mapping from data to our desired tasks. The pending embryos referred to the unthawed embryos The deep learning literature that has been reported on which could be exploited in our future work, but were embryo selection is a design study with blastocyst forma- excluded in the experiments of this paper. Meanwhile, tion or clinical pregnancy as the end point. To the best only the single blastocyst transfer embryos were col- of our knowledge, there is no research on deep learning lected, including fresh cycle and frozen-thaw cycle. models designed with the end of live birth outcome. In u Th s, the engaged dataset in this paper contained 15,434 this study, we want to analyze the data of single-center, embryos with positive and negative labels. Table 1 Classification of the outcome of each embryo involved Classification Outcome Positive Live birth after a complete pregnancy cycle Negative Fail to live birth or embryo discarded because of a failed or abnormal fertilization, grossly abnormal morphology or aneuploidy from preimplan- tation genetic testing Pending Embryo in storage and not yet used Huang  et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 3 of 7 Fig. 1 The outcomes of the embryos being studied Embryo culture and frozen‑embryo transfer (FET) Serum hCG was measured to diagnosis a pregnancy The methods used for sperm preparation, for IVF and 2 weeks after embryo transfer and then was tested serially embryo culture, have been described previously [18]. to monitor rising titers. A clinical pregnancy was defined Briefly, semen was collected in sterile containers by as the presence of a gestational sac with fetal heart activ- masturbation after 3–5 d of sexual abstinence and then ity observed on ultrasound examination 5 weeks after maintained at 37 °C for 30 min. After liquefaction, sam- oocyte retrieval [19]. The live birth outcome data were ples were analyzed for sperm concentration, motility obtained by telephone interview of the parents after and morphology according to the World Health Organ- delivery. ization criteria. The oocytes were incubated in G-IVF medium (Vitrolife) and fertilized 3 to 4 h after retrieval. Deep learning model Normal fertilization was defined as zygotes with two In this work, we designed an end-to-end deep learn- pronuclei (2PN) and fertilized oocytes were continu- ing model to predict live birth probability. We label our ously cultured in G1 medium for 2 more days. Then, embryo samples by 0 and 1 according to real live birth the embryos were transferred to G2 medium and con- outcomes, where 1 represents live birth whereas 0 rep- tinued to be cultured for 3 more days. The additional resents not. The designed supervised network regresses good-quality blastocysts were cryopreserved for sub- the discrete prediction value between 0 and 1 under the sequent frozen-embryo transfer (FET) cycles. For the guidance of ground truth labels. FET cycles, oral estradiol (Progynova, Bayer) was pro- The network structure consists of seven convolution vided, 2 mg/d from cycle day 1–4, 4 mg/d from day 5–8 modules and two fully connected layers. The first mod - and 6 mg/d from day 9–12. Transvaginal ultrasound ule contains three convolution blocks which represents scanning was performed to assess the endometrial a combination of a convolution layer, a batch normali- thickness and ovulation from day 13; the estradiol dos- zation layer and a following ReLU (Rectified Linear age was adjusted based on the endometrial thickness. Unit) as an activation function. As is widely known that Administration of 40 mg progesterone intramuscularly the residual block proposed in ResNet [20] is demon- was given when the endometrium reached a thickness strated effective in numerous classification tasks, the of 8 mm or maximum. Administration of 60–80 mg of subsequent six convolution modules who share the progesterone was provided for the following 5 days. same architecture are composed of three basic residual Blastocysts transfer was performed on day 6, after blocks and a convolution block. Feature maps are down 5 days of progesterone administration. sampled at the last convolution block of each module. Huang et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 4 of 7 Training strategies The whole network in this work can be described a Aimed at the extremely imbalance of the positive and ResNet like network, as shown in Table 2. but the num- negative samples, we implement the following measures ber of modules differs from that in benchmark struc - during the training term. In the cross-validation experi- ture. Also, the complexity of our model is much higher ment, we perform data augmentation after splitting the than the benchmark model, specifically reflected on the dataset according to Table  3. The specific method is as number of convolution kernels. follows: Firstly, we conduct abundant data augmenta- We utilize BCE-Loss (binary cross entropy loss) as tion measures, including affine transformations and ran - a loss function to guide the backpropagation during domly coarse dropout. Affine transformations refer to training term when the model constantly optimizes flip, translation, rotation, scaling, each operation occurs itself. Since the loss function calculate the distance randomly at a probability of 50 %. Coarse dropout means between output predictions and target labels, our pur- randomly drop some local pixels, the selected local pix- pose is to minimize the loss value. els are painted in solid black, we set the probability rang- ing from 2 to 5%. Secondly, we over sample the positive samples at a certain multiple, which equals to the ratio of positive and negative samples, i.e., sixteen in our experi- Table 2 Network structure of the proposed method. The basic ments. The original images captured by time-lapse incu - block is engaged from ResNet18 [20] bation are 800 pixels, which should be further resized to Layer Filter Size Output Size 224 for network training after data augmentation. We used the SGD optimizer with an initial learning Conv1_x 7 × 7, 64 224 × 224 3 × 3, 64 224 × 224 rate of 0.025 and cosine learning rate reduction strategy. 3 × 3, 128, stride 2 112 × 112 The network is randomly initialized and trained for 200 Conv2_x 112 × 112 3 × 3, 128 epochs from scratch. × 3 56 × 56 3 × 3, 128 3 × 3, 256, stride 2 Performance testing Conv3_x 56 × 56 3 × 3, 256 × 3 The model is quantitively evaluated over a 5-fold cross- 3 × 3, 256 28 × 28 validation by the average area under the curve (AUC) of 3 × 3, 512, stride 2 the receiver operating characteristic (ROC) curve. Conv4_x 28 × 28 3 × 3, 512 × 3 ROC curve connects all points described by true posi- 14 × 14 3 × 3, 512 tive rate and false positive rate under all possible thresh- 3 × 3, 1024, stride 2 olds, which is a boundary value between positive and Conv5_x 14 × 14 3 × 3, 1024 × 3 7 × 7 3 × 3, 1024 negative samples. Considering that true positive rate 3 × 3, 2048, stride 2 and false positive rate are in a trade-off relationship cor - Conv6_x 7 × 7 responding to thresholds, we can quantify the discrimi- 3 × 3, 2048 × 2 5 × 5 3 × 3, 2048 nating power by calculating the area under the curve, this 3 × 3, 2048 is so-called AUC. A binary classifier who has incompa - Conv7_x 5 × 5 3 × 3, 2048 rable discriminating power can possess an AUC value × 2 3 × 3 3 × 3, 2048 of 1, whereas the weakest who almost emerge the judge- 3 × 3, 2048 ment randomly possess an AUC value of 0.5, and a higher Fc1 Max pool 3 × 3 1 × 1 AUC value implies a better performance. AUC is more 2048-d fc Table 3 Result of the 5-fold cross-validation analysis Fold 1 Fold 2 Fold 3 Fold 4 Fold 5 AUC (n = 3812) (n = 3812) (n = 3811) (n = 3811) (n = 3811) 1 Test Train Train Train Train 0.970 2 Train Test Train Train Train 0.964 3 Train Train Test Train Train 0.968 4 Train Train Train Test Train 0.976 5 Train Train Train Train Test 0.960 Average 0.968 Average AUC, The mean area under the curve across 5 cross-validation steps Huang  et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 5 of 7 reasonable than accuracy especially in classification tasks with imbalance data. In order to comprehensively evaluate the performance of our model, we perform a hold-out test and a 5-fold cross-validation simultaneously [21]. In the hold-out test or so-called train-val-test approach, we randomly split the dataset in a ratio of 5:1:1 for training set, vali- dation set, and test set, respectively. In the latter evalu- ation method, we randomly divide our data into five parts with equal size, where the proportion of positive and negative samples in each separate is same. Then, five models should be trained. In each case, a specific subset is selected for validation while the remaining four subsets serve as a training set. Finally, we can figure out the mean AUC of the five folds to evaluate the performance on the whole dataset. Compared with hold-out test, cross- validation can eliminate the possible overestimating or underestimating caused by undesired sample division. Results Fig. 2 ROC curve for prediction of live birth. ROC, Receiver operating characteristic. AUC, area under the curve From January 2018 to November 2019, a total of 5913 cycles used the time-lapse culture system. Among them, some patients have not been transferred in fresh Discussion cycle, and their embryos have not yet been thawed. In This study is a preliminary study of deep learning with the end, 3382 fresh cycles and 3270 frozen-thaw trans- live birth data as the end point during the IVF cycle. Our fer (FET) cycles were included in the study and 33,738 results show that Timelapse images can be combined embryos samples were analyzed. Basic information of the with deep learning technology for clinical applications. patients included in this study was shown in Table 4. Morales et al. [22], Xu et al. [23] and Santos Filho et al. [24] used static images to assess embryo quality or select Roc the best embryos to be transferred in the absence of early Analysis of the ROC was shown in Fig.  2. The resulting embryo development data. These methods lack support AUC of this research to predict live birth on the testing of more comprehensive data. dataset was 0.968. Dirvanauskas et al. [25] used convolutional neural net- work (CNN) to predict the developmental stage of the 5‑fold cross‑validation embryo analyze by analyze embryo images obtained from Table 3 showed the results of 5-fold cross-validation. The the time-lapse photography system, with a success rate average value of AUC was 0.968. The AUC was reproduc - of 97.62%. However, this method does not have the abil- ible in individual train-validation runs. ity to predict pregnancy. Khosravi et al. developed a new framework (STORK) based on the inception of Google’s Hold‑out test model to predict the quality of embryos with an AUC as The AUC value of the conducted hold-out test was 0.957, high as 0.98. The study has a large sample size, complex which was evaluated on the test set. The result was com - model, and high accuracy, but it cannot be used to pre- parable with the 5-fold cross-validation. dict live births [26]. It is demonstrated that our model has a better performance when compared with existing Table 4 Basic information of the patients included in this study benchmark model, but it still deserves to be optimized Age (y) 30.4 ± 3.9 since it’s high complexity. Such a complex network Duration of infertility (y) 3.3 ± 2.3 requires considerable computing resources, so it depends Duration of stimulation (d) 10.4 ± 1.8 highly on hardware device. Basal FSH (IU/L) 7.4 ± 1.9 There is also a latest report that creates the predictive No. of oocytes retrieved 12.9 ± 4.2 model of blastocyst transfer [27]. The author analyzed the No. of mature oocytes 11.0 ± 3.8 data of more than 10,000 embryos and obtained a predic- No. of embryo cultured 7.9 ± 3.5 tive model with an AUC of 0.93. However, the predictive Huang et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 6 of 7 endpoint of this study is the clinical pregnancy, which systems. On the other hand, we directly used the live is the most prominent difference from our study. In this birth outcome as the deep learning model label. The study, we hope to get the best predictive effect, so we false positive data of aborted embryos can be excluded. chose to predict the blastocyst transfer based on the final There is another flaw in this study, that is, the samples live birth outcome. are all from blastocyst transfer, and there is no model Obviously, there is no single method that can solve all design for cleavage embryo transfer. In fact, we have the problems in the field of assisted reproduction, and tried deep learning for the evolution of the cleavage different methods have their own key research direc - stage, but the effect is not satisfactory. This may be one tions. The model we developed was very complex and has of the reasons why there is no model reported for pre- a high accuracy rate. That includes a large sample size, dicting the outcome of the cleavage stage embryo [27, and the sample database covers patients and clinical pro- 31, 32]. grams with various conditions. The results are repeatable In conclusion, this model has good predictive value and have high clinical guidance significance. However, for embryos selection by deep learning. It can help we have to admit that our data come from the embryo embryologists choose the best embryos for transfer, images obtained by the time-lapse photography system freezing and thaw, and can shorten the time for patients after fertilization, ranging from 105 h to 125 h, instead of from embryo transfer to becoming a parent. video data which lacks early embryo development data. If we generalize this model into the task of prediction from Supplementary Information 3-day embryos, more refined works need to be done. As The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s12884- 021- 04373-5. we all known, more spatiotemporal features can be cap- tured if we use the entire video as an input. But we find Additional file 1. A brief explanatory diagram of this research. the predictive power will not progress obviously if we Additional file 2. A brief description video of this research. use the whole video as input rather than the blastocyst frames, considering the parameters of a model are greatly restricted due to the capacity of machines when faced Acknowledgements The author would like to thank the nurses and doctors who helped to collect with video data. and organize the data. Thanks to Dr. Zhou Li for his suggestions on the experi- There is no clear evidence that AI applied to IVF can mental design of this article, and Dr. Xinling Ren, Dr. Li Wu, and Dr. Lixia Zhu for increase the cumulative success rate [28, 29]. Whether a their suggestions on writing this article. patient can finally give birth to a healthy baby is not only Authors’ contributions related to the embryo itself, but also to the patient’s own B.H. was responsible for experimental design, data analysis and manuscript health, age, reproductive history, clinical plan and many writing. S.Z. was responsible for carrying out the deep learning analysis. B.M. and Y.Y. were responsible for coordinating the study and assembling the time- other factors. Our deep learning model does not include lapse data. S.Z. and L.J. contributed to the analysis of the project. The author(s) these variables in the database, which is also the direc- read and approved the final manuscript. tion we need to work hard in the future. It is worth not- Funding ing that the live birth rate in this study showed a high This work was supported by the National Natural Science Foundation of China level (45.6%). As we all know, age and ovarian reserve are (81801531). very important factors that determine the clinical preg- Availability of data and materials nancy rate and live birth rate of IVF [30]. This higher live The datasets used and/or analysed during the current study are available from birth rate may be related to the younger population in the corresponding author on reasonable request. this study (average age is 30.4 years) and better ovarian reserve (average number of oocytes retrieved is 12.9). Declarations In 2019, an important paper was reported in AI- Ethics approval and consent to participate assisted embryo selection, the author retrospectively The study conformed to the Declaration of Helsinki for Medical Research analyzed time-lapse videos and clinical outcomes of involving Human Subjects. All patients signed written informed consent and 10,638 embryos from eight different IVF clinics [31]. underwent the routine clinical treatment performed in our center. No addi- tional intervention was performed. It was approved by the Ethical Committee The deep learning model they reported was able to pre - of Reproductive Medicine Center, Tongji Hospital, Tongji Medicine College, dict fetal heart pregnancy from time-lapse videos with Huazhong University of Science and Technology (No. S097). an AUC of 0.93. We think our research is different. This Consent for publication article is a single-center research. The advantage of Not applicable. this lies in the data analysis of large samples in a single center, which avoids the influence of different embryo Competing interests The authors report no financial or commercial conflicts of interest. operation procedures and different embryo culture Huang  et al. BMC Pregnancy and Childbirth (2022) 22:36 Page 7 of 7 Received: 10 June 2021 Accepted: 29 December 2021 23. Xu L, Wei X, Yin Y, Wang W, Zhou M. Automatic classification of human embryo microscope images based on LBP feature. In: Chinese Confer- ence on Image & Graphics Technologies, vol. 2014; 2014. 24. Santos Filho E, Noble JA, Poli M, Griffiths T, Emerson G, Wells D. A method for semi-automatic grading of human blastocyst microscope images. Human Reprod (Oxford, England). 2012;27(9):2641–8. References 25. Dirvanauskas D, Maskeliunas R, Raudonis V, Damasevicius R. Embryo 1. Steptoe PC, Edwards RG. Birth after the reimplantation of a human development stage prediction algorithm for automated time lapse embryo. Lancet. 1978;2(8085):366. incubators. Comput Methods Prog Biomed. 2019;177:161–74. 2. Tiitinen A. Single embryo transfer: why and how to identify the embryo 26. Khosravi P, Kazemi E, Zhan Q, Malmsten JE, Toschi M, Zisimopoulos P, with the best developmental potential. Best Pract Res Clin Endocrinol et al. Deep learning enables robust assessment and selection of human Metab. 2019;33(1):77–88. blastocysts after in vitro fertilization. NPJ digital Med. 2019;2:21. 3. Pinborg A, Wennerholm UB, Romundstad LB, Loft A, Aittomaki K, 27. Chavez-Badiola A, Mendizabal-Ruiz G, Flores-Saiffe FA, Garcia-Sanchez R, Söderström-Anttila V, et al. Why do singletons conceived after assisted Drakeley AJ. Deep learning as a predictive tool for fetal heart pregnancy reproduction technology have adverse perinatal outcome? Systematic following time-lapse incubation and blastocyst transfer. Hum Reprod. review and meta-analysis. Hum Reprod Update. 2013;19(2):87–104. 2020;2:2. 4. Tiitinen A. Prevention of multiple pregnancies in infertility treatment. Best 28. Ahlstrom A, Park H, Bergh C, Selleskog U, Lundin K. Conventional mor- Pract Res Clin Obstetr Gynaecol. 2012;26(6):829–40. phology performs better than morphokinetics for prediction of live birth 5. Bergh C, Wennerholm UB. Obstetric outcome and long-term follow up after day 2 transfer. Reprod BioMed Online. 2016;33(1):61–70. of children conceived through assisted reproduction. Best Pract Res Clin 29. Goodman LR, Goldberg J, Falcone T, Austin C, Desai N. Does the addition Obstetr Gynaecol. 2012;26(6):841–52. of time-lapse morphokinetics in the selection of embryos for transfer 6. Ledger WL, Anumba D, Marlow N, Thomas CM, Wilson EC. The costs improve pregnancy rates? A randomized controlled trial. Fertil Sterility. to the NHS of multiple births after IVF treatment in the UK. BJOG. 2016;105(2):275–285.e210. 2006;113(1):21–5. 30. Aboulghar MM, El-Faissal Y, Kamel A, Mansour R, Serour G, Aboulghar 7. Aparicio B, Cruz M, Meseguer M. Is morphokinetic analysis the answer? M, et al. The effect of early administration of rectal progesterone in IVF/ Reprod BioMed Online. 2013;27(6):654–63. ICSI twin pregnancies on the preterm birth rate: a randomized trial. BMC 8. Gallego RD, Remohi J, Meseguer M. Time-lapse imaging: the state of the Pregnancy Childbirth. 2020;20(1):351. artdagger. Biol Reprod. 2019;101(6):1146–54. 31. Tran D, Cooke S, Illingworth PJ, Gardner DK. Deep learning as a predic- 9. Sutherland K, Leitch J, Lyall H, Woodward BJ. Time-lapse imaging of tive tool for fetal heart pregnancy following time-lapse incubation and inner cell mass splitting with monochorionic triamniotic triplets after blastocyst transfer. Hum Reprod. 2019;34(6):1011–8. elective single embryo transfer: a case report. Reprod BioMed Online. 32. Babayev E, Feinberg EC. Embryo through the lens: from time-lapse 2019;38(4):491–6. cinematography to artificial intelligence. Fertil Steril. 2020;113(2):342–3. 10. Ebner T, Sesli O, Kresic S, Enengl S, Stoiber B, Reiter E, et al. Time-lapse imaging of cytoplasmic strings at the blastocyst stage suggests their association with spontaneous blastocoel collapse. Reprod BioMed Publisher’s Note Online. 2020;40(2):191–9. Springer Nature remains neutral with regard to jurisdictional claims in pub- 11. Bormann CL, Thirumalaraju P, Kanakasabapathy MK, Kandula H, Souter lished maps and institutional affiliations. I, Dimitriadis I, et al. Consistency and objectivity of automated embryo assessments using deep neural networks. Fertil Steril. 2020;113(4):781–7 e781. 12. Rubio I, Kuhlmann R, Agerholm I, Kirk J, Herrero J, Escribá MJ, et al. Limited implantation success of direct-cleaved human zygotes: a time-lapse study. Fertil Steril. 2012;98(6):1458–63. 13. Meseguer M. Time-lapse: the remaining questions to be answered. Fertil Steril. 2016;105(2):295–6. 14. Kirkegaard K, Ahlström A, Ingerslev HJ, Hardarson T. Choosing the best embryo by time lapse versus standard morphology. Fertil Steril. 2015;103(2):323–32. 15. Fernandez EI, Ferreira AS, Cecilio MHM, Cheles DS, de Souza RCM, Nogueira MFG, et al. Artificial intelligence in the IVF laboratory: overview through the application of different types of algorithms for the classifica- tion of reproductive data. J Assist Reprod Genet. 2020;37:2359–76. 16. Curchoe CL, Bormann CL. Artificial intelligence and machine learning for human reproduction and embryology presented at ASRM and ESHRE 2018. J Assist Reprod Genet. 2019;36(4):591–600. 17. Bori L, Paya E, Alegre L, Viloria TA, Remohi JA, Naranjo V, et al. Novel and conventional embryo parameters as input data for artificial neural networks: an artificial intelligence model applied for prediction of the Re Read ady y to to submit y submit your our re researc search h ? Choose BMC and benefit fr ? Choose BMC and benefit from om: : implantation potential. Fertil Steril. 2020;114(6):1232–41. 18. Huang B, Qian K, Li Z, Yue J, Yang W, Zhu G, et al. Neonatal outcomes fast, convenient online submission after early rescue intracytoplasmic sperm injection: an analysis of a 5-year thorough peer review by experienced researchers in your field period. Fertil Steril. 2015;103(6):1432–7 e1431. 19. Zhu L, Xi Q, Zhang H, Li Y, Ai J, Jin L. Blastocyst culture and cryopreserva- rapid publication on acceptance tion to optimize clinical outcomes of warming cycles. Reprod BioMed support for research data, including large and complex data types Online. 2013;27(2):154–60. • gold Open Access which fosters wider collaboration and increased citations 20. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition; maximum visibility for your research: over 100M website views per year 2016. • 21. Vabalas A, Gowen E, Poliakoff E, Casson AJ. Machine learning algorithm validation with a limited sample size. PLoS One. 2019;14(11):e0224365. At BMC, research is always in progress. 22. Morales DA, Bengoetxea E, Larrañaga P. Selection of human Learn more biomedcentral.com/submissions embryos for transfer by Bayesian classifiers. Comput Biol Med. 2008;38(11–12):1177–86.

Journal

BMC Pregnancy and Childbirth – Springer Journals

Published: Jan 16, 2022

Keywords: Time-lapse microscopy; Embryo development; Embryo quality; Pregnancy

There are no references for this article.