TY - JOUR AU - Yu, Yaoliang AB - Petuum: A New Platform for Distributed Machine Learning on Big Data Eric P. Xing, *Qirong Ho, Wei Dai, Jin Kyu Kim, Jinliang Wei, Seunghak Lee, Xun Zheng, Pengtao Xie, Abhimanu Kumar and Yaoliang Yu School of Computer Science, Carnegie Mellon University; *Institute for Infocomm Research, A*STAR Pittsburgh, Pennsylvania, USA; *Singapore {epxing,wdai,jinkyuk,jinlianw,seunghak,xunzheng,pengtaox,yaoliang}@cs.cmu.edu {hoqirong,abhimanyu.kumar}@gmail.com ABSTRACT How can one build a distributed framework that allows efficient deployment of a wide spectrum of modern advanced machine learning (ML) programs for industrial-scale problems using Big Models (100s of billions of parameters) on Big Data (terabytes or petabytes)? Contemporary parallelization strategies employ fine-grained operations and scheduling beyond the classic bulk-synchronous processing paradigm popularized by MapReduce, or even specialized operators relying on graphical representations of ML programs. The variety of approaches tends to pull systems and algorithms design in different directions, and it remains difficult to find a universal platform applicable to a wide range of different ML programs at scale. We propose a general-purpose framework that systematically addresses data- and model-parallel challenges in large-scale ML, by leveraging several fundamental properties underlying ML programs that make them different from conventional operation-centric programs: error tolerance, dynamic structure, and nonuniform convergence; all stem from the optimization-centric nature TI - Petuum: A New Platform for Distributed Machine Learning on Big Data DA - 2015-08-10 UR - https://www.deepdyve.com/lp/association-for-computing-machinery/petuum-a-new-platform-for-distributed-machine-learning-on-big-data-1ux9Xbslbk DP - DeepDyve ER -