<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">[apologize for multiple posting]<div class=""><br class=""></div><div class=""><div class=""><b class="">ScaDL 2022: </b></div><div class=""><b class="">Scalable Deep Learning over Parallel And Distributed Infrastructure - An IPDPS 2022 Workshop</b></div><div class=""><br class=""></div><div class=""><a href="https://2022.scadl.org" class="">https://2022.scadl.org</a> </div><div class=""><br class=""></div><div class=""><b class="">Scope of the Workshop</b></div><div class=""><br class=""></div><div class="">Recently, Deep Learning (DL) has received tremendous attention in the research community because of the impressive results obtained for a large number of machine learning problems. The success of state-of-the-art deep learning systems relies on training deep neural networks over a massive amount of training data, which typically requires a large-scale distributed computing infrastructure to run. In order to run these jobs in a scalable and efficient manner, on cloud infrastructure or dedicated HPC systems, several interesting research topics have emerged which are specific to DL. The sheer size and complexity of deep learning models when trained over a large amount of data makes them harder to converge in a reasonable amount of time. It demands advancement along multiple research directions such as, model/data parallelism, model/data compression, distributed optimization algorithms for DL convergence, synchronization strategies, efficient communication and specific hardware acceleration.</div><div class=""><br class=""></div><div class=""><b class="">SCADL seeks to advance the following research directions:</b></div><div class="">- Asynchronous and Communication-Efficient SGD: Stochastic gradient descent is at the core of large-scale machine learning. Parallelizing SGD gradient computation across multiple nodes increases the data processed per iteration, but exposes the SGD to communication and synchronization delays and unpredictable node failures in the system. Thus, there is a critical need to design robust and scalable distributed SGD methods to achieve fast error-convergence in spite of such system variabilities.</div><div class="">- High performance computing aspects: Deep learning is highly compute intensive. Algorithms for kernel computations on commonly used accelerators (e.g. GPUs), efficient techniques for communicating gradients and loading data from storage are critical for training performance.</div><div class="">- Model and Gradient Compression Techniques: Techniques such as reducing weights and the size of weight tensors help in reducing the compute complexity. Using lower-bit representations such as quantization and sparsification allow for more optimal use of memory and communication bandwidth.</div><div class="">- Distributed Trustworthy AI: New techniques are needed to meet the goal of global trustworthiness (e.g., fairness and adversarial robustness) efficiently in a distributed DL setting.</div><div class="">- Emerging AI hardware Accelerators: with the proliferation of new hardware accelerators for AI such in memory computing (Analog AI) and neuromorphic computing, novel methods and algorithms need to be introduced to adapt to the underlying properties of the new hardware (example: the non-idealities of the phase-change memory (PCM) and the cycle-to-cycle statistical variations). </div><div class="">- The intersection of Distributed DL and Neural Architecture Search (NAS): NAS is increasingly being used to automate the synthesis of neural networks. However, given the huge computational demands of NAS, distributed DL is critical to make NAS computationally tractable (e.g., differentiable distributed NAS).</div><div class="">This intersection of distributed/parallel computing and deep learning is becoming critical and demands specific attention to address the above topics which some of the broader forums may not be able to provide. The aim of this workshop is to foster collaboration among researchers from distributed/parallel computing and deep learning communities to share the relevant topics as well as results of the current approaches lying at the intersection of these areas.</div><div class=""><br class=""></div><div class=""><b class="">Areas of Interest</b></div><div class="">In this workshop, we solicit research papers focused on distributed deep learning aiming to achieve efficiency and scalability for deep learning jobs over distributed and parallel systems. Papers focusing both on algorithms as well as systems are welcome. We invite authors to submit papers on topics including but not limited to:</div><div class="">- Deep learning on cloud platforms, HPC systems, and edge devices</div><div class="">- Model-parallel and data-parallel techniques</div><div class="">- Asynchronous SGD for Training DNNs</div><div class="">- Communication-Efficient Training of DNNs</div><div class="">- Scalable and distributed graph neural networks, Sampling techniques for graph neural networks</div><div class="">- Federated deep learning, both horizontal and vertical, and its challenges</div><div class="">- Model/data/gradient compression</div><div class="">- Learning in Resource constrained environments</div><div class="">- Coding Techniques for Straggler Mitigation</div><div class="">- Elasticity for deep learning jobs/spot market enablement</div><div class="">- Hyper-parameter tuning for deep learning jobs</div><div class="">- Hardware Acceleration for Deep Learning including digital and analog accelerators</div><div class="">- Scalability of deep learning jobs on large clusters</div><div class="">- Deep learning on heterogeneous infrastructure</div><div class="">- Efficient and Scalable Inference</div><div class="">- Data storage/access in shared networks for deep learning</div><div class="">- Communication-efficient distributed fair and adversarially robust learning</div><div class="">- Distributed learning techniques applied to speed up neural architecture search</div><div class=""><br class=""></div><div class=""><b class="">Workshop Format</b></div><div class="">Due to the continuing impact of COVID-19, ScaDL 2022 will also adopt relevant IPDPS 2022 policies on virtual participation and presentation. Consequently, the organizers are currently planning a hybrid (in-person and virtual) event.</div><div class=""><br class=""></div><div class=""><b class="">Submission Link</b></div><div class="">Submissions will be managed through linklings. Submission link available at: <a href="https://2022.scadl.org/call-for-papers" class="">https://2022.scadl.org/call-for-papers</a> </div><div class=""><br class=""></div><div class=""><b class="">Key Dates</b></div><div class="">- Paper Submission: January 24, 2022</div><div class="">- Acceptance Notification: March 1, 2022</div><div class="">- Camera ready papers due: March 15, 2022 (hard deadline)</div><div class="">- Workshop Date: TBA (May 30th or June 3rd, 2022)</div><div class=""><br class=""></div><div class=""><b class="">Author Instructions</b></div><div class="">ScaDL 2022 accepts submissions in two categories:</div><div class="">Regular papers: 8-10 pages</div><div class="">Short papers/Work in progress: 4 pages</div><div class="">The aforementioned lengths include all technical content, references and appendices.</div><div class="">We encourage submissions that are original research work, work in progress, case studies, vision papers, and industrial experience papers.</div><div class="">Papers should be formatted using IEEE conference style, including figures, tables, and references. The IEEE conference style templates for MS Word and LaTeX provided by IEEE eXpress Conference Publishing are available for download. See the latest versions at <a href="https://www.ieee.org/conferences/publishing/templates.html" class="">https://www.ieee.org/conferences/publishing/templates.html</a> </div><div class=""><br class=""></div><div class=""><b class="">General Chairs</b></div><div class="">Danilo Ardagna, Politecnico di Milano, Italy</div><div class="">Stacy Patterson, Rensselaer Polytechnic Institute (RPI), USA</div><div class=""><br class=""></div><div class=""><b class="">Program Committee Chairs</b></div><div class="">Alex Gittens, Rensselaer Polytechnic Institute (RPI), USA</div><div class="">Kaoutar El Maghraoui, IBM Research AI, USA</div><div class=""><br class=""></div><div class=""><b class="">Steering Committee</b></div><div class="">Parijat Dube, IBM Research AI, USA</div><div class="">Vinod Muthusamy, IBM Research AI</div><div class="">Ashish Verma, IBM Research AI</div><div class="">Jayaram K. R., IBM Research AI, USA</div><div class="">Yogish Sabharwal, IBM Research AI, India</div></div>
<br class=""></body></html>