<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div>Dear colleague,</div>
<div><br>
</div>
<div>apologize for multiple posting</div>
<div><br>
</div>
<div><b>ScaDL 2023: Scalable Deep Learning over Parallel And Distributed</b></div>
<div><b>Infrastructure - An IPDPS 2023 Workshop</b></div>
<div><br>
</div>
<div><a href="https://2023.scadl.org/">https://2023.scadl.org</a> </div>
<div><br>
</div>
<div><b>Scope of the Workshop:</b></div>
<div>Recently, Deep Learning (DL) has received tremendous attention in the research</div>
<div>community because of the impressive results obtained for a large number of</div>
<div>machine learning problems. The success of state-of-the-art deep learning</div>
<div>systems relies on training deep neural networks over a massive amount of</div>
<div>training data, which typically requires a large-scale distributed computing</div>
<div>infrastructure to run. In order to run these jobs in a scalable and efficient</div>
<div>manner, on cloud infrastructure or dedicated HPC systems, several interesting</div>
<div>research topics have emerged which are specific to DL. The sheer size and</div>
<div>complexity of deep learning models when trained over a large amount of data</div>
<div>makes them harder to converge in a reasonable amount of time. It demands</div>
<div>advancement along multiple research directions such as, model/data</div>
<div>parallelism, model/data compression, distributed optimization algorithms for</div>
<div>DL convergence, synchronization strategies, efficient communication and</div>
<div>specific hardware acceleration.</div>
<div><br>
</div>
<div><b>SCADL seeks to advance the following research directions:</b></div>
<div>- Asynchronous and Communication-Efficient SGD: Stochastic gradient descent is</div>
<div>at the core of large-scale machine learning. Parallelizing SGD gradient</div>
<div>computation across multiple nodes increases the data processed per iteration,</div>
<div>but exposes the SGD to communication and synchronization delays and</div>
<div>unpredictable node failures in the system. Thus, there is a critical need to</div>
<div>design robust and scalable distributed SGD methods to achieve fast error-</div>
<div>convergence in spite of such system variabilities.</div>
<div>High performance computing aspects: Deep learning is highly compute intensive.</div>
<div>Algorithms for kernel computations on commonly used accelerators (e.g. GPUs),</div>
<div>efficient techniques for communicating gradients and loading data from storage</div>
<div>are critical for training performance.</div>
<div><br>
</div>
<div>- Model and Gradient Compression Techniques: Techniques such as reducing</div>
<div>weights and the size of weight tensors help in reducing the compute</div>
<div>complexity. Using lower-bit representations such as quantization and</div>
<div>sparsification allow for more optimal use of memory and communication</div>
<div>bandwidth.</div>
<div><br>
</div>
<div>- Distributed Trustworthy AI: New techniques are needed to meet the goal of</div>
<div>global trustworthiness (e.g., fairness and adversarial robustness) efficiently</div>
<div>in a distributed DL setting.</div>
<div><br>
</div>
<div>- Emerging AI hardware Accelerators: with the proliferation of new hardware</div>
<div>accelerators for AI such in memory computing (Analog AI) and neuromorphic</div>
<div>computing, novel methods and algorithms need to be introduced to adapt to the</div>
<div>underlying properties of the new hardware (example: the non-idealities of the</div>
<div>phase-change memory (PCM) and the cycle-to-cycle statistical variations).</div>
<div><br>
</div>
<div>- The intersection of Distributed DL and Neural Architecture Search (NAS): NAS</div>
<div>is increasingly being used to automate the synthesis of neural networks.</div>
<div>However, given the huge computational demands of NAS, distributed DL is</div>
<div>critical to make NAS computationally tractable (e.g., differentiable</div>
<div>distributed NAS).</div>
<div><br>
</div>
<div>This intersection of distributed/parallel computing and deep learning is</div>
<div>becoming critical and demands specific attention to address the above topics</div>
<div>which some of the broader forums may not be able to provide. The aim of this</div>
<div>workshop is to foster collaboration among researchers from distributed/</div>
<div>parallel computing and deep learning communities to share the relevant topics</div>
<div>as well as results of the current approaches lying at the intersection of</div>
<div>these areas.</div>
<div><br>
</div>
<div><b>Areas of Interest</b></div>
<div>In this workshop, we solicit research papers focused on distributed deep</div>
<div>learning aiming to achieve efficiency and scalability for deep learning jobs</div>
<div>over distributed and parallel systems. Papers focusing both on algorithms as</div>
<div>well as systems are welcome. We invite authors to submit papers on topics</div>
<div>including but not limited to:</div>
<div><br>
</div>
<div>- Deep learning on cloud platforms, HPC systems, and edge devices</div>
<div>- Model-parallel and data-parallel techniques</div>
<div>- Asynchronous SGD for Training DNNs</div>
<div>- Communication-Efficient Training of DNNs</div>
<div>- Scalable and distributed graph neural networks, Sampling techniques for</div>
<div>graph neural networks</div>
<div>- Federated deep learning, both horizontal and vertical, and its challenges</div>
<div>- Model/data/gradient compression</div>
<div>- Learning in Resource constrained environments</div>
<div>- Coding Techniques for Straggler Mitigation</div>
<div>- Elasticity for deep learning jobs/spot market enablement</div>
<div>- Hyper-parameter tuning for deep learning jobs</div>
<div>- Hardware Acceleration for Deep Learning including digital and analog</div>
<div>accelerators</div>
<div>- Scalability of deep learning jobs on large clusters</div>
<div>- Deep learning on heterogeneous infrastructure</div>
<div>- Efficient and Scalable Inference</div>
<div>- Data storage/access in shared networks for deep learning</div>
<div>- Communication-efficient distributed fair and adversarially robust learning</div>
<div>- Distributed learning techniques applied to speed up neural architecture</div>
<div>search</div>
<div><br>
</div>
<div><br>
</div>
<div><b>Workshop Format:</b></div>
<div>Due to the continuing impact of COVID-19, ScaDL 2023 will also adopt relevant</div>
<div>IPDPS 2023 policies on virtual participation and presentation. Consequently,</div>
<div>the organizers are currently planning a hybrid (in-person and virtual) event.</div>
<div><br>
</div>
<div><b>Submission Link:</b></div>
<div>Submissions will be managed through linklings. Submission link available at:</div>
<div><a href="https://2023.scadl.org/call-for-papers">https://2023.scadl.org/call-for-papers</a> </div>
<div><br>
</div>
<div><b>Key Dates</b></div>
<div><b>Paper Submission: January 29th, 2023</b></div>
<div><b>Acceptance Notification: February 26th, 2023</b></div>
<div><b>Camera ready papers due: March 7th, 2023</b></div>
<div><b>Workshop Date: May 19th, 2023 </b></div>
<div><br>
</div>
<div><b>Author Instructions</b></div>
<div>ScaDL 2023 accepts submissions in two categories:</div>
<div>- Regular papers: 8-10 pages</div>
<div>- Short papers/Work in progress: 4 pages</div>
<div>The aforementioned lengths include all technical content, references and</div>
<div>appendices.</div>
<div>We encourage submissions that are original research work, work in progress,</div>
<div>case studies, vision papers, and industrial experience papers.</div>
<div>Papers should be formatted using IEEE conference style, including figures,</div>
<div>tables, and references. The IEEE conference style templates for MS Word and</div>
<div>LaTeX provided by IEEE eXpress Conference Publishing are available for</div>
<div>download. See the latest versions at</div>
<div><a href="https://www.ieee.org/conferences/publishing/templates.html">https://www.ieee.org/conferences/publishing/templates.html</a> </div>
<div><br>
</div>
<div><b>General Chairs</b></div>
<div>Kaoutar El Maghraoui, IBM Research AI, USA</div>
<div>Daniele Lezzi, Barcelona Supercomputing Center, Spain</div>
<div><br>
</div>
<div><b>Program Committee Chairs</b></div>
<div>Misbah Mubarak, NVIDIA, USA</div>
<div>Alex Gittens, Rensselaer Polytechnic Institute (RPI), USA</div>
<div><br>
</div>
<div><b>Publicity Chairs</b></div>
<div>Federica Filippini, Politecnico di Milano, Italy</div>
<div>Hadjer Benmeziane, Université Polytechnique des Hauts-de-France</div>
<div><br>
</div>
<div><b>Web Chair </b></div>
<div>Praveen Venkateswaran, IBM Research AI, USA</div>
<div><br>
</div>
<div><b>Steering Committee</b></div>
<div>Parijat Dube, IBM Research AI, USA</div>
<div>Vinod Muthusamy, IBM Research AI, USA</div>
<div>Ashish Verma, IBM Research AI, USA</div>
<div>Jayaram K. R., IBM Research AI, USA</div>
<div>Yogish Sabharwal, IBM Research AI, India</div>
<div>Danilo Ardagna, Politecnico di Milano, Italy</div>
<br>
</body>
</html>