Description
Jupyter Notebooks have become the "de facto" platform used by scientists and engineers to build Python interactive applications to tackle scientific and machine learning problems. However, with the popularity of big data analytics and complex deep learning workloads, there is a growing requirement to extend the computation across a cluster of computers in a parallel fashion. In this talk, we will describe how to use multiple Jupyter Notebook components to enable the orchestration and distribution of interactive machine learning and deep learning workloads across different types of computing clusters including Apache Spark and Kubernetes. This talk is intended to attendees interested in distributed platforms and scientists experiencing difficulties on scaling their scientific workloads across multiple machines. Bio: Luciano Resende is an STSM and Open Source Data Science/AI Platform Architect at IBM CODAIT (formerly Spark Technology Center). He has been contributing to open source at The ASF for over 10 years, he is a member of ASF and is currently contributing to various big data related Apache projects around the Apache Spark ecosystem. Currently, Luciano is contributing to Jupyter Ecosystem projects building scalable, secure and flexible Enterprise Data Science platform.