BEGIN:VCALENDAR VERSION:2.0 PRODID:researchseminars.org CALSCALE:GREGORIAN X-WR-CALNAME:researchseminars.org BEGIN:VEVENT SUMMARY:Nathan Kutz (University of Washington) DTSTART:20200701T150000Z DTEND:20200701T151000Z DTSTAMP:20260404T094752Z UID:SciDL/1 DESCRIPTION:Title: Opening remarks\nby Nathan Kutz (University of Washington) as par t of Workshop on Scientific-Driven Deep Learning (SciDL)\n\nAbstract: TBA\ n LOCATION:https://stable.researchseminars.org/talk/SciDL/1/ END:VEVENT BEGIN:VEVENT SUMMARY:George Em Karniadakis (Brown University) DTSTART:20200701T151000Z DTEND:20200701T160000Z DTSTAMP:20260404T094752Z UID:SciDL/2 DESCRIPTION:Title: DeepOnet: Learning nonlinear operators based on the universal approxi mation theorem of operators\nby George Em Karniadakis (Brown Universit y) as part of Workshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbs tract\nIt is widely known that neural networks (NNs) are universal approxi mators of continuous functions\, however\, a less known but powerful resul t is that a NN with a single hidden layer can approximate accurately any n onlinear continuous operator. This universal approximation theorem of oper ators is suggestive of the potential of NNs in learning from scattered dat a any continuous operator or complex system. To realize this theorem\, we design a new NN with small generalization error\, the deep operator networ k (DeepONet)\, consisting of a NN for encoding the discrete input function space (branch net) and another NN for encoding the domain of the output f unctions (trunk net). We demonstrate that DeepONet can learn various expli cit operators\, e.g.\, integrals and fractional Laplacians\, as well as im plicit operators that represent deterministic and stochastic differential equations. We study\, in particular\, different formulations of the input function space and its effect on the generalization error.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/2/ END:VEVENT BEGIN:VEVENT SUMMARY:Frank Noe (FU Berlin) DTSTART:20200701T160000Z DTEND:20200701T162500Z DTSTAMP:20260404T094752Z UID:SciDL/3 DESCRIPTION:Title: PauliNet: Deep neural network solution of the electronic Schrödinger Equation\nby Frank Noe (FU Berlin) as part of Workshop on Scientific- Driven Deep Learning (SciDL)\n\n\nAbstract\nThe electronic Schrödinger eq uation describes fundamental properties of molecules and materials\, but c an only be solved analytically for the hydrogen atom. The numerically exac t full configuration-interaction method is exponentially expensive in the number of electrons. Quantum Monte Carlo is a possible way out: it scales well to large molecules\, can be parallelized\, and its accuracy has\, as yet\, only been limited by the flexibility of the used wave function ansat z. Here we propose PauliNet\, a deep-learning wave function ansatz that ac hieves nearly exact solutions of the electronic Schrödinger equation. Pau liNet has a multireference Hartree-Fock solution built in as a baseline\, incorporates the physics of valid wave functions\, and is trained using va riational quantum Monte Carlo (VMC). PauliNet outperforms comparable state -of-the-art VMC ansatzes for atoms\, diatomic molecules and a strongly-cor related hydrogen chain by a margin and is yet computationally efficient. W e anticipate that thanks to the favourable scaling with system size\, this method may become a new leading method for highly accurate electronic-str ucutre calculations on medium-sized molecular systems.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/3/ END:VEVENT BEGIN:VEVENT SUMMARY:Alejandro Queiruga (Google\, LLC) DTSTART:20200701T162500Z DTEND:20200701T165000Z DTSTAMP:20260404T094752Z UID:SciDL/4 DESCRIPTION:Title: Continuous-in-Depth Neural Networks through Interpretation of Learned Dynamics\nby Alejandro Queiruga (Google\, LLC) as part of Workshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nData-driven learni ng of dynamical systems is of interest to the scientific community\, which wants to recover information about the true physics from the discretized model\, and the machine learning community\, which wants to improve model interpretability and performance. We present a refined interpretation of l earned dynamical models by investigating canonical systems. Recent ML lite rature draws a metaphor between residual components of neural networks and a forward Euler time integrator\, but we show that these components actua lly learn a more accurate integrator. We examine\, the harmonic oscillator \, 1D wave equation\, and the pendulum in two forms\, using purely linear models\, feed-forward shallow neural networks\, and neural networks embedd ed in time integrators. Each of the model configurations overfit to a bett er operator than commonly understood\, confounding recovery of physics and attempts to improve the algorithms. We show two analytical methods for re constructing underlying operators from linear systems. For the nonlinear p roblems\, unmodified neural networks outperform the expected numerical met hods\, but do not allow for inspection or generalization. Embedding the mo dels in integrators such as RK4 improves performance and generalizability. However\, for the constrained pendulum\, the model is still better than e xcepted\, exhibiting better than expected stiffness-stability. We conclude by revisiting the components of neural networks where improvements are su ggested.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/4/ END:VEVENT BEGIN:VEVENT SUMMARY:Michael Muehlebach (UC Berkeley) DTSTART:20200701T165000Z DTEND:20200701T171500Z DTSTAMP:20260404T094752Z UID:SciDL/5 DESCRIPTION:Title: Optimization with Momentum: Dynamical\, Control-Theoretic\, and Sympl ectic Perspectives\nby Michael Muehlebach (UC Berkeley) as part of Wor kshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nMy talk wi ll focus on the analysis of accelerated first-order optimization algorithm s. I will show how the continuous dependence of the iterates with respect to their initial condition can be exploited to characterize the convergenc e rate. The result establishes criteria for accelerated convergence that a re easily verifiable and applicable to a large class of first-order optimi zation algorithms. The analysis is not restricted to the convex setting an d unifies discrete-time and continuous-time models. It also rigorously exp lains why structure-preserving discretization schemes are important for mo mentum-based algorithms.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/5/ END:VEVENT BEGIN:VEVENT SUMMARY:Tess Smidt (LBL) DTSTART:20200701T204000Z DTEND:20200701T210500Z DTSTAMP:20260404T094752Z UID:SciDL/6 DESCRIPTION:Title: Euclidean Neural Networks for Emulating Ab Initio Calculations and Ge nerating Atomic Geometries\nby Tess Smidt (LBL) as part of Workshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nAtomic systems (mo lecules\, crystals\, proteins\, nanoclusters\, etc.) are naturally represe nted by a set of coordinates in 3D space labeled by atom type. This is a c hallenging representation to use for neural networks because the coordinat es are sensitive to 3D rotations and translations and there is no canonica l orientation or position for these systems. We present a general neural n etwork architecture that naturally handles 3D geometry and operates on the scalar\, vector\, and tensor fields that characterize physical systems. O ur networks are locally equivariant to 3D rotations and translations at ev ery layer. In this talk\, we describe how the network achieves these equiv ariances and demonstrate the capabilities of our network using simple task s. We’ll also present examples of applying Euclidean networks to applica tions in quantum chemistry and discuss techniques for using these networks to encode and decode geometry.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/6/ END:VEVENT BEGIN:VEVENT SUMMARY:Michael P. Brenner (Harvard University) DTSTART:20200701T190000Z DTEND:20200701T195000Z DTSTAMP:20260404T094752Z UID:SciDL/7 DESCRIPTION:Title: Machine Learning for Partial Differential Equations\nby Michael P . Brenner (Harvard University) as part of Workshop on Scientific-Driven De ep Learning (SciDL)\n\n\nAbstract\nI will discuss several ways in which ma chine learning can be used for solving and understanding the solutions of nonlinear partial differential equations. Most of the talk will focus on l earning discretizations for coarse graining the numerical solutions of PDE s. I will start with examples in 1d\, and then move on to advection/diffus ion in a turbulent flow and then the Navier Stokes equation.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/7/ END:VEVENT BEGIN:VEVENT SUMMARY:Elizabeth Qian (MIT) DTSTART:20200701T195000Z DTEND:20200701T201500Z DTSTAMP:20260404T094752Z UID:SciDL/8 DESCRIPTION:Title: Lift & Learn: Analyzable\, Generalizable Data-Driven Models for Nonli near PDEs\nby Elizabeth Qian (MIT) as part of Workshop on Scientific-D riven Deep Learning (SciDL)\n\n\nAbstract\nWe present Lift & Learn\, a phy sics-informed method for learning low-dimensional models for nonlinear PDE s. The method exploits knowledge of a system’s governing equations to id entify a coordinate transformation in which the system dynamics have quadr atic structure. This transformation is called a lifting map because it oft en adds auxiliary variables to the system state. The lifting map is applie d to data obtained by evaluating a model for the original nonlinear system . This lifted data is projected onto its leading principal components\, an d low-dimensional linear and quadratic matrix operators are fit to the lif ted reduced data using a least-squares operator inference procedure. Analy sis of our method shows that the Lift & Learn models are able to capture t he system physics in the lifted coordinates at least as accurately as trad itional intrusive model reduction approaches. This preservation of system physics makes the Lift & Learn models robust to changes in inputs. Numeric al experiments on the FitzHugh-Nagumo neuron activation model and the comp ressible Euler equations demonstrate the generalizability of our model.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/8/ END:VEVENT BEGIN:VEVENT SUMMARY:Lars Ruthotto (Emory University) DTSTART:20200701T201500Z DTEND:20200701T204000Z DTSTAMP:20260404T094752Z UID:SciDL/9 DESCRIPTION:Title: Deep Neural Networks Motivated by PDEs\nby Lars Ruthotto (Emory U niversity) as part of Workshop on Scientific-Driven Deep Learning (SciDL)\ n\n\nAbstract\nOne of the most promising areas in artificial intelligence is deep learning\, a form of machine learning that uses neural networks co ntaining many hidden layers. Recent success has led to breakthroughs in ap plications such as speech and image recognition. However\, more theoretica l insight is needed to create a rigorous scientific basis for designing an d training deep neural networks\, increasing their scalability\, and provi ding insight into their reasoning. This talk bridges the gap between parti al differential equations (PDEs) and neural networks and presents a new ma thematical paradigm that simplifies designing\, training\, and analyzing d eep neural networks. It shows that training deep neural networks can be ca st as a dynamic optimal control problem similar to path-planning and optim al mass transport. The talk outlines how this interpretation can improve t he effectiveness of deep neural networks. First\, the talk introduces new types of neural networks inspired by to parabolic\, hyperbolic\, and react ion-diffusion PDEs. Second\, the talk outlines how to accelerate training by exploiting reversibility properties of the underlying PDEs.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/9/ END:VEVENT BEGIN:VEVENT SUMMARY:Yasaman Bahri (Google Brain) DTSTART:20200701T171500Z DTEND:20200701T174000Z DTSTAMP:20260404T094752Z UID:SciDL/10 DESCRIPTION:Title: Learning Dynamics of Wide\, Deep Neural Networks: Beyond the Limit o f Infinite Width\nby Yasaman Bahri (Google Brain) as part of Workshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nWhile many pract ical advancements in deep learning have been made in recent years\, a scie ntific\, and ideally theoretical\, understanding of modern neural networks is still in its infancy. At the heart of this would be to better understa nd the learning dynamics of such systems. In a first step towards tackling this problem\, one can try to identify limits that have theoretical tract ability and are potentially practically relevant. I’ll begin by surveyin g our body of work that has investigated the infinite width limit of deep networks. These results establish exact mappings between deep networks and other\, existing machine learning methods (namely\, Gaussian processes an d kernel methods) but with novel modifications to them that had not been p reviously encountered. With these exact mappings in hand\, the natural que stion is to what extent they bear relevance to neural networks at finite w idth. I’ll argue that the choice of learning rate is a crucial factor in dynamics away from this limit and naturally classifies deep networks into two classes separated by a sharp phase transition. This is elucidated in a class of solvable simple models we present\, which give quantitative pre dictions for the two phases. Quite remarkably\, we test these empirically in practical settings and find excellent agreement.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/10/ END:VEVENT BEGIN:VEVENT SUMMARY:Omri Azencot (UCLA) DTSTART:20200701T210500Z DTEND:20200701T213000Z DTSTAMP:20260404T094752Z UID:SciDL/11 DESCRIPTION:Title: Robust Prediction of High-Dimensional Dynamical Systems using Koopma n Deep Networks\nby Omri Azencot (UCLA) as part of Workshop on Scienti fic-Driven Deep Learning (SciDL)\n\n\nAbstract\nWe present a new deep lear ning approach for the analysis and processing of time series data. At the core of our work is the Koopman operator which fully encodes a nonlinear d ynamical system. Unlike the majority of Koopman-based models\, we consider dynamics for which the Koopman operator is invertible. We exploit the str ucture of these systems to design a novel Physically-Constrained Learning (PCL) model that takes into account the inverse dynamics while penalizing for inverse prediction. Our architecture is composed of an autoencoder com ponent and two Koopman layers for the dynamics and their inverse. To motiv ate our network design\, we investigate the connection between invertible Koopman operators and pointwise maps\, and our analysis yields a loss term which we employ in practice. To evaluate our work\, we consider several c hallenging nonlinear systems including the pendulum\, fluid flows on curve d domains and real climate data. We compare our approach to several baseli ne methods\, and we demonstrate that it yields the best results for long t ime predictions and in noisy settings.\n LOCATION:https://stable.researchseminars.org/talk/SciDL/11/ END:VEVENT END:VCALENDAR