Speaker: Frank Hutter
Date: 2022-05-05
Abstract
Deep Learning (DL) has been able to revolutionize learning from raw data (images, text, speech, etc) by replacing domain-specific hand-crafted features with features that are jointly learned for the particular task at hand. I propose to take deep learning to the next level, by also jointly (meta-)learning other, currently hand-crafted, elements of the deep learning pipeline: neural architectures and their initializations, training pipelines and their hyperparameters, self-supervised learning pipelines, etc.
I dub this new research direction Deep Learning 2.0 (DL 2.0), since
- Like deep learning, it replaces previous hand-crafted solutions with learned solutions (but now on the meta-level),
- It allows for a qualitatively new level of automated customization to the task at hand, and
- It allows for automatically finding meta-level tradeoffs with the objectives specified by the user, such as algorithmic fairness, interpretability, calibration of uncertainty estimates, robustness, energy consumption, etc. It can thus achieve Trustworthy AI by design, breaking qualitatively new ground for deep learning.
