Automated Deep Learning: Neural Architecture Search Is Not the End

Deep learning (DL) has proven to be a highly effective approach for developing models in diverse contexts, including visual perception, speech recognition, and machine translation. Automated deep learning (AutoDL) endeavors to minimize the need for human involvement and is best known for its achievements in neural architecture search (NAS). In this monograph, the authors examine research efforts into automation across the entirety of an archetypal DL workflow. In so doing, they propose a comprehensive set of ten criteria by which to assess existing work in both individual publications and broader research areas, namely novelty, solution quality, efficiency, stability, interpretability, reproducibility, engineering quality, scalability, generalizability, and eco-friendliness. Aimed at students and researchers, this monograph provides an evaluative overview of AutoDL in the early 2020s, identifying where future opportunities for progress may exist.