The Technological Emergence of AutoML: A Survey of Performant Software and Applications in the Context of Industry

The Technological Emergence of AutoML presents a comprehensive snapshot of how AutoML has permeated into mainstream use within the early 2020s. This work surveys both their implementation and application in the context of industry. It also defines what a ‘performant’ AutoML system is – HCI support is valued highly here – and assesses how the current crop of available packages and services lives up to expectations. To do so in a systematic manner, this survey is structured as follows. Section 2 begins by elaborating on the notion of an ML workflow, conceptually framing AutoML in terms of the high-level operations required to develop, deploy and maintain an ML model. Section 3 uses this workflow to support the introduction of industry-related stakeholders and their interests/obligations. These requirements are unified into a comprehensive set of criteria, supported by methods of assessment, that determine whether an AutoML system can be considered performant. Section 4 launches the survey in earnest, assessing the nature and capabilities of existing AutoML technology beginning with an examination of open-source AutoML packages. The section additionally investigates AutoML systems that are designed for specific domains, as well as commercial products. Subsequently, Section 5 assesses where AutoML technology has been used and how it has fared. Academic work focusing on real-world applications is surveyed, as are vendor-based case studies. All key findings and assessments are then synthesized in Section 6, with commentary around how mature AutoML technology is, as well as whether there are obstacles and opportunities for future uptake. Finally, Section 7 provides a concluding overview on the technological emergence of AutoML.