Sun Y. Evolutionary Deep Neural Architecture Search. Fundamentals,...2023 torrent

Textbook in PDF format

This book systematically narrates the fundamentals, methods, and recent advances of evolutionary deep neural architecture search chapter by chapter. This will provide the target readers with sufficient details learning from scratch. In particular, the method parts are devoted to the architecture search of unsupervised and supervised deep neural networks (DNN). The people, who would like to use deep neural networks but have no/limited expertise in manually designing the optimal deep architectures, will be the main audience. This may include the researchers who focus on developing novel evolutionary deep architecture search methods for general tasks, the students who would like to study the knowledge related to evolutionary deep neural architecture search and perform related research in the future, and the practitioners from the fields of computer vision, natural language processing, and others where the deep neural networks have been successfully and largely used in their respective fields.
The use of Evolutionary Computation (EC) methods to create optimal or nearly optimal Deep Neural Network (DNN) architectures is referred to as evolutionary deep neural architecture design. The design process of architectures is often formalized as an optimization problem, where EC algorithms are correctly created to tackle the optimization problem.
DNNs have had remarkable success in many complicated practical applications in recent years. It is well known that the performance of a DNN is only promising when the architecture is appropriate. The architecture, on the other hand, is typically created by hand, needing a high level of skill that is in scarce supply in practice. A promising deep architecture is difficult to create in practice without this kind of skill, which is frequently DNN expertise and domain understanding of the problem to be solved. Reinforcement learning techniques, gradient-based optimization algorithms, and EC methods are the three common methods utilized to construct the architectures of DNNs in the literature. This book mainly focuses on the EC methods for deep neural architecture design.
In this book, we will first introduce the fundamentals of commonly used EC methods, including Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Differential Evolution (DE), and Genetic Programming (GP). Following that, we will go through two different forms of evolutionary deep neural architecture design algorithms. They are the design algorithms for unsupervised DNNs, and the design algorithms for supervised DNNs. In addition, we will also discuss some recent efforts to speed up the execution of such algorithms. These algorithms are primarily based on the authors recent work, which has been published in journals and international conferences devoted to EC and neural networks. We think that by presenting them all together in this book, readers will be able to absorb related information more quickly and conveniently