A New Type of Neurons for Machine Learning

In machine learning, an artificial neural network is the mainstream approach. Such a network consists of many neurons. These neurons are of the same type characterized by the 2 features: (1) an inner product of an input vector and a matching weighting vector of trainable parameters and (2) a nonlinear excitation function. Here, we investigate the possibility of replacing the inner product with a quadratic function of the input vector, thereby upgrading the first‐order neuron to the second‐order neuron, empowering individual neurons and facilitating the optimization of neural networks. Also, numerical examples are provided to illustrate the feasibility and merits of the second‐order neurons. Finally, further topics are discussed.