Adaptive Group Lasso Neural Network Models for Functions of Few Variables and Time-Dependent Data

Learning nonlinear functions from time-varying measurements is always difficult due to the high correlation among observations. This task is more challenging when the target function is high dimensional. In this work, we propose a new method to learn high dimensional functions which depends only on a few unknown coordinates from a set of time-varying measurements. More precisely, we approximate the target function by a neural network and enforce an adaptive group Lasso constraint on the suitable weight matrix to represent the low-dimensional property of the unknown function. Using the non-negative property of the Bregman distance, we show that the proposed optimization procedure achieves loss decay. Our empirical studies show that the proposed method outperforms recent state-of-the-art methods including the sparse dictionary matrix method, and neural networks with or without group Lasso penalty.