Affiliations: [a]
School of Sciences, Southwest Petroleum University, Chengdu, China
| [b]
Institute for Artificial Intelligence, Southwest Petroleum University, Chengdu, China
| [c]
China Aerodynamics Research and Development Center, Mianyang, China
| [d]
College of Big Data and Software Engineering, Zhejiang Wanli University, Ningbo, China
Correspondence:
[*]
Corresponding author: Weibin Li, China Aerodynamics Research and Development Center, Mianyang, 621000 China. E-mail: liweibin@nudt.edu.cn.
Abstract: Whether the exact amount of training data is enough for a specific task is an important question in machine learning, since it is always very expensive to label many data while insufficient data lead to underfitting. In this paper, the topic that what is the least amount of training data for a model is discussed from the perspective of sampling theorem. If the target function of supervised learning is taken as a multi-dimensional signal and the labeled data as samples, the training process can be regarded as the process of signal recovery. The main result is that the least amount of training data for a bandlimited task signal corresponds to a sampling rate which is larger than the Nyquist rate. Some numerical experiments are carried out to show the comparison between the learning process and the signal recovery, which demonstrates our result. Based on the equivalence between supervised learning and signal recovery, some spectral methods can be used to reveal underlying mechanisms of various supervised learning models, especially those “black-box” neural networks.
Keywords: Machine learning, sampling theorem, frequency principle, signal recovery, neural network, Gaussian process regression
