Using learning algorithm behavior to chart task space: The DICES distance

Article type: Research Article

Authors: Peterson, Adam H.^{; *} | Martinez, Tony R.

Affiliations: Computer Science Department, Brigham Young University, Provo, UT, USA

Correspondence: [*] Corresponding author. E-mail: adam@axon.cs.byu.edu.

Abstract: In theory, learning is not possible over all tasks in general. In practice, the tasks for which learning is desired exhibit significant regularity, which makes learning practical. For the most effective learning, it is valuable to understand the nature of this regularity and how it manifests in the tasks where learning is applied. This research presents the DICES distance metric for finding similarity between learning tasks. With this distance metric, a collection of learning tasks can be given a distance matrix. This distance matrix can be used for visualizing the relationships between learning tasks and searching through task space for tasks which are similar in nature. Examples of task visualization are given, and other possible applications of this tool are touched upon. Such applications include learning algorithm selection, transfer learning, and analysis of empirical results.

Keywords: Machine learning, meta learning, task similarity, task space

DOI: 10.3233/IDA-2010-0425

Journal: Intelligent Data Analysis, vol. 14, no. 3, pp. 355-367, 2010