Iterative Privileged Learning

Abstract

While in the learning using privileged information paradigm, privileged information may not be as informative as example features in the context of making accurate label predictions, it may be able to provide some effective comments (e.g., the values of the auxiliary function) like a human teacher on the efficacy of the learned model. In a departure from conventional static manipulations of privileged information within the support vector machine framework, this paper investigates iterative privileged learning within the context of gradient boosted decision trees (GBDTs). As the learned model evolves, the comments learned from privileged information to assess the model should also be actively upgraded instead of remaining static and passive. During the learning phase of the GBDT method, new DTs are discovered to enhance the performance of the model, and iteratively update the comments generated from the privileged information to accurately assess and coach the up-to-date model. The resulting objective function can be efficiently solved within the gradient boosting framework. Experimental results on real-world data sets demonstrate the benefits of studying privileged information in an iterative manner, as well as the effectiveness of the proposed algorithm.