Christian Limberg, Heiko Wersing, Helge Ritter, "Efficient Accuracy Estimation for Instance-Based Incremental Active Learning", European Symposium on Artificial Neural Networks, 2018.Abstract
Estimating systems accuracy is crucial for applications of in- cremental learning. In this paper, we introduce the Distogram Estimation (DGE) approach to estimate the accuracy of instance-based classifiers. By calculating relative distances to samples it is possible to train an offline regression model, capable of predicting the classifiers accuracy on unseen data. Our approach requires only a few supervised samples for training and can instantaneously be applied on unseen data afterwards. We evaluate our method on five benchmark data sets and for a robot object recognition task. Our algorithm clearly outperforms two baseline methods both for random and active selection of incremental training examples.