Stephen Friess, Peter Tino, Stefan Menzel, Zhao Xu, Bernhard Sendhoff, Xin Yao, "Spatio-Temporal Activity Recognition for Evolutionary Search Behavior Prediction", International Joint Conference on Neural Networks, 2022.

Traditional methods for solving problems within computer science rely mostly upon the application of handcrafted algorithms. As however manual engineering of them can be considered to be a tedious process, it is interesting to consider how far internal mechanisms can be directly learned in an end-to-end manner instead. This is especially tempting when considering metaheuristic and evolutionary optimization routines which rely inherently upon stochastic noise. While prior research indicated, that problem-dependent derandomization during run-time can significantly boost their performance, it has as of yet not been investigated within the context of end-to-end learning. This can be partly attributed to the fact that in many scenarios characteristics to map an optimization problem to an optimal solver may not be accessible in advance. Thus, to unveil this black box character, one has to rely instead upon metadata generated during runtime. But while methods for extracting spatial features have been already proposed within the literature, these unfortunately fail to acknowledge the time-dependent nature of the retrievable meta-data. For this reason we specifically propose architectures for spatio-temporal data processing within this work. Particularly, we find that our proposed GCN-GRU and LSTM architectures, which take inspiration from CNN-LSTMs originally proposed for activity recognition in multimedia data-streams, demonstrate high efficiency and most consistent performance on time series of variable length. Further, we can also demonstrate that the class activation map for interpretable learning with time series data helps to understand and reflects problem-dependent properties of the search behavior of an optimization algorithm.