graph convolutional networks, deep learning, topic modeling

Document classification is a frequently occurring task in Natural Lan-guage Processing. Here we consider a classification of scientific papersto categories which correspond to the venues where these papers weresubmitted to. As a proof of concept, we work with a dataset containing17000 papers classified to 3 different categories (AI, ML, NLP). We usea novel neural network architecture called Graph Convolutional Networkwhich operates on a graph. In our case, the nodes of the graph correspondto scientific papers and edges correspond to citations. These nodes arerepresented as feature vectors extracted from the abstracts of papers andedges are represented as matrices of learnable parameters which propa-gate the signal during the forward pass. Thus, when classifying a paper toits category, the network can use information from the papers which areconnected to that particular paper. We show that with a representationobtained by Topic Modeling, which is being fed to Graph ConvolutionalNetwork, we are able to achieve reasonable accuracy with a very smallnumber of training examples. Concretely, we compare to bi-directionalLSTM which, when trained on 15000 examples, achieves an accuracy of87%. Our model is not able to improve over this LSTM in the large-training-set regime but is able to achieve an accuracy of 84% when trainedon 1000 examples only which is a big improvement over the same LSTMwhich correctly classifies only 67% of examples in this small-training-setregime. Our ablation experiments show that the main reason for thissample efficiency is a low-dimensional representation obtained by TopicModeling. When we train a feed forward network with the same countof parameters using the representation obtained by Topic Modeling, weachieve an accuracy of 81%. By further exploiting the information aboutcited articles we gain additional 3% improvement.