Merging Comparable Data Sources for the Discrimination of Similar Languages: The DSL Corpus Collection

Marcos Zampieri1, Nikola Ljubesic2, Jorg Tiedemann3
1Saarland University, 2University of Zagreb, 3University of Uppsala


Abstract

This paper presents the compilation of the DSL corpus collection created for the DSL (Discriminating Similar Languages) shared task to be held at the VarDial workshop at COLING 2014. The DSL corpus collection were merged from three comparable corpora to provide a suitable dataset for automatic classification to discriminate similar languages and language varieties. Along with the description of the DSL corpus collection we also present results of baseline discrimination experiments reporting performance of up to 87.4% accuracy.