Look at length vs metric score

If your sentences in the training data have high variability in lengths, check if there is a correlation between the misclassification rate and length.