During the past few years, neural network based models have shown great success in natural language processing tasks. By encoding text units as dense low-dimensional representations, these models aim to understanding the semantics of the text through an end-to-end learning procedure. In this thesis, we delve into neural network based natural language understanding in terms of different granularities, i.e., from words, sentences, to documents. In particular, we investigate one classical task for each granularity, namely, word embedding learning, sentence classification, and document summarization.

Word embedding learning targets at projecting the words to a vector space where semantically related words are geometrically close. Such embeddings can facilitate many high-level tasks like text classification, summarization, and translation. We propose a grouping-based context predictive word embedding model. Based on the relatedness, words in a context window are split into several groups based on their relatedness, where words in the same group are combined while different groups are independent. A non-parametric clustering approach is proposed to conduct the context grouping. We show our model can learn high quality word representations capturing both semantic and syntactic information through experiments.

Sentence classification is one of the hottest applied areas for neural models. While achieving high accuracy, deep neural models have been shown easy to be attacked by adversarial examples. As a defense, we can improve the robustness of models by joint training with adversarial examples. We propose an adversarial learning framework for sentence classification based on generative adversarial networks, where a generator is responsible for crafting adversarial examples, and a classifier is trained to resist wrong predictions for the generated adversarial examples. Experimental results demonstrate our framework can learn a classifier with better generalization performance.

Document summarization aims to generating a shorter version for a given document by keeping the most important information. Focusing on the reviews-like sentiment-oriented documents, we propose the concept of sentiment-preserving document summarization, with the goal of summarizing the documents by preserving its main sentiments and not sacrificing readability. To tackle this problem, we devise an weakly-supervised extractive framework with a document encoder, a sentence extractor, a sentiment classifier, and a naturalness discriminator. The effectiveness of our framework is validated on the benchmark datasets.