Plagiarism-detection framework for digital libraries

In digital libraries, documents are in digital forms and they are especially vulnerable from being copied. Existing copy detection methods exhaustively compare every single sentence of two documents and identify the degree of overlapping of the two documents. This approach is not scalable as the number of sentences for each document is often plentiful. In this paper, we propose a copy detection mechanism which could eliminate unnecessary comparisons. This is achieved by pre-parsing the documents to quantify their semantic meanings; comparisons between documents describing different topics could be eliminated s it will not serve any purpose to copy from a document describing an unrelated topic. This process is recursively applied to sections, subsections, subsubsections, etc. until we find two paragraphs which are highly related semantically. The paragraphs are then compared in detail, i.e., per-sentence basis, to determine if the paragraphs are overlapped in a substantive way. The parsing process is based on document retrieval techniques with some helpful heuristics that extract keywords from the documents to index the semantics for each document, section, subsection, and so forth. Weights based on relative occurrences of the keywords are assigned to individual keywords to form a keyword vector. The semantic relationships between different documents, sections, subsections, or paragraphs can be represented by the dot product of their corresponding keyword vectors as in document retrieval systems.