View a markdown version of this page

Plagiarism detection on AWS - Education Reference Architecture Diagrams

Plagiarism detection on AWS

Publication date: July 26, 2021 (Diagram history)

This architecture helps you create a plagiarism-detection service using AWS Step Functions, AWS Lambda, Amazon SageMaker AI, and Amazon OpenSearch Service.

Plagiarism detection on AWS diagram

A document uploaded to Amazon S3 triggers a Step Functions workflow that scores it for plagiarism.

The following steps describe the architecture:

  1. Copy the document you'd like to run plagiarism detection on to Amazon Simple Storage Service (Amazon S3).

  2. Amazon S3 event triggers start of Step Functions workflow.

  3. Lambda function extracts text from document using Tika (a content analysis toolkit that detects and extracts metadata and text from over a thousand different file types).

  4. For each paragraph in the document, text is passed to a pre-trained Bidirectional Encoder Representations from Transformers (BERT)-based model to extract word embedding vectors.

  5. For each word embedding vector, a K-Nearest Neighbor (KNN) search is run using a cosine-similarity algorithm.

  6. Amazon OpenSearch Service (OpenSearch Service) domain stores an index of pre-processed works that have been converted into word embedding vectors and indexed.

  7. Based on the configured similarity threshold that is compared against the OpenSearch Service query result score, an Amazon EventBridge event is raised, specifying source document information that has possibly been plagiarized with reference to relevant works.

Further reading

For additional information, see the following resources: