Webis-TLDR-17 Corpus
- zenodo.org
- paperswithcode.com
zipUpdated Jan 24, 2020+ more versions
Share
Facebook
Twitter
EmailClick to copy linkLink copiedCiteShahbaz Syed; Michael Voelske; Martin Potthast; Benno Stein; Shahbaz Syed; Michael Voelske; Martin Potthast; Benno Stein (2020). Webis-TLDR-17 Corpus [Dataset]. http://doi.org/10.5281/zenodo.1043504zipAvailable download formatsUnique identifierhttps://doi.org/10.5281/zenodo.1043504Dataset updatedJan 24, 2020AuthorsShahbaz Syed; Michael Voelske; Martin Potthast; Benno Stein; Shahbaz Syed; Michael Voelske; Martin Potthast; Benno SteinLicenseAttribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automaticallyDescriptionThis corpus contains preprocessed posts from the Reddit dataset, suitable for abstractive summarization using deep learning. The format is a json file where each line is a JSON object representing a post. The schema of each post is shown below:
- author: string (nullable = true)
- body: string (nullable = true)
- normalizedBody: string (nullable = true)
- content: string (nullable = true)
- content_len: long (nullable = true)
- summary: string (nullable = true)
- summary_len: long (nullable = true)
- id: string (nullable = true)
- subreddit: string (nullable = true)
- subreddit_id: string (nullable = true)
- title: string (nullable = true)
Specifically, the content and summary fields can be directly used as inputs to a deep learning model (e.g. Sequence to Sequence model ). The dataset consists of 3,848,330 posts with an average length of 270 words for content, and 28 words for the summary. The dataset is a combination of both the Submissions and Comments merged on the common schema. As a result, most of the comments which do not belong to any submission have null as their title.
Note : This corpus does not contain a separate test set. Thus it is up to the users to divide the corpus into appropriate training, validation and test sets.
- W
Webis-TLDR-17
- webis.de
1043504Updated 2017ShareFacebookTwitterEmailClick to copy linkLink copiedCiteShahbaz Syed; Michael Völske; Martin Potthast; Benno Stein (2017). Webis-TLDR-17 [Dataset]. http://doi.org/10.5281/zenodo.10435041043504Available download formatsUnique identifierhttps://doi.org/10.5281/zenodo.1043504Dataset updated2017Dataset provided byLeipzig University
The Web Technology & Information Systems Network
Bauhaus-Universität Weimar
University of Kassel, hessian.AI, and ScaDS.AIAuthorsShahbaz Syed; Michael Völske; Martin Potthast; Benno SteinLicenseAttribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automaticallyDescriptionThe Webis TLDR Corpus (2017) consists of approximately 4 Million content-summary pairs extracted for Abstractive Summarization, from the Reddit dataset for the years 2006-2016. This corpus is first of its kind from the social media domain in English and has been created to compensate the lack of variety in the datasets used for abstractive summarization research using deep learning models.
- O
tldr-17
- opendatalab.com
zipUpdated Dec 16, 2023+ more versionsShareFacebookTwitterEmailClick to copy linkLink copiedCiteBauhaus University, Weimar (2023). tldr-17 [Dataset]. https://opendatalab.com/OpenDataLab/tldr-17zipAvailable download formatsDataset updatedDec 16, 2023Dataset provided byBauhaus University, WeimarLicenseAttribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automaticallyDescriptionThis corpus contains preprocessed posts from the Reddit dataset (Webis-TLDR-17). The dataset consists of 3,848,330 posts with an average length of 270 words for content, and 28 words for the summary. Features includes strings: author, body, normalizedBody, content, summary, subreddit, subreddit-id. Content is used as document and summary is used as summary.
- E
Webis-TLDR-17 Corpus
- live.european-language-grid.eu
jsonUpdated Dec 30, 2017ShareFacebookTwitterEmailClick to copy linkLink copiedCite(2017). Webis-TLDR-17 Corpus [Dataset]. https://live.european-language-grid.eu/catalogue/corpus/5176jsonAvailable download formatsDataset updatedDec 30, 2017LicenseAttribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automaticallyDescriptionDataset contains 3 Million pairs of content and self-written summaries mined from Reddit. It is one of the first large-scale summarization dataset from the social media domain.
Not seeing a result you expected?
Learn how you can add new datasets to our index.
FacebookTwitterEmailAttribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automatically
This corpus contains preprocessed posts from the Reddit dataset, suitable for abstractive summarization using deep learning. The format is a json file where each line is a JSON object representing a post. The schema of each post is shown below:
- author: string (nullable = true)
- body: string (nullable = true)
- normalizedBody: string (nullable = true)
- content: string (nullable = true)
- content_len: long (nullable = true)
- summary: string (nullable = true)
- summary_len: long (nullable = true)
- id: string (nullable = true)
- subreddit: string (nullable = true)
- subreddit_id: string (nullable = true)
- title: string (nullable = true)
Specifically, the content and summary fields can be directly used as inputs to a deep learning model (e.g. Sequence to Sequence model ). The dataset consists of 3,848,330 posts with an average length of 270 words for content, and 28 words for the summary. The dataset is a combination of both the Submissions and Comments merged on the common schema. As a result, most of the comments which do not belong to any submission have null as their title.
Note : This corpus does not contain a separate test set. Thus it is up to the users to divide the corpus into appropriate training, validation and test sets.