DH{}DataHunter

Dataset from Hugging Face · Salesforce

WikiText

Open on Hugging Face
License
CC BY-SA 3.0, gfdlcommercial use allowed
This license allows commercial use. Usually you must credit the author; read the license for the exact terms.
Formats
parquet, PARQUET
Size
614 MB
Updated
Jan 4, 2024

Description

Dataset Card for "wikitext" Dataset Summary The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons Attribution-ShareAlike License. Compared to the preprocessed version of Penn Treebank (PTB), WikiText-2 is over 2 times larger and WikiText-103 is over 110 times larger. The WikiText dataset also features a far…

Description from Hugging Face, shortened.

Files (14)

  • PARQUETwikitext-103-raw-v1/test-00000-of-00001.parquet715 kB
  • PARQUETwikitext-103-raw-v1/train-00000-of-00002.parquet150 MB
  • PARQUETwikitext-103-raw-v1/train-00001-of-00002.parquet150 MB
  • PARQUETwikitext-103-raw-v1/validation-00000-of-00001.parquet642 kB
  • PARQUETwikitext-103-v1/test-00000-of-00001.parquet705 kB
  • PARQUETwikitext-103-v1/train-00000-of-00002.parquet149 MB
  • PARQUETwikitext-103-v1/train-00001-of-00002.parquet149 MB
  • PARQUETwikitext-103-v1/validation-00000-of-00001.parquet640 kB
  • PARQUETwikitext-2-raw-v1/test-00000-of-00001.parquet715 kB
  • PARQUETwikitext-2-raw-v1/train-00000-of-00001.parquet6.1 MB
  • PARQUETwikitext-2-raw-v1/validation-00000-of-00001.parquet642 kB
  • PARQUETwikitext-2-v1/test-00000-of-00001.parquet669 kB
  • PARQUETwikitext-2-v1/train-00000-of-00001.parquet5.8 MB
  • PARQUETwikitext-2-v1/validation-00000-of-00001.parquet603 kB

DataHunter does not store or host these files: every link leads to the original source. Check the license there before using the data.

Search other datasets