Dataset from Hugging Face · lightonai
lightonai/trec-covid-decontaminated
- License
- MITcommercial use allowedLicense text
- This license allows commercial use. Usually you must credit the author; read the license for the exact terms.
- Formats
- parquet, PARQUET
- Size
- 54 MB
- Updated
- Mar 25, 2026
Description
trec-covid (Decontaminated) A decontaminated version of the trec-covid dataset from the BEIR benchmark, with samples found in the mgte-en pre-training dataset removed. Decontamination methodology Contamination was detected using a two-pass approach against the full mgte-en dataset (484 GB, 1,235 parquet files): Pass 1: Exact hash matching All texts (queries and corpus documents) were normalized (lowercased, unicode NFKD, whitespace collapsed) and…
Description from Hugging Face, shortened.
Files (3)
DataHunter does not store or host these files: every link leads to the original source. Check the license there before using the data.