HuggingFaceFW/clean-wikipedia
Viewer
•
Updated
•
61.2M
•
1.95k
•
22
AI & ML interests
We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)
Recent Activity
View all activity
Papers
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Filtered Cebuano?
➕
1
3
#3 opened 4 days ago
by
KaraKaraWitch
OCR or not classifier
7
#6 opened about 2 months ago
by
MaartenMarx
A Few Questions About the Implementation Details of the finepdfs Project
5
#24 opened 11 days ago
by
yoliax
docs: fix typo
1
#2 opened 5 days ago
by
stefan-it
guipenedo
updated
a
dataset
5 days ago
guipenedo
published
a
dataset
5 days ago
guipenedo
published
a
Space
5 days ago
hynky
published
a
dataset
5 days ago
Deciding on extraction path
4
#10 opened about 1 month ago
by
Mdspike
Were the original PDFs saved?
10
#2 opened about 2 months ago
by
staghado
Docling output
1
#4 opened about 2 months ago
by
akreal
Can additional corpuses further train this model?
1
#13 opened about 1 month ago
by
fenjamin
hynky
updated
a
dataset
6 days ago
guipenedo
updated
a
collection
6 days ago
The "file_path" data field appears to primarily contain cc-index paths rather than WARC paths.
4
#16 opened about 1 month ago
by
lnstrument
Github LInk or XGBoost Model
2
#22 opened 26 days ago
by
richard-ac