extio-de cb52446985 Batched loading and chunking of text datasets;

2025-02-09 16:17:27 +01:00

..

dataset

Batched loading and chunking of text datasets;

2025-02-09 16:17:27 +01:00

validation

initial commit

2024-10-20 20:59:27 +02:00

work

initial commit

2024-10-20 20:59:27 +02:00

config.ini

Batched loading and chunking of text datasets;

2025-02-09 16:17:27 +01:00

llama3_prompt_min.txt

Dataset is now directly tokenized to avoid further post-processing of the trainer; locCustomPromptTemplate to specify a custom Jinja template; showChatTemplate for debugging;

2024-12-26 21:42:59 +01:00