extio-de
|
ded2b6cecc
|
Shuffle concatenated dataset;
|
2025-02-16 10:51:39 +01:00 |
|
extio-de
|
cb52446985
|
Batched loading and chunking of text datasets;
|
2025-02-09 16:17:27 +01:00 |
|
extio-de
|
f253fde344
|
Don't reload model for the merge step;
|
2025-02-08 11:25:12 +01:00 |
|
extio-de
|
a0e9a5a33a
|
Batched, parallel dataset loading; Disable dataset caching;
|
2025-01-25 18:12:31 +01:00 |
|
extio-de
|
2799b9c555
|
Process conversation datasets;
|
2025-01-11 21:58:29 +01:00 |
|
extio-de
|
63192f2bc0
|
Dataset is now directly tokenized to avoid further post-processing of the trainer; locCustomPromptTemplate to specify a custom Jinja template; showChatTemplate for debugging;
|
2024-12-26 21:42:59 +01:00 |
|
extio-de
|
cb9e62dbac
|
Dataset: Support history/instruct/completion style;
|
2024-12-19 14:05:38 +01:00 |
|
extio-de
|
b10546abc2
|
lm-finetuner: Close assistant responses;
|
2024-12-02 11:33:08 +01:00 |
|
extio-de
|
ff2385bf50
|
lm-finetuner: Merger: Dequantize before merging lora adapter; More lora options;
|
2024-12-01 13:03:45 +01:00 |
|
extio-de
|
aa958eeb81
|
Add eot tokens to texts;
|
2024-10-27 20:47:03 +01:00 |
|
extio-de
|
8e2f75e6f4
|
Added more training parameters; Removed chat emulation of text datasets; Fixes;
|
2024-10-27 09:59:16 +01:00 |
|
extio-de
|
91b211547b
|
Readme
|
2024-10-20 21:53:33 +02:00 |
|
extio-de
|
838664935b
|
initial commit
|
2024-10-20 20:59:27 +02:00 |
|