max_window_layers and sliding_window are set when use_sliding_window is false
#4
by
seungduk
- opened
Hello,
While reviewing the code, I noticed that these values seem to be ignored when use_sliding_window is set to False:
https://github.com/huggingface/transformers/blob/5c75087aeee7081025370e10d1f571a11600f1ae/src/transformers/models/qwen2/modular_qwen2.py#L74
Is it acceptable to ignore these values, or are they being used elsewhere, even when use_sliding_window is False?
I would greatly appreciate any information you can share.
Best,
Seungduk
Additionally, the bos_token is set to the same value as the eos_token in the config.json file, but bos_token is set to None in the tokenizer_config.json.