deepseek-ai

/

DeepSeek-R1-Distill-Qwen-32B

like 333

Follow

DeepSeek 9.15k

I noticed that the config.json here has a 128k context size (like you might have with the yarn settings enabled for Qwen 2.5 models) but no yarn specific config like:

  "rope_scaling": {
    "factor": 4.0,
    "original_max_position_embeddings": 32768,
    "type": "yarn"
  }

I imagine we should add these, because you did not in fact change the original max_positional_embeddings, right?