mradermacher

/

model_requests

like 74

oh, wow, mythomax. sure, it's queued :) next time, please provide a url to the repo, if at all possible, though.
cheers!

Unfortunately, it has a broken vocabulary, so can't be converted till it's being fiixed:

WARNING:hf-to-gguf:ignore token 32000: id is out of range, max=31999

Oops forgot about the full url, sorry, I'll remember next time.

Unfortunately, it has a broken vocabulary, so can't be converted till it's being fiixed:

WARNING:hf-to-gguf:ignore token 32000: id is out of range, max=31999

Ah damn, that's a shame. It's a super rare model.

Undi had a similar model Amethyst-13B-Mistral but the repo is gone now.

Well thanks for trying anyway. :)

Yeah, these things suck.

I'll see if I get a look at maybe working around the problem.

I simply removed the offending token. No clue if the resulting model will run. Please give it a try (once finished) and report back if its broken.

The quants should appear at https://hf.tst.eu/model#RemmMistral13b-i1-GGUF

I just downloaded the Q8_0 and it works perfectly. It's rather good for a 13b too, prose and story flow is a little unique (probably due to the mistral influence).

What did you do to make it work exactly? Might look into creating a larger version of it (possibly a 20B) and merging it with some of Undi's models.

Thanks again! :)

I simply edited tokenizer_config.json and removed the out-of range pad token (32000). I think the older convert.py or whatever is called had some options to pad/cut the vocabulary for this purpose, which might have helped back then when it was quantized in the past.

Anyway, thanks for testing, I might apply this "fix" more in the future.

No problem.

Thanks for the quant and the info. Will be using it to tinker with this model, I think it has potential. :)

gooood :)

@Elfrino maybe there is a better fix possible, if the tensors support more tokens, I'll experiment

Nope, in this case, the fix seems correct the model definitely doesn't support the pad token.