chatglm2-6b-int4报错RuntimeError: expected m1 and m2 to have the same dtype, but got: c10::Half != float
#23
by
frank08932
- opened
chatglm2-6b-int4 这个版本是不是不能用啊，一运行就提示这个
frank08932
changed discussion title from
RuntimeError: expected m1 and m2 to have the same dtype, but got: c10::Half != float
to chatglm2-6b-int4报错RuntimeError: expected m1 and m2 to have the same dtype, but got: c10::Half != float
我也是
同样问题
gpu上没问题，cpu上就会报这个错误
cpu上使用float32运行，改成
model = AutoModel.from_pretrained("THUDM/chatglm2-6b-int4", trust_remote_code=True).float()