Im trying to load guanaco 33b, it says it works with all version of gptq for llama but im getting errors like
size mismatch for model.layers.59.mlp.up_proj.scales: copying a param with shape torch.Size([1, 17920]) from checkpoint, the shape in current model is torch.Size([52, 17920]).
I am loading with
load_quant(llamaModel.path, llamaModel.path+"/"+llamaModel.modelFile, 4, 128)
which works for the other models i've tested. anyone know what the problem could be?