"2 bit" HQQ again
#1
by
ProphetOfBostrom
- opened
ProphetOfBostrom/BagelMix-8x7B-2b-HQQ
'default' 2bit linear 4 bit attention in the main branch - but i might experiment with other more aggressive quants in another branch.