Quantization

BMQuant

class quant.BMQuant[source]

BMQuant enables quantization-aware training of PLMs by using cpm-kernels.

classmethod quantize(model, config)[source]

Practitioners can turn on quantization by is_quant in the config, which will replace all linear layers with quantized linear layers. BMCook provides the simulation of 8-bit quantization.

Parameters

model – Model to quantize.
config – Configuration of the quantization.