Model Compression & Adaptation Support Grid
Below are tables summarizing the support for various compression and adaptation techniques across different models. Apart from these, other similar models may be supported but have not been tested.
Nyuntam Text-Generation
Model | AWQ | LMQuant (QoQ) | AQLM | TensorRT | FLAP |
---|---|---|---|---|---|
LLaMA | ✓ | ✓ | ✓ | ✓ | ✓ |
LLaMA-2 | ✓ | ✓ | ✓ | ✓ | ✓ |
LLaMA-3 | ✓ | ✓ | ✓ | - | ✓ |
Vicuna | ✓ | ✓ | ✓ | ✓ | ✓ |
Mistral | ✓ | ✓ | ✓ | ✓ | ✓ |
Mixtral | ✓ | ✓ | ✓ | ✓ | ✓ |
Gemma | ✓ | - | ✓ | ✓ | - |