Streamlining the Compression of Indic Language Models with NyunZero
Background
In the rapidly evolving landscape of AI, LLMs play a pivotal role in understanding and generating human-like text. OpenHathi, based on the impressive LLaMA-7B architecture, stands out as a powerful Indic language model. Leveraging its capabilities can significantly enhance natural language processing tasks in various applications. In this article, we will explore the seamless compression of OpenHathi with AWQ quantization and TensorRT-LLM engine conversion made possible through NyunZero.