The paper proposes an efficient weight-only quantization method for large language models (LLMs) to reduce memory consumption and accelerate inference. The method utilizes a heuristic approach that only uses the model weights of a pre-trained model, without requiring additional fine-tuning. The approach addresses the challenges and issues associated with LLM quantization and achieves higher throughput on the same number of GPUs with minimal accuracy loss.