Lossy Compression of LLM Weights in Safetensors Format

Abstract

Large language models (LLMs) such as DeepSeek and Google’s Gemma require hundreds of gigabytes of storage when shared via Hugging Face. These models are typically split into many safetensor files--secure, fast tensor storage formats--each often 4-5 GB in size. Downloading or distributing such massive models is time-consuming. Common solutions like 8-bit or 4-bit quantization reduce size by lowering precision, but may still yield sizable files or require retraining. In contrast, error-bounded scientific compressors (for example, SZ) can achieve much higher compression ratios with controlled error. This work applies the Python SZ implementation (PySZ) to a DeepSeek-V3 safetensor shard. A compression ratio of approximately 13.33× was obtained (from ~4.4 GB to ~0.33 GB) while successfully reconstructing the tensor file for potential model use. Methodology, compression statistics, and implications for model fidelity versus quantization are described.