This repository was archived by the owner on Jul 21, 2025. It is now read-only.
This repository was archived by the owner on Jul 21, 2025. It is now read-only.
Can int8 in pre-training large model ??? #521
Description
Hello guys! I would like to know if you have experimented with int8 precision in the pre-training of your large models. Can int8 replace fp16 and fp32 to achieve faster training speeds? Are there any relevant case studies or experiments?