[High Risk]Support for immediate saving #965
Open
+382
−72
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
for more information, see https://pre-commit.ci
|
Thanks for the great work! Could you check the maximum RAM usage to see whether it has been reduced significantly, as expected? |
wenhuach21
reviewed
Oct 30, 2025
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
… into kaihui/save_block
wenhuach21
changed the title
wenhuach21
reviewed
Oct 31, 2025
wenhuach21
reviewed
Oct 31, 2025
wenhuach21
reviewed
Oct 31, 2025
wenhuach21
approved these changes
Oct 31, 2025
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>
wenhuach21
requested changes
Oct 31, 2025
| self.is_packing_immediate = False # whether to pack the layer immediately after tuning | ||
|
|
||
| # Whether to pack the layer immediately after tuning | ||
| self.is_packing_immediate = kwargs.pop("is_packing_immediate", False) |
There was a problem hiding this comment.
Packing immediate is set automatically before. Have you handled this when exporting >1 formats? So it’s better not to set it in the API. Besides, as discussed, set save_immediate to True.
Another thing to verify is the time cost of save_immediate, have you measured the total quantization time comparing to main branch ?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#788
memory check
Qwen2.5-7B-Instruct-w4g32 RTN auto_round
mprof peak
16659.441MiB->9200.250MiB~55%