Use setup_training_state to fix standalone checkpoint unit tests#4394
Open
xibinliu wants to merge 1 commit into
Open
Use setup_training_state to fix standalone checkpoint unit tests#4394xibinliu wants to merge 1 commit into
xibinliu wants to merge 1 commit into
Conversation
02e5767 to
9f10e80
Compare
9f10e80 to
f9d883b
Compare
…checkpointer restoration
f9d883b to
21f76b3
Compare
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
NuojCheng
approved these changes
Jul 9, 2026
shralex
approved these changes
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix the checkpoint restore issue in standalone checkpoint test:
Replaced manual abstract state evaluation and load_state_if_possible calls in standalone_checkpointer.py with setup_training_state(...). This aligns standalone checkpointer with standard training and ensures proper NNX state unboxing, sharding resolution, and fallback handling.
Updated setup_initial_state and setup_training_state to return a was_restored: bool flag.
In standalone_checkpointer.py, guarded add_entropy_to_checkpoint(state) so synthetic entropy is only injected on clean runs, preserving restored optimizer moments (mu and nu) when testing checkpoint resumption.
When running under pure_nnx=True, converted state using to_linen_checkpoint_dict(state.to_pure_dict()) before calling save_checkpoint, guaranteeing that saved checkpoints conform to the standard Linen disk layout and restoring cleanly without ShapeDtypeStruct type errors.
FIXES: b/532183169
Tests
standalone_dl_ckpt_test.py test pass:
Checklist
Before submitting this PR, please make sure (put X in square brackets):
gemini-reviewlabel.