Skip to content

[Refactor] Weight sync schemes refactor #3230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Nov 6, 2025
Merged

[Refactor] Weight sync schemes refactor #3230

merged 10 commits into from
Nov 6, 2025
+1,455 −274

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Nov 5, 2025 •
edited
Loading

@vmoens
Update
eb8dd34 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 5, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3230

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Nov 5, 2025
@vmoens
[Refactor] Weight sync schemes refactor
709bec7 
ghstack-source-id: f44e83d
Pull-Request: #3230
This was referenced Nov 5, 2025
Merged
Merged
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 5, 2025
@github-actions
Copy link

github-actions bot commented Nov 5, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 81.4945μs 80.9528μs 12.3529 KOps/s 12.1935 KOps/s $\color{#35bf28}+1.31\%$
test_tensor_to_bytestream_speed[torch.save] 0.1469ms 0.1453ms 6.8804 KOps/s 7.1915 KOps/s $\color{#d91a1a}-4.33\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1109s 0.1103s 9.0647 Ops/s 8.9955 Ops/s $\color{#35bf28}+0.77\%$
test_tensor_to_bytestream_speed[numpy] 2.8013μs 2.7959μs 357.6626 KOps/s 361.7384 KOps/s $\color{#d91a1a}-1.13\%$
test_tensor_to_bytestream_speed[safetensors] 45.7605μs 44.9896μs 22.2273 KOps/s 23.9342 KOps/s $\textbf{\color{#d91a1a}-7.13\%}$
test_simple 0.6492s 0.5671s 1.7635 Ops/s 1.7821 Ops/s $\color{#d91a1a}-1.04\%$
test_transformed 1.2040s 1.1122s 0.8991 Ops/s 0.9018 Ops/s $\color{#d91a1a}-0.29\%$
test_serial 1.6694s 1.6455s 0.6077 Ops/s 0.6045 Ops/s $\color{#35bf28}+0.53\%$
test_parallel 1.0708s 1.0457s 0.9563 Ops/s 0.9378 Ops/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-True-True-True-True] 0.2603ms 44.6163μs 22.4133 KOps/s 22.8933 KOps/s $\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-True-True-True-False] 50.9820μs 24.5168μs 40.7884 KOps/s 41.1109 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-True-True-False-True] 55.8830μs 25.2652μs 39.5801 KOps/s 40.8532 KOps/s $\color{#d91a1a}-3.12\%$
test_step_mdp_speed[True-True-True-False-False] 79.5430μs 13.5864μs 73.6031 KOps/s 75.2138 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[True-True-False-True-True] 78.0540μs 47.4600μs 21.0704 KOps/s 21.5701 KOps/s $\color{#d91a1a}-2.32\%$
test_step_mdp_speed[True-True-False-True-False] 63.6830μs 27.2298μs 36.7245 KOps/s 37.2184 KOps/s $\color{#d91a1a}-1.33\%$
test_step_mdp_speed[True-True-False-False-True] 59.4230μs 28.0750μs 35.6189 KOps/s 36.6226 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[True-True-False-False-False] 48.9420μs 16.4820μs 60.6722 KOps/s 62.4436 KOps/s $\color{#d91a1a}-2.84\%$
test_step_mdp_speed[True-False-True-True-True] 83.0540μs 49.6662μs 20.1344 KOps/s 19.9010 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-False-True-True-False] 98.4850μs 29.0305μs 34.4465 KOps/s 33.8039 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[True-False-True-False-True] 54.7630μs 27.8405μs 35.9189 KOps/s 36.4007 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-False-True-False-False] 44.3520μs 16.2796μs 61.4267 KOps/s 63.4589 KOps/s $\color{#d91a1a}-3.20\%$
test_step_mdp_speed[True-False-False-True-True] 89.7540μs 52.4947μs 19.0496 KOps/s 19.2335 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-False-False-True-False] 63.7830μs 32.4270μs 30.8385 KOps/s 31.5473 KOps/s $\color{#d91a1a}-2.25\%$
test_step_mdp_speed[True-False-False-False-True] 65.3430μs 30.4246μs 32.8681 KOps/s 33.9108 KOps/s $\color{#d91a1a}-3.07\%$
test_step_mdp_speed[True-False-False-False-False] 48.3420μs 18.8282μs 53.1118 KOps/s 54.0996 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[False-True-True-True-True] 79.9440μs 49.3008μs 20.2836 KOps/s 20.5639 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-True-True-True-False] 59.0430μs 29.6160μs 33.7655 KOps/s 34.2817 KOps/s $\color{#d91a1a}-1.51\%$
test_step_mdp_speed[False-True-True-False-True] 2.4062ms 32.2017μs 31.0542 KOps/s 32.1103 KOps/s $\color{#d91a1a}-3.29\%$
test_step_mdp_speed[False-True-True-False-False] 45.6620μs 18.0303μs 55.4621 KOps/s 56.6150 KOps/s $\color{#d91a1a}-2.04\%$
test_step_mdp_speed[False-True-False-True-True] 93.9040μs 53.4113μs 18.7226 KOps/s 19.9026 KOps/s $\textbf{\color{#d91a1a}-5.93\%}$
test_step_mdp_speed[False-True-False-True-False] 60.3130μs 32.3575μs 30.9047 KOps/s 31.5442 KOps/s $\color{#d91a1a}-2.03\%$
test_step_mdp_speed[False-True-False-False-True] 68.3330μs 34.1881μs 29.2499 KOps/s 30.2100 KOps/s $\color{#d91a1a}-3.18\%$
test_step_mdp_speed[False-True-False-False-False] 54.4730μs 20.6623μs 48.3973 KOps/s 47.9757 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-True-True-True] 78.2640μs 54.9597μs 18.1952 KOps/s 18.4166 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[False-False-True-True-False] 77.0130μs 35.0879μs 28.4998 KOps/s 29.0853 KOps/s $\color{#d91a1a}-2.01\%$
test_step_mdp_speed[False-False-True-False-True] 0.1084ms 33.3100μs 30.0210 KOps/s 30.3708 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[False-False-True-False-False] 39.3520μs 20.2322μs 49.4262 KOps/s 48.6317 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[False-False-False-True-True] 92.7640μs 57.0160μs 17.5389 KOps/s 17.7868 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[False-False-False-True-False] 69.3140μs 37.1024μs 26.9525 KOps/s 26.9040 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-False-False-False-True] 62.5330μs 35.6436μs 28.0555 KOps/s 28.4075 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-False-False-False-False] 63.5530μs 22.4663μs 44.5112 KOps/s 43.0278 KOps/s $\color{#35bf28}+3.45\%$
test_values[generalized_advantage_estimate-True-True] 9.4508ms 9.1866ms 108.8543 Ops/s 106.7829 Ops/s $\color{#35bf28}+1.94\%$
test_values[vec_generalized_advantage_estimate-True-True] 15.5091ms 11.4312ms 87.4799 Ops/s 88.0062 Ops/s $\color{#d91a1a}-0.60\%$
test_values[td0_return_estimate-False-False] 0.2213ms 0.1239ms 8.0680 KOps/s 8.0719 KOps/s $\color{#d91a1a}-0.05\%$
test_values[td1_return_estimate-False-False] 25.7583ms 25.4311ms 39.3220 Ops/s 39.8042 Ops/s $\color{#d91a1a}-1.21\%$
test_values[vec_td1_return_estimate-False-False] 11.5662ms 11.3858ms 87.8284 Ops/s 88.0544 Ops/s $\color{#d91a1a}-0.26\%$
test_values[td_lambda_return_estimate-True-False] 39.7392ms 38.0150ms 26.3054 Ops/s 26.5140 Ops/s $\color{#d91a1a}-0.79\%$
test_values[vec_td_lambda_return_estimate-True-False] 11.6115ms 11.3475ms 88.1253 Ops/s 87.8725 Ops/s $\color{#35bf28}+0.29\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.1015ms 7.9805ms 125.3051 Ops/s 124.9403 Ops/s $\color{#35bf28}+0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7007ms 1.4840ms 673.8717 Ops/s 662.2536 Ops/s $\color{#35bf28}+1.75\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6043ms 0.3893ms 2.5686 KOps/s 2.5966 KOps/s $\color{#d91a1a}-1.08\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.2127ms 30.3356ms 32.9645 Ops/s 37.5063 Ops/s $\textbf{\color{#d91a1a}-12.11\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8225ms 1.7133ms 583.6637 Ops/s 582.6098 Ops/s $\color{#35bf28}+0.18\%$
test_dqn_speed[False-None] 6.4335ms 1.3652ms 732.4685 Ops/s 737.8344 Ops/s $\color{#d91a1a}-0.73\%$
test_dqn_speed[False-backward] 2.2399ms 1.8487ms 540.9281 Ops/s 549.3426 Ops/s $\color{#d91a1a}-1.53\%$
test_dqn_speed[True-None] 0.8181ms 0.5216ms 1.9173 KOps/s 1.9876 KOps/s $\color{#d91a1a}-3.54\%$
test_dqn_speed[True-backward] 1.0475ms 0.9345ms 1.0701 KOps/s 952.0248 Ops/s $\textbf{\color{#35bf28}+12.40\%}$
test_dqn_speed[reduce-overhead-None] 0.8941ms 0.5064ms 1.9748 KOps/s 1.9326 KOps/s $\color{#35bf28}+2.18\%$
test_dqn_speed[reduce-overhead-backward] 0.9658ms 0.9288ms 1.0767 KOps/s 970.8831 Ops/s $\textbf{\color{#35bf28}+10.90\%}$
test_ddpg_speed[False-None] 3.1912ms 2.8225ms 354.2976 Ops/s 360.1245 Ops/s $\color{#d91a1a}-1.62\%$
test_ddpg_speed[False-backward] 4.1280ms 4.0361ms 247.7669 Ops/s 252.8881 Ops/s $\color{#d91a1a}-2.03\%$
test_ddpg_speed[True-None] 1.5212ms 1.3676ms 731.1983 Ops/s 710.8495 Ops/s $\color{#35bf28}+2.86\%$
test_ddpg_speed[True-backward] 2.5011ms 2.2881ms 437.0371 Ops/s 436.9000 Ops/s $\color{#35bf28}+0.03\%$
test_ddpg_speed[reduce-overhead-None] 1.5639ms 1.3527ms 739.2565 Ops/s 745.4788 Ops/s $\color{#d91a1a}-0.83\%$
test_ddpg_speed[reduce-overhead-backward] 2.3516ms 2.2677ms 440.9840 Ops/s 362.1864 Ops/s $\textbf{\color{#35bf28}+21.76\%}$
test_sac_speed[False-None] 8.3246ms 7.6598ms 130.5523 Ops/s 127.9307 Ops/s $\color{#35bf28}+2.05\%$
test_sac_speed[False-backward] 11.5698ms 10.9926ms 90.9700 Ops/s 92.5903 Ops/s $\color{#d91a1a}-1.75\%$
test_sac_speed[True-None] 2.1740ms 2.0712ms 482.8105 Ops/s 468.2916 Ops/s $\color{#35bf28}+3.10\%$
test_sac_speed[True-backward] 3.9798ms 3.8923ms 256.9195 Ops/s 225.3422 Ops/s $\textbf{\color{#35bf28}+14.01\%}$
test_sac_speed[reduce-overhead-None] 2.2320ms 2.0856ms 479.4801 Ops/s 467.1931 Ops/s $\color{#35bf28}+2.63\%$
test_sac_speed[reduce-overhead-backward] 4.1730ms 3.9824ms 251.1064 Ops/s 249.0141 Ops/s $\color{#35bf28}+0.84\%$
test_redq_speed[False-None] 11.8319ms 10.1194ms 98.8200 Ops/s 101.5293 Ops/s $\color{#d91a1a}-2.67\%$
test_redq_speed[False-backward] 19.0630ms 17.4499ms 57.3070 Ops/s 57.9490 Ops/s $\color{#d91a1a}-1.11\%$
test_redq_speed[True-None] 4.5940ms 4.3064ms 232.2120 Ops/s 230.8762 Ops/s $\color{#35bf28}+0.58\%$
test_redq_speed[True-backward] 10.1508ms 9.8247ms 101.7847 Ops/s 100.4200 Ops/s $\color{#35bf28}+1.36\%$
test_redq_speed[reduce-overhead-None] 4.4745ms 4.2665ms 234.3842 Ops/s 228.5513 Ops/s $\color{#35bf28}+2.55\%$
test_redq_speed[reduce-overhead-backward] 10.1718ms 9.8730ms 101.2860 Ops/s 100.3017 Ops/s $\color{#35bf28}+0.98\%$
test_redq_deprec_speed[False-None] 11.2852ms 10.7107ms 93.3649 Ops/s 92.8227 Ops/s $\color{#35bf28}+0.58\%$
test_redq_deprec_speed[False-backward] 16.3208ms 15.6927ms 63.7239 Ops/s 63.7177 Ops/s $+0.01\%$
test_redq_deprec_speed[True-None] 4.0737ms 3.5714ms 279.9992 Ops/s 277.3742 Ops/s $\color{#35bf28}+0.95\%$
test_redq_deprec_speed[True-backward] 7.7687ms 7.5310ms 132.7852 Ops/s 132.8085 Ops/s $\color{#d91a1a}-0.02\%$
test_redq_deprec_speed[reduce-overhead-None] 3.6950ms 3.5267ms 283.5477 Ops/s 274.1440 Ops/s $\color{#35bf28}+3.43\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.5981ms 7.3640ms 135.7961 Ops/s 126.9487 Ops/s $\textbf{\color{#35bf28}+6.97\%}$
test_td3_speed[False-None] 7.8697ms 7.6709ms 130.3623 Ops/s 131.4757 Ops/s $\color{#d91a1a}-0.85\%$
test_td3_speed[False-backward] 11.0246ms 10.4364ms 95.8188 Ops/s 96.4666 Ops/s $\color{#d91a1a}-0.67\%$
test_td3_speed[True-None] 2.0446ms 1.7612ms 567.8076 Ops/s 567.6813 Ops/s $\color{#35bf28}+0.02\%$
test_td3_speed[True-backward] 3.8001ms 3.5053ms 285.2846 Ops/s 283.4914 Ops/s $\color{#35bf28}+0.63\%$
test_td3_speed[reduce-overhead-None] 1.7655ms 1.7104ms 584.6453 Ops/s 586.5625 Ops/s $\color{#d91a1a}-0.33\%$
test_td3_speed[reduce-overhead-backward] 3.6534ms 3.5336ms 282.9969 Ops/s 281.6043 Ops/s $\color{#35bf28}+0.49\%$
test_cql_speed[False-None] 28.7511ms 25.4369ms 39.3129 Ops/s 40.0919 Ops/s $\color{#d91a1a}-1.94\%$
test_cql_speed[False-backward] 37.4224ms 34.5750ms 28.9227 Ops/s 29.2823 Ops/s $\color{#d91a1a}-1.23\%$
test_cql_speed[True-None] 12.8198ms 12.3611ms 80.8992 Ops/s 81.9483 Ops/s $\color{#d91a1a}-1.28\%$
test_cql_speed[True-backward] 18.6683ms 18.1361ms 55.1385 Ops/s 56.2841 Ops/s $\color{#d91a1a}-2.04\%$
test_cql_speed[reduce-overhead-None] 15.2750ms 12.5010ms 79.9937 Ops/s 81.2635 Ops/s $\color{#d91a1a}-1.56\%$
test_cql_speed[reduce-overhead-backward] 19.0101ms 18.5561ms 53.8908 Ops/s 55.7193 Ops/s $\color{#d91a1a}-3.28\%$
test_a2c_speed[False-None] 5.6132ms 5.2965ms 188.8049 Ops/s 188.9460 Ops/s $\color{#d91a1a}-0.07\%$
test_a2c_speed[False-backward] 12.0704ms 11.6953ms 85.5044 Ops/s 85.1087 Ops/s $\color{#35bf28}+0.46\%$
test_a2c_speed[True-None] 4.0128ms 3.6464ms 274.2454 Ops/s 270.1691 Ops/s $\color{#35bf28}+1.51\%$
test_a2c_speed[True-backward] 8.9254ms 8.6143ms 116.0855 Ops/s 115.3982 Ops/s $\color{#35bf28}+0.60\%$
test_a2c_speed[reduce-overhead-None] 4.4677ms 3.7662ms 265.5178 Ops/s 274.3866 Ops/s $\color{#d91a1a}-3.23\%$
test_a2c_speed[reduce-overhead-backward] 8.9860ms 8.7068ms 114.8527 Ops/s 102.8590 Ops/s $\textbf{\color{#35bf28}+11.66\%}$
test_ppo_speed[False-None] 6.1696ms 5.6986ms 175.4818 Ops/s 172.2234 Ops/s $\color{#35bf28}+1.89\%$
test_ppo_speed[False-backward] 12.6896ms 12.2214ms 81.8235 Ops/s 81.7149 Ops/s $\color{#35bf28}+0.13\%$
test_ppo_speed[True-None] 3.9872ms 3.6112ms 276.9166 Ops/s 274.0606 Ops/s $\color{#35bf28}+1.04\%$
test_ppo_speed[True-backward] 8.6132ms 8.2953ms 120.5505 Ops/s 118.7033 Ops/s $\color{#35bf28}+1.56\%$
test_ppo_speed[reduce-overhead-None] 3.7219ms 3.5758ms 279.6578 Ops/s 277.6805 Ops/s $\color{#35bf28}+0.71\%$
test_ppo_speed[reduce-overhead-backward] 8.7718ms 8.5645ms 116.7612 Ops/s 92.1879 Ops/s $\textbf{\color{#35bf28}+26.66\%}$
test_reinforce_speed[False-None] 4.8932ms 4.4575ms 224.3397 Ops/s 201.2710 Ops/s $\textbf{\color{#35bf28}+11.46\%}$
test_reinforce_speed[False-backward] 7.8322ms 7.2633ms 137.6791 Ops/s 136.5431 Ops/s $\color{#35bf28}+0.83\%$
test_reinforce_speed[True-None] 3.1324ms 2.8170ms 354.9927 Ops/s 329.2741 Ops/s $\textbf{\color{#35bf28}+7.81\%}$
test_reinforce_speed[True-backward] 7.9201ms 7.5118ms 133.1241 Ops/s 129.9449 Ops/s $\color{#35bf28}+2.45\%$
test_reinforce_speed[reduce-overhead-None] 2.9983ms 2.7759ms 360.2496 Ops/s 347.7989 Ops/s $\color{#35bf28}+3.58\%$
test_reinforce_speed[reduce-overhead-backward] 8.1069ms 7.8520ms 127.3560 Ops/s 110.6826 Ops/s $\textbf{\color{#35bf28}+15.06\%}$
test_iql_speed[False-None] 25.5050ms 20.1034ms 49.7429 Ops/s 48.8321 Ops/s $\color{#35bf28}+1.87\%$
test_iql_speed[False-backward] 36.6930ms 30.0398ms 33.2892 Ops/s 33.0539 Ops/s $\color{#35bf28}+0.71\%$
test_iql_speed[True-None] 8.6784ms 8.3167ms 120.2403 Ops/s 118.5301 Ops/s $\color{#35bf28}+1.44\%$
test_iql_speed[True-backward] 16.6961ms 16.3046ms 61.3324 Ops/s 64.7594 Ops/s $\textbf{\color{#d91a1a}-5.29\%}$
test_iql_speed[reduce-overhead-None] 8.7582ms 8.3878ms 119.2203 Ops/s 127.6073 Ops/s $\textbf{\color{#d91a1a}-6.57\%}$
test_iql_speed[reduce-overhead-backward] 17.2016ms 16.8308ms 59.4149 Ops/s 63.9054 Ops/s $\textbf{\color{#d91a1a}-7.03\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.2900ms 5.8532ms 170.8454 Ops/s 171.4491 Ops/s $\color{#d91a1a}-0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6107ms 0.3243ms 3.0836 KOps/s 2.7939 KOps/s $\textbf{\color{#35bf28}+10.37\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6202ms 0.3146ms 3.1788 KOps/s 2.9053 KOps/s $\textbf{\color{#35bf28}+9.41\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8435ms 5.6134ms 178.1465 Ops/s 180.5011 Ops/s $\color{#d91a1a}-1.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7369ms 0.3241ms 3.0857 KOps/s 3.1222 KOps/s $\color{#d91a1a}-1.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5948ms 0.3157ms 3.1675 KOps/s 3.9800 KOps/s $\textbf{\color{#d91a1a}-20.42\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6346ms 1.3428ms 744.7342 Ops/s 749.5390 Ops/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4893ms 1.2586ms 794.5555 Ops/s 806.0378 Ops/s $\color{#d91a1a}-1.42\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7976ms 5.7125ms 175.0561 Ops/s 174.6747 Ops/s $\color{#35bf28}+0.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0689ms 0.5047ms 1.9812 KOps/s 2.1543 KOps/s $\textbf{\color{#d91a1a}-8.03\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7770ms 0.4870ms 2.0532 KOps/s 2.3524 KOps/s $\textbf{\color{#d91a1a}-12.72\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8348ms 5.6472ms 177.0787 Ops/s 177.4546 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0354ms 0.3393ms 2.9468 KOps/s 3.5869 KOps/s $\textbf{\color{#d91a1a}-17.84\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7681ms 0.2895ms 3.4539 KOps/s 2.7973 KOps/s $\textbf{\color{#35bf28}+23.47\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8780ms 5.5834ms 179.1014 Ops/s 180.9936 Ops/s $\color{#d91a1a}-1.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8418ms 0.3173ms 3.1516 KOps/s 3.1224 KOps/s $\color{#35bf28}+0.94\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4892ms 0.3158ms 3.1662 KOps/s 3.2261 KOps/s $\color{#d91a1a}-1.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.9440ms 5.8678ms 170.4212 Ops/s 174.4470 Ops/s $\color{#d91a1a}-2.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0764ms 0.4812ms 2.0780 KOps/s 2.0504 KOps/s $\color{#35bf28}+1.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6964ms 0.4690ms 2.1323 KOps/s 2.1307 KOps/s $\color{#35bf28}+0.08\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4757s 14.5604ms 68.6794 Ops/s 198.0474 Ops/s $\textbf{\color{#d91a1a}-65.32\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 11.4385ms 2.1028ms 475.5561 Ops/s 491.9337 Ops/s $\color{#d91a1a}-3.33\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2983ms 1.0758ms 929.5669 Ops/s 838.0963 Ops/s $\textbf{\color{#35bf28}+10.91\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 8.8520ms 5.1716ms 193.3630 Ops/s 197.3285 Ops/s $\color{#d91a1a}-2.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 6.7624ms 1.9983ms 500.4273 Ops/s 482.5804 Ops/s $\color{#35bf28}+3.70\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 9.1901ms 1.2750ms 784.3177 Ops/s 848.8836 Ops/s $\textbf{\color{#d91a1a}-7.61\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4325s 13.9350ms 71.7620 Ops/s 58.0320 Ops/s $\textbf{\color{#35bf28}+23.66\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.0475ms 2.1331ms 468.8042 Ops/s 516.7848 Ops/s $\textbf{\color{#d91a1a}-9.28\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.1169ms 0.9910ms 1.0091 KOps/s 1.0242 KOps/s $\color{#d91a1a}-1.47\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.6179ms 32.5930ms 30.6814 Ops/s 30.7751 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.0267ms 17.0948ms 58.4975 Ops/s 59.0858 Ops/s $\color{#d91a1a}-1.00\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.9171ms 33.5358ms 29.8189 Ops/s 29.3853 Ops/s $\color{#35bf28}+1.48\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 18.8992ms 17.0860ms 58.5276 Ops/s 60.3226 Ops/s $\color{#d91a1a}-2.98\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.4229ms 35.0263ms 28.5500 Ops/s 27.8191 Ops/s $\color{#35bf28}+2.63\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 0.3790s 25.8765ms 38.6451 Ops/s 54.6747 Ops/s $\textbf{\color{#d91a1a}-29.32\%}$

@vmoens vmoens added the Refactoring Refactoring of an existing feature label Nov 5, 2025
@vmoens
Update
3cf1674 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 5, 2025
@vmoens
[Refactor] Weight sync schemes refactor
bb3fb15 
ghstack-source-id: 902cff2
Pull-Request: #3230
@vmoens
Update
c858a0a 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 5, 2025
@vmoens
[Refactor] Weight sync schemes refactor
5375557 
ghstack-source-id: 36dbdc9
Pull-Request: #3230
@vmoens
Update
3852c9f 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 6, 2025
@vmoens
[Refactor] Weight sync schemes refactor
3ac1ee0 
ghstack-source-id: ec75dca
Pull-Request: #3230
vmoens added a commit that referenced this pull request Nov 6, 2025
@vmoens
[Refactor] Weight sync schemes refactor
bc08c62 
ghstack-source-id: ec75dca
Pull-Request: #3230
@vmoens
Update
00baa00 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 6, 2025
@vmoens
[Refactor] Weight sync schemes refactor
41e0c22 
ghstack-source-id: bbae696
Pull-Request: #3230
@vmoens
Update
44cfcca 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 6, 2025
@vmoens
[Refactor] Weight sync schemes refactor
60bc9f0 
ghstack-source-id: e89633e
Pull-Request: #3230
@vmoens vmoens mentioned this pull request Nov 6, 2025
Merged
vmoens added 2 commits November 6, 2025 14:09
@vmoens
Update
a044032 
[ghstack-poisoned]
@vmoens
Update
5607984 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Nov 6, 2025
Merged
vmoens added 2 commits November 6, 2025 17:29
@vmoens
Update
5684d08 
[ghstack-poisoned]
@vmoens
Update
fe5d877 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 6, 2025
@vmoens
[Refactor] Weight sync schemes refactor
ae0ae06 
ghstack-source-id: 3b9e105
Pull-Request: #3230
@vmoens vmoens merged commit fe5d877 into gh/vmoens/172/base Nov 6, 2025
50 of 61 checks passed
@vmoens vmoens deleted the gh/vmoens/172/head branch November 6, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Refactoring Refactoring of an existing feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens