[Feature] Trainer Algorithms - Configuration System

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] Documentation #3192
• [Feature] Add timing options #3191
• [Feature] PPO Trainer Updates #3190
• -> [Feature] Trainer Algorithms - Configuration System #3189
• [Feature] Trainer Infrastructure - Timing and Utilities #3188

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3189

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures, 1 Unrelated Failure

As of commit cb06bae with merge base 01d2801 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 31edf6400008fbd0f8a4523a7392e168619b7d285f9633e22c37c12c62dbe7f1 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 96379005109a58d7c6d0eac1ce5fb305b7b4348018498559a213b46f2fe9076c /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 5686267adff0ad755c3c6ac074f62e4b2b38e61ebb4f4c2fd72bd7d620ad3fd3 /exec failed with exit code 1
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t b873f90c5c8450aa70ebb76b5373262f19381d3bd960c872aacd172bdc4958a3 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_configs.py::TestHydraParsing::test_trainer_parsing_with_file
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_configs.py::TestHydraParsing::test_trainer_parsing_with_file
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  test/test_configs.py::TestHydraParsing::test_trainer_parsing_with_file
• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh)
  test/test_configs.py::TestHydraParsing::test_trainer_parsing_with_file
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  test/test_configs.py::TestHydraParsing::test_trainer_parsing_with_file

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh) (trunk failure)
  test/test_collector.py::TestPolicyFactory::test_weight_update

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}17$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	83.6661μs	82.0235μs	12.1916 KOps/s	11.7599 KOps/s	$\color{#35bf28}+3.67\%$
test_tensor_to_bytestream_speed[torch.save]	0.1430ms	0.1417ms	7.0558 KOps/s	7.0380 KOps/s	$\color{#35bf28}+0.25\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1203s	0.1203s	8.3151 Ops/s	8.3523 Ops/s	$\color{#d91a1a}-0.45\%$
test_tensor_to_bytestream_speed[numpy]	2.8301μs	2.8229μs	354.2431 KOps/s	355.3032 KOps/s	$\color{#d91a1a}-0.30\%$
test_tensor_to_bytestream_speed[safetensors]	44.1213μs	43.6230μs	22.9237 KOps/s	24.0510 KOps/s	$\color{#d91a1a}-4.69\%$
test_simple	0.5536s	0.5519s	1.8120 Ops/s	1.7256 Ops/s	$\textbf{\color{#35bf28}+5.01\%}$
test_transformed	1.1179s	1.1161s	0.8960 Ops/s	0.8761 Ops/s	$\color{#35bf28}+2.27\%$
test_serial	1.6736s	1.6714s	0.5983 Ops/s	0.5885 Ops/s	$\color{#35bf28}+1.66\%$
test_parallel	1.1709s	1.0938s	0.9143 Ops/s	0.9221 Ops/s	$\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-True-True-True-True]	0.2090ms	45.4293μs	22.0122 KOps/s	21.7253 KOps/s	$\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-True-True-False]	56.7210μs	25.6223μs	39.0285 KOps/s	40.0427 KOps/s	$\color{#d91a1a}-2.53\%$
test_step_mdp_speed[True-True-True-False-True]	0.1342ms	25.4463μs	39.2984 KOps/s	38.9034 KOps/s	$\color{#35bf28}+1.02\%$
test_step_mdp_speed[True-True-True-False-False]	60.3710μs	14.0202μs	71.3255 KOps/s	71.5120 KOps/s	$\color{#d91a1a}-0.26\%$
test_step_mdp_speed[True-True-False-True-True]	83.2620μs	48.2360μs	20.7314 KOps/s	20.4387 KOps/s	$\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-True-False-True-False]	56.6210μs	28.4319μs	35.1718 KOps/s	35.8514 KOps/s	$\color{#d91a1a}-1.90\%$
test_step_mdp_speed[True-True-False-False-True]	58.3510μs	28.2411μs	35.4093 KOps/s	34.7978 KOps/s	$\color{#35bf28}+1.76\%$
test_step_mdp_speed[True-True-False-False-False]	48.2310μs	16.9712μs	58.9235 KOps/s	59.3212 KOps/s	$\color{#d91a1a}-0.67\%$
test_step_mdp_speed[True-False-True-True-True]	88.9510μs	51.2296μs	19.5200 KOps/s	19.5209 KOps/s	$-0.00\%$
test_step_mdp_speed[True-False-True-True-False]	78.8620μs	31.1999μs	32.0514 KOps/s	32.4855 KOps/s	$\color{#d91a1a}-1.34\%$
test_step_mdp_speed[True-False-True-False-True]	77.5010μs	28.2256μs	35.4288 KOps/s	35.7055 KOps/s	$\color{#d91a1a}-0.77\%$
test_step_mdp_speed[True-False-True-False-False]	71.8310μs	16.3891μs	61.0163 KOps/s	59.7661 KOps/s	$\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-False-False-True-True]	85.6310μs	52.8718μs	18.9137 KOps/s	18.5532 KOps/s	$\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-False-False-True-False]	83.8120μs	33.9659μs	29.4413 KOps/s	30.0258 KOps/s	$\color{#d91a1a}-1.95\%$
test_step_mdp_speed[True-False-False-False-True]	54.1910μs	30.7973μs	32.4704 KOps/s	32.1538 KOps/s	$\color{#35bf28}+0.98\%$
test_step_mdp_speed[True-False-False-False-False]	48.8010μs	19.7565μs	50.6163 KOps/s	51.4326 KOps/s	$\color{#d91a1a}-1.59\%$
test_step_mdp_speed[False-True-True-True-True]	87.5620μs	50.3634μs	19.8557 KOps/s	19.3435 KOps/s	$\color{#35bf28}+2.65\%$
test_step_mdp_speed[False-True-True-True-False]	57.5310μs	31.4455μs	31.8011 KOps/s	32.3951 KOps/s	$\color{#d91a1a}-1.83\%$
test_step_mdp_speed[False-True-True-False-True]	2.4054ms	32.5309μs	30.7400 KOps/s	30.8004 KOps/s	$\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-True-True-False-False]	47.3010μs	18.7169μs	53.4278 KOps/s	53.9433 KOps/s	$\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-True-False-True-True]	94.7710μs	54.4442μs	18.3674 KOps/s	18.5798 KOps/s	$\color{#d91a1a}-1.14\%$
test_step_mdp_speed[False-True-False-True-False]	0.1078ms	33.6447μs	29.7224 KOps/s	30.0243 KOps/s	$\color{#d91a1a}-1.01\%$
test_step_mdp_speed[False-True-False-False-True]	69.0610μs	35.4230μs	28.2303 KOps/s	28.6106 KOps/s	$\color{#d91a1a}-1.33\%$
test_step_mdp_speed[False-True-False-False-False]	58.7810μs	21.3956μs	46.7386 KOps/s	46.9066 KOps/s	$\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-False-True-True-True]	93.4820μs	57.4623μs	17.4027 KOps/s	17.7100 KOps/s	$\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-False-True-True-False]	82.5820μs	36.7180μs	27.2346 KOps/s	27.3496 KOps/s	$\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-False-True-False-True]	97.5920μs	35.1613μs	28.4404 KOps/s	28.5636 KOps/s	$\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-False-True-False-False]	50.9310μs	21.5344μs	46.4374 KOps/s	47.1358 KOps/s	$\color{#d91a1a}-1.48\%$
test_step_mdp_speed[False-False-False-True-True]	0.1182ms	58.9630μs	16.9598 KOps/s	17.1383 KOps/s	$\color{#d91a1a}-1.04\%$
test_step_mdp_speed[False-False-False-True-False]	79.9910μs	39.4906μs	25.3225 KOps/s	26.4166 KOps/s	$\color{#d91a1a}-4.14\%$
test_step_mdp_speed[False-False-False-False-True]	71.4010μs	36.9519μs	27.0622 KOps/s	27.0869 KOps/s	$\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-False-False-False-False]	64.8610μs	23.7270μs	42.1460 KOps/s	41.9453 KOps/s	$\color{#35bf28}+0.48\%$
test_values[generalized_advantage_estimate-True-True]	10.5332ms	10.1911ms	98.1252 Ops/s	99.6446 Ops/s	$\color{#d91a1a}-1.52\%$
test_values[vec_generalized_advantage_estimate-True-True]	21.7005ms	17.8138ms	56.1362 Ops/s	89.6367 Ops/s	$\textbf{\color{#d91a1a}-37.37\%}$
test_values[td0_return_estimate-False-False]	0.2359ms	0.1231ms	8.1211 KOps/s	7.8863 KOps/s	$\color{#35bf28}+2.98\%$
test_values[td1_return_estimate-False-False]	27.8750ms	27.6336ms	36.1879 Ops/s	36.4501 Ops/s	$\color{#d91a1a}-0.72\%$
test_values[vec_td1_return_estimate-False-False]	21.1332ms	17.8414ms	56.0493 Ops/s	89.3139 Ops/s	$\textbf{\color{#d91a1a}-37.24\%}$
test_values[td_lambda_return_estimate-True-False]	44.3551ms	41.2578ms	24.2378 Ops/s	24.3997 Ops/s	$\color{#d91a1a}-0.66\%$
test_values[vec_td_lambda_return_estimate-True-False]	22.2544ms	17.7938ms	56.1994 Ops/s	89.3663 Ops/s	$\textbf{\color{#d91a1a}-37.11\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.8587ms	8.7502ms	114.2833 Ops/s	115.0098 Ops/s	$\color{#d91a1a}-0.63\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.7683ms	1.5363ms	650.9040 Ops/s	635.1343 Ops/s	$\color{#35bf28}+2.48\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5355ms	0.4197ms	2.3828 KOps/s	2.3800 KOps/s	$\color{#35bf28}+0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.8144ms	34.3915ms	29.0770 Ops/s	33.2514 Ops/s	$\textbf{\color{#d91a1a}-12.55\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.7768ms	1.7063ms	586.0640 Ops/s	580.8666 Ops/s	$\color{#35bf28}+0.89\%$
test_dqn_speed[False-None]	6.4115ms	1.4384ms	695.2043 Ops/s	699.2334 Ops/s	$\color{#d91a1a}-0.58\%$
test_dqn_speed[False-backward]	2.0712ms	1.9665ms	508.5185 Ops/s	519.2588 Ops/s	$\color{#d91a1a}-2.07\%$
test_dqn_speed[True-None]	0.6652ms	0.5193ms	1.9258 KOps/s	1.9052 KOps/s	$\color{#35bf28}+1.08\%$
test_dqn_speed[True-backward]	0.9963ms	0.9530ms	1.0494 KOps/s	938.4899 Ops/s	$\textbf{\color{#35bf28}+11.81\%}$
test_dqn_speed[reduce-overhead-None]	0.8593ms	0.4995ms	2.0021 KOps/s	1.9301 KOps/s	$\color{#35bf28}+3.73\%$
test_dqn_speed[reduce-overhead-backward]	1.0031ms	0.9469ms	1.0561 KOps/s	944.7850 Ops/s	$\textbf{\color{#35bf28}+11.78\%}$
test_ddpg_speed[False-None]	3.2746ms	2.8868ms	346.4050 Ops/s	344.3440 Ops/s	$\color{#35bf28}+0.60\%$
test_ddpg_speed[False-backward]	4.2017ms	4.0877ms	244.6375 Ops/s	242.3520 Ops/s	$\color{#35bf28}+0.94\%$
test_ddpg_speed[True-None]	1.7608ms	1.3655ms	732.3514 Ops/s	708.2174 Ops/s	$\color{#35bf28}+3.41\%$
test_ddpg_speed[True-backward]	2.4467ms	2.3447ms	426.4868 Ops/s	421.6921 Ops/s	$\color{#35bf28}+1.14\%$
test_ddpg_speed[reduce-overhead-None]	1.5590ms	1.3642ms	733.0414 Ops/s	735.1188 Ops/s	$\color{#d91a1a}-0.28\%$
test_ddpg_speed[reduce-overhead-backward]	2.3933ms	2.3396ms	427.4293 Ops/s	413.3934 Ops/s	$\color{#35bf28}+3.40\%$
test_sac_speed[False-None]	8.7207ms	7.9630ms	125.5804 Ops/s	127.1048 Ops/s	$\color{#d91a1a}-1.20\%$
test_sac_speed[False-backward]	11.5685ms	11.1950ms	89.3255 Ops/s	90.1351 Ops/s	$\color{#d91a1a}-0.90\%$
test_sac_speed[True-None]	2.3634ms	2.0906ms	478.3349 Ops/s	478.4385 Ops/s	$\color{#d91a1a}-0.02\%$
test_sac_speed[True-backward]	4.1258ms	3.9919ms	250.5074 Ops/s	220.7974 Ops/s	$\textbf{\color{#35bf28}+13.46\%}$
test_sac_speed[reduce-overhead-None]	2.4498ms	2.0983ms	476.5659 Ops/s	473.6215 Ops/s	$\color{#35bf28}+0.62\%$
test_sac_speed[reduce-overhead-backward]	4.1210ms	3.9607ms	252.4808 Ops/s	246.9456 Ops/s	$\color{#35bf28}+2.24\%$
test_redq_speed[False-None]	10.6732ms	10.1465ms	98.5562 Ops/s	96.5832 Ops/s	$\color{#35bf28}+2.04\%$
test_redq_speed[False-backward]	23.8068ms	18.1884ms	54.9801 Ops/s	57.0983 Ops/s	$\color{#d91a1a}-3.71\%$
test_redq_speed[True-None]	4.5544ms	4.3705ms	228.8093 Ops/s	219.3559 Ops/s	$\color{#35bf28}+4.31\%$
test_redq_speed[True-backward]	9.9520ms	9.6834ms	103.2692 Ops/s	97.5863 Ops/s	$\textbf{\color{#35bf28}+5.82\%}$
test_redq_speed[reduce-overhead-None]	4.5707ms	4.3565ms	229.5415 Ops/s	236.6195 Ops/s	$\color{#d91a1a}-2.99\%$
test_redq_speed[reduce-overhead-backward]	10.1304ms	9.8689ms	101.3284 Ops/s	101.0240 Ops/s	$\color{#35bf28}+0.30\%$
test_redq_deprec_speed[False-None]	11.6033ms	10.9736ms	91.1277 Ops/s	92.1847 Ops/s	$\color{#d91a1a}-1.15\%$
test_redq_deprec_speed[False-backward]	16.2681ms	15.8490ms	63.0955 Ops/s	64.5157 Ops/s	$\color{#d91a1a}-2.20\%$
test_redq_deprec_speed[True-None]	3.8631ms	3.6125ms	276.8165 Ops/s	285.5301 Ops/s	$\color{#d91a1a}-3.05\%$
test_redq_deprec_speed[True-backward]	7.8419ms	7.5584ms	132.3029 Ops/s	137.1557 Ops/s	$\color{#d91a1a}-3.54\%$
test_redq_deprec_speed[reduce-overhead-None]	3.8433ms	3.5683ms	280.2422 Ops/s	263.0003 Ops/s	$\textbf{\color{#35bf28}+6.56\%}$
test_redq_deprec_speed[reduce-overhead-backward]	7.7581ms	7.5178ms	133.0170 Ops/s	124.7557 Ops/s	$\textbf{\color{#35bf28}+6.62\%}$
test_td3_speed[False-None]	8.5886ms	7.9378ms	125.9787 Ops/s	119.4052 Ops/s	$\textbf{\color{#35bf28}+5.51\%}$
test_td3_speed[False-backward]	11.3207ms	10.8081ms	92.5236 Ops/s	92.0584 Ops/s	$\color{#35bf28}+0.51\%$
test_td3_speed[True-None]	1.8030ms	1.7651ms	566.5450 Ops/s	564.1305 Ops/s	$\color{#35bf28}+0.43\%$
test_td3_speed[True-backward]	3.7022ms	3.5638ms	280.5969 Ops/s	265.8923 Ops/s	$\textbf{\color{#35bf28}+5.53\%}$
test_td3_speed[reduce-overhead-None]	1.7790ms	1.7314ms	577.5538 Ops/s	557.9375 Ops/s	$\color{#35bf28}+3.52\%$
test_td3_speed[reduce-overhead-backward]	3.6556ms	3.5559ms	281.2266 Ops/s	277.9119 Ops/s	$\color{#35bf28}+1.19\%$
test_cql_speed[False-None]	26.4888ms	25.5934ms	39.0726 Ops/s	38.7897 Ops/s	$\color{#35bf28}+0.73\%$
test_cql_speed[False-backward]	36.1244ms	34.8606ms	28.6857 Ops/s	28.6151 Ops/s	$\color{#35bf28}+0.25\%$
test_cql_speed[True-None]	12.5421ms	12.1660ms	82.1966 Ops/s	79.9898 Ops/s	$\color{#35bf28}+2.76\%$
test_cql_speed[True-backward]	18.4931ms	17.8355ms	56.0678 Ops/s	54.7408 Ops/s	$\color{#35bf28}+2.42\%$
test_cql_speed[reduce-overhead-None]	12.4044ms	12.1758ms	82.1304 Ops/s	80.3904 Ops/s	$\color{#35bf28}+2.16\%$
test_cql_speed[reduce-overhead-backward]	18.1880ms	17.8526ms	56.0142 Ops/s	55.6091 Ops/s	$\color{#35bf28}+0.73\%$
test_a2c_speed[False-None]	5.9146ms	5.4293ms	184.1841 Ops/s	182.8274 Ops/s	$\color{#35bf28}+0.74\%$
test_a2c_speed[False-backward]	12.1685ms	11.8407ms	84.4548 Ops/s	82.9214 Ops/s	$\color{#35bf28}+1.85\%$
test_a2c_speed[True-None]	4.0000ms	3.6321ms	275.3233 Ops/s	266.9970 Ops/s	$\color{#35bf28}+3.12\%$
test_a2c_speed[True-backward]	8.8231ms	8.5911ms	116.3993 Ops/s	115.3447 Ops/s	$\color{#35bf28}+0.91\%$
test_a2c_speed[reduce-overhead-None]	4.0787ms	3.7122ms	269.3803 Ops/s	267.4727 Ops/s	$\color{#35bf28}+0.71\%$
test_a2c_speed[reduce-overhead-backward]	8.9857ms	8.7149ms	114.7462 Ops/s	109.7521 Ops/s	$\color{#35bf28}+4.55\%$
test_ppo_speed[False-None]	6.3978ms	5.9636ms	167.6827 Ops/s	172.6565 Ops/s	$\color{#d91a1a}-2.88\%$
test_ppo_speed[False-backward]	12.7388ms	12.4795ms	80.1312 Ops/s	81.8708 Ops/s	$\color{#d91a1a}-2.12\%$
test_ppo_speed[True-None]	3.7863ms	3.6373ms	274.9272 Ops/s	270.5934 Ops/s	$\color{#35bf28}+1.60\%$
test_ppo_speed[True-backward]	8.6337ms	8.3826ms	119.2954 Ops/s	118.9380 Ops/s	$\color{#35bf28}+0.30\%$
test_ppo_speed[reduce-overhead-None]	4.1081ms	3.6274ms	275.6820 Ops/s	270.5501 Ops/s	$\color{#35bf28}+1.90\%$
test_ppo_speed[reduce-overhead-backward]	9.0666ms	8.6630ms	115.4338 Ops/s	103.0230 Ops/s	$\textbf{\color{#35bf28}+12.05\%}$
test_reinforce_speed[False-None]	5.0133ms	4.6311ms	215.9322 Ops/s	221.4416 Ops/s	$\color{#d91a1a}-2.49\%$
test_reinforce_speed[False-backward]	7.7148ms	7.4423ms	134.3679 Ops/s	137.4669 Ops/s	$\color{#d91a1a}-2.25\%$
test_reinforce_speed[True-None]	2.9898ms	2.8486ms	351.0456 Ops/s	344.5197 Ops/s	$\color{#35bf28}+1.89\%$
test_reinforce_speed[True-backward]	8.1165ms	7.6950ms	129.9549 Ops/s	121.4411 Ops/s	$\textbf{\color{#35bf28}+7.01\%}$
test_reinforce_speed[reduce-overhead-None]	3.2380ms	2.8554ms	350.2101 Ops/s	350.8540 Ops/s	$\color{#d91a1a}-0.18\%$
test_reinforce_speed[reduce-overhead-backward]	8.2094ms	7.7777ms	128.5727 Ops/s	119.8046 Ops/s	$\textbf{\color{#35bf28}+7.32\%}$
test_iql_speed[False-None]	24.8539ms	19.9444ms	50.1393 Ops/s	49.0891 Ops/s	$\color{#35bf28}+2.14\%$
test_iql_speed[False-backward]	37.2588ms	30.5983ms	32.6815 Ops/s	32.4270 Ops/s	$\color{#35bf28}+0.78\%$
test_iql_speed[True-None]	8.7655ms	8.4364ms	118.5343 Ops/s	115.9742 Ops/s	$\color{#35bf28}+2.21\%$
test_iql_speed[True-backward]	16.8087ms	16.4975ms	60.6153 Ops/s	57.8510 Ops/s	$\color{#35bf28}+4.78\%$
test_iql_speed[reduce-overhead-None]	11.2111ms	8.6145ms	116.0833 Ops/s	115.3735 Ops/s	$\color{#35bf28}+0.62\%$
test_iql_speed[reduce-overhead-backward]	17.9359ms	17.1971ms	58.1494 Ops/s	59.1453 Ops/s	$\color{#d91a1a}-1.68\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5507ms	6.0221ms	166.0556 Ops/s	165.8285 Ops/s	$\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5282ms	0.2902ms	3.4454 KOps/s	3.5908 KOps/s	$\color{#d91a1a}-4.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7014ms	0.2617ms	3.8215 KOps/s	3.8848 KOps/s	$\color{#d91a1a}-1.63\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.0033ms	5.7364ms	174.3254 Ops/s	174.5898 Ops/s	$\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5721s	0.6458ms	1.5484 KOps/s	3.7065 KOps/s	$\textbf{\color{#d91a1a}-58.22\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4617ms	0.2561ms	3.9050 KOps/s	3.9602 KOps/s	$\color{#d91a1a}-1.39\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.4385ms	1.2472ms	801.8190 Ops/s	806.1337 Ops/s	$\color{#d91a1a}-0.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.4991ms	1.1658ms	857.7461 Ops/s	859.0985 Ops/s	$\color{#d91a1a}-0.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.1628ms	5.9199ms	168.9207 Ops/s	170.7167 Ops/s	$\color{#d91a1a}-1.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0483ms	0.4594ms	2.1765 KOps/s	2.1131 KOps/s	$\color{#35bf28}+3.00\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.5965ms	0.4057ms	2.4650 KOps/s	2.2315 KOps/s	$\textbf{\color{#35bf28}+10.47\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8685ms	5.7577ms	173.6815 Ops/s	175.5172 Ops/s	$\color{#d91a1a}-1.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8055ms	0.3371ms	2.9667 KOps/s	747.1845 Ops/s	$\textbf{\color{#35bf28}+297.04\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6878ms	0.3071ms	3.2558 KOps/s	3.1649 KOps/s	$\color{#35bf28}+2.87\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.0502ms	5.7430ms	174.1248 Ops/s	173.7547 Ops/s	$\color{#35bf28}+0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.9316ms	0.3691ms	2.7093 KOps/s	3.0359 KOps/s	$\textbf{\color{#d91a1a}-10.76\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5740ms	0.3544ms	2.8217 KOps/s	3.1604 KOps/s	$\textbf{\color{#d91a1a}-10.72\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.0848ms	5.9229ms	168.8375 Ops/s	168.3285 Ops/s	$\color{#35bf28}+0.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.8581ms	0.4768ms	2.0973 KOps/s	2.1360 KOps/s	$\color{#d91a1a}-1.81\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7554ms	0.4673ms	2.1401 KOps/s	2.2334 KOps/s	$\color{#d91a1a}-4.18\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.5114s	15.2020ms	65.7807 Ops/s	194.4454 Ops/s	$\textbf{\color{#d91a1a}-66.17\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	10.3378ms	2.1403ms	467.2285 Ops/s	442.3799 Ops/s	$\textbf{\color{#35bf28}+5.62\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.0430ms	1.0297ms	971.1200 Ops/s	827.2798 Ops/s	$\textbf{\color{#35bf28}+17.39\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	7.5839ms	5.0924ms	196.3729 Ops/s	56.1683 Ops/s	$\textbf{\color{#35bf28}+249.62\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	7.1640ms	1.9627ms	509.5085 Ops/s	507.0497 Ops/s	$\color{#35bf28}+0.48\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	8.3415ms	1.2060ms	829.1803 Ops/s	845.9802 Ops/s	$\color{#d91a1a}-1.99\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	8.2014ms	5.2619ms	190.0469 Ops/s	185.0346 Ops/s	$\color{#35bf28}+2.71\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	10.5479ms	2.1279ms	469.9532 Ops/s	450.7630 Ops/s	$\color{#35bf28}+4.26\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.8933ms	1.3273ms	753.4127 Ops/s	724.9186 Ops/s	$\color{#35bf28}+3.93\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	35.2312ms	33.1839ms	30.1351 Ops/s	30.1728 Ops/s	$\color{#d91a1a}-0.12\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.0097ms	17.4507ms	57.3044 Ops/s	57.4786 Ops/s	$\color{#d91a1a}-0.30\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	35.9345ms	33.7979ms	29.5877 Ops/s	29.2988 Ops/s	$\color{#35bf28}+0.99\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.3171ms	17.7662ms	56.2868 Ops/s	56.7488 Ops/s	$\color{#d91a1a}-0.81\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	37.7173ms	35.7804ms	27.9483 Ops/s	27.7119 Ops/s	$\color{#35bf28}+0.85\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.0171ms	18.9320ms	52.8207 Ops/s	51.9177 Ops/s	$\color{#35bf28}+1.74\%$