Random State Duplication

Random states can easily be duplicated when parallel processing causing duplicate number generation
Python
Author

Isaac Flath

Published

November 5, 2022

Today I Learned

You should be really careful about random state when parallel processing. If you aren’t careful, the random seed will be copied over to other processes and each process will generate the same random numbers! I learned this from a fastai lecture in the 2022 part 2 course.

Image showing duplicate random numbers being creating on separate processes due to random seeds being accidentally duplicated to multiple processes