mardi 9 février 2021

Correct way to add a column of random numbers to a dask dataframe

What is the correct way of adding a column of random numbers to a dask dataframe? I could obviously use map_partitions to add the column to each partition but I'm not sure how the random state is handled when dask parallelises that computation. (I.e. will it use the same random state in all workers and therefore generate the same random numbers in each worker?)

There seem to be relevant functions in dask.array.random (https://docs.dask.org/en/latest/_modules/dask/array/random.html) but I wasn't able to find an example for how to use these together with a dask dataframe.




Aucun commentaire:

Enregistrer un commentaire