mardi 17 décembre 2019

How should the sequence number in curand_init be chosen when repeating a set of multiple kernels

I am writing CUDA code for the simulation of spiking neural networks. Every time step, the same set of kernels are called which have different grid and block dimensions (and total number of threads) and of which all produce random numbers using the cuRAND device API. The cuRAND device API documentation says

For the highest quality parallel pseudorandom number generation, each experiment should be assigned a unique seed. Within an experiment, each thread of computation should be assigned a unique sequence number. If an experiment spans multiple kernel launches, it is recommended that threads between kernel launches be given the same seed, and sequence numbers be assigned in a monotonically increasing way. If the same configuration of threads is launched, random state can be preserved in global memory between launches to avoid state setup time.

Question: Can I create as many cuRAND states (with same seed and different sequence number) as the maximum number of threads used in any of my kernels and use the same states when kernels with less threads are called? Or do I need to generate one state per thread per kernel that is repeated?

Example: I have two kernels, which are called non-concurrently every simulation time step: kernal 1 uses 10 threads, kernel 2 uses 100 threads. Can I create 100 cuRAND states and use only 10 of them in kernel 1 and all of them in kernel 2? Or do I need to create 110 cuRAND states and use different states in kernel 1 and kernel 2?

Since random number sequences with same seed and different sequence number have statistically not correlated values (according to the documentation), I would assume it doesn't matter that one sequence is moving forward faster than the other (assuming I do not generate 2^67 numbers, which seems to be the period or random numbers for one sequence number?). But I don't know enough about random number generation to be sure.




Aucun commentaire:

Enregistrer un commentaire