mardi 23 juin 2020

std::discrete_distribution for large arrays

I need to perform weighted random sampling with replacement. I'm given the input weights as the probabilities themselves so I don't need to normalize. I believe std::discrete_distribution would help me do that.

I'm using it as:

long long int numElements = 1L << 34;
double *probabilities = {0.001, 0.0003 ....... } (Points to an array with 2^34 elements)
std::default_random_engine generator(std::random_device {}());
std::discrete_distribution<int64_t> distribution(probabilities, probabilities + numElements);

This seems to work upto 1L << 33 elements but breaks after that. It returns only 0 or 2^33. I'm running this in a compute environment where memory is not an issue. So I assume that this has to do with the discrete_distribution implementation not supporting integers greater than 33 bits, or an issue with the way I'm using it.

Could someone please help point out if I'm using it incorrectly? Or if there's a better way to accomplish what I'm doing?




Aucun commentaire:

Enregistrer un commentaire