jeudi 14 avril 2016

R snow rlecuyer: apply function with temporary seed

I want to apply a function in parallel using snow::clusterApply. My function uses a temporary (pre-defined) seed in one part of the function but should maintain independent random numbers in general. The temporary seeds differ for each "job".

I can do the following:

# setting up cluster of type="SOCK"
library(snow)
cl <- makeSOCKcluster(2)

# this is my function
myfu <- function(seed){
    # temporary seed:
    x <- R.utils::withSeed(rnorm(10),seed)
    # more calculation with independent RNG:
    y <- x*rnorm(10)
    return(list(stays=x,changes=y))
}

# run example:
seed <- 1:4
clusterApply(cl,seed,myfu)
# reproduce
clusterApply(cl,seed,myfu)

stopCluster(cl)

However, following the link given in the "Uniform Random Number Generation in SNOW Clusters" section in the snow package documentation (http://ift.tt/1JKCOPz), I read that "The default random number generators are likely to be quite correlated" and one should use extra packages such as e.g. the package rlecuyer.

Now, if I try to include that in my code, set.seed or withSeed are no use any more:

# setting up cluster of type="SOCK"
library(snow)
library(rlecuyer)
cl <- makeSOCKcluster(2)

# setup RNG Stream
clusterSetupRNGstream(cl,seed=1:6)

# run example:
seed <- 1:4
clusterApply(cl,seed,myfu)
# can't reproduce
clusterApply(cl,seed,myfu)

stopCluster(cl)

How can I work around set.seed or withSeed in parallel when I need to call them on a job basis and not prior to the clusterApply call?




Aucun commentaire:

Enregistrer un commentaire