I'm using Mydia to extract random frames from videos. Because I have a lot of videos, I want to parallelize this workflow while maintaining repeatability. mydia.Videos accepts a random seed, which is important for ensuring repeatability. Now I need to work on the parallelization piece.
Given n videos and a random seed, r, how can I ensure that the extracted frames for each video is the same regardless of the number of workers?
My initial thought was to use multiprocessing.Pool; however, there will be a race condition in sampling the frames if the processes' completion times are non-determinstic. i.e., if proc 1 takes longer than proc 0, the sampled frames from the Videos class will be different than if proc 0 takes longer than proc 1.
Aucun commentaire:
Enregistrer un commentaire