How to scale random simulations with parallelism in Swift?

I'm playing around in Swift 5.1 (on the Mac) running little simulations of tennis matches. Naturally part of the simulation is randomly choosing who wins each point.

Below is the relevant part of the code where I do the parallelism.

func combine(result: MatchTally)
{
    overallTally.add(result: result)
}

DispatchQueue.concurrentPerform(iterations: cycleCount){iterationNumber in
    var counter = MatchTally()
    for _ in 1...numberOfSimulations
    {
        let result = playMatch(between: playerOne, and: playerTwo)
        counter[result.0, result.1] += 1
    }
    combiningQueue.sync {combine(result: counter)}
}

With an appropriate simulation run count chosen, a single queue takes about 5s. If I set the concurrent queues to 2, the simulation now takes 3.8s per queue (i.e. it took 7.2s). Doubling again to 4 queues results in 4.8s / queue. And finally with 6 queues (the machine is a 6 core Intel i7) things take 5.6s / queue.

For those who need more convincing that this relates to random number generating (I'm using Double.random(0...1)) I replaced the code where most of the random outcomes are generated with a fixed result (I couldn't replace the second place as I still needed a tie-break) and adjusted the number of simulations appropriately, the outcomes were as follows:

• 1 queue: 5s / queue
• 2 queues: 2.7s / queue
• 4 queues: 1.9s / queue
• 6 queues: 1.7s / queue

So as you can see, it appears that the randomness part is resistant to running in parallel.

I've also tried with drand48() and encountered the same issues. Anybody know whether this is just the way things are?

Xcode 11.3, Swift 5.1, macOS 10.15.3, Mac mini 2018, 6 core i7 (but have encountered the same thing over the years on different hardware)