mardi 11 août 2015

Why is there no standard test for random number generators?

I've been looking at this problem for the last year, and can't understand why there's no standard way to test for randomness in the real world. It seems to be that it's just what make you comfortable.

(I'm ruling out random sequences that are really really not random, like 0123456789...repeating.)

Randomness testing issues lists some widely known tests, and a whole list of issues with them. I can add others. Diehard - how big should the input file be, and should it consist only of 32 bit integers? ENT - only seems suitable for gross errors, but the chi test is useful. The NIST user manual is >100 pages long - good luck. TestU01 - has compilation issues on certain platforms. And once you've shoehorned it into your computer, is it running correctly? How can you then trust the output? And how do you know if a test has failed? What level of p or KS is considered too extreme?

I would further add that you should consider the development of randomness test suites vis a vis real-politic. It's in an academic's self interest to develop tests that discredit random number generators. After all, you don't get no funding producing results that say "it's all okay, nothing found, no further research (read: money) required".

Consider what happens in the real world that we live in, not on an academic's bookshelf:-

Random.org - used an undergrad to undertake some homebrew tests for a thesis. And essentially count the number of 1's and 0's. ENT does similar. They stake their business model on this.

Hotbits - champion the simplistic ENT, and a hacked version of Dieharder that most people will have difficulty in getting to execute, never mind trying to comprehend myriad test initialisers.

Academic generator papers - much recourse to Knuth's writings and homespun techniques. Some use some of the above tools. Some then accept a number of test failures within those suites.

The only examples I've found so far in this man's universe that seems to carry any real weight (i.e. if it fails you go to prison type of weight) is the certification for:-

Playtech PLC, a UK supplier of gambling software. They supply some of the largest online betting companies where real money changes hands. Still, they use homebrew tests and the Diehard test.

ERNIE for the UK Premium Bonds. They use basic statistics tests for frequency and correlation. Effectively home brew and not using a published suite.

The two latter examples seem to suggest that the current Zeitgeist is being molded by financial bodies. Random numbers are a form of maths, a reasonably established discipline. Why isn't there a tried and tested program suite that everyone uses, and it's output says yea or ney?




Aucun commentaire:

Enregistrer un commentaire