vendredi 7 août 2015

Lossless Compression of Random Data

tl;dr

I recently started listening to a security podcast, and heard the following sentence (paraphrasing)

One of the good hallmarks of a cryptographically strong random number is its lack of compressibility

Which immediately got me thinking, can random data be lossless-ly compressed? I started reading, and found this wikipedia article. A quoted block is below

In particular, files of random data cannot be consistently compressed by any conceivable lossless data compression algorithm: indeed, this result is used to define the concept of randomness in algorithmic complexity theory.

I understand the pigeon hole principle, so I'm assuming I'm way wrong here somewhere, but what am I missing?

IDEA:

Assume you have an asymmetric variable-length encryption method by which you could convert any N bit into either a N-16 bit number or N+16 bit number. Is this possible?

IF we had an assymetric algorithm could either make the data say 16 bits bigger or 16 bits smaller, then I think I can come up with an algorithm for reliably producing lossless compression.

Lossless Compression Algorithm for Arbitrary Data

Break the initial data into chunks of a given size. Then use a "key" and attempt to compress each chunk as follows.

function compress(data)
  compressedData = []
  chunks = data.splitBy(chunkSize);

  foreach chunk in chunks
    encryptedChunk = encrypt(chunk, key)
    if (encryptedChunk.Length <= chunk.Length - 16) // arbitrary amount
      compressedData.append(0) // 1 bit, not an integer
      compressedData.append(encryptedChunk)
    else
      compressedData.append(1) // 1 bit, not an integer
      compressedData.append(chunk)
  end foreach

  return compressedData;

end function

And for de-compression, if you know the chunk-size, then each chunk that begins with 0 perform the asymmetric encryption and append the data to the on going array. If the chunk begins with a 0 simply append the data as-is. If the encryption method produces the 16-bit smaller value even 1/16 as often as the 16-bit larger value, then this will work right? Each chunk is either 1 bit bigger, or 15 bits smaller.

One other consideration is that the "key" used by the compression algorithm can be either fixed or perhaps appended to the beginning of the compressed data. Same consideration for the chunk size.




Aucun commentaire:

Enregistrer un commentaire