A Python application I'm writing needs to extract identifiers and text strings out of source code. A small percentage of what it finds are (seemingly) random strings. I'd like to filter them out, but so far have not been able to create a regexp to do it. It is not possible to filter by length only, because there are some very long identifiers that are valid. Here is an example taken at random, compared to a valid identifier of the same length:
UGxhemEgZGUgaWZXNaWdhZGyOiBDSUWRVNUQVYtSVBOIFVuaWQ
NSApplicationDidChangeScreenParametersNotification
Is there a way to write a regexp or other detection system that would detect junk sequences like this? I am beginning to suspect it can't be done without testing strings against a large dictionary of words, which I believe would be prone to errors as well as be compute-intensive. However, maybe someone more clever knows of an approach to detecting or matching random sequences like this?
An ideal solution to this problem would be a function that can take a string as input, and report if it is "probably" random. It could produce false negatives (misreport some random strings as not random), preferably with low probability, but it must not report false positives (something is random when it is not). In case it matters, the strings seem to range in length from 25 to 80 characters.
Aucun commentaire:
Enregistrer un commentaire