vendredi 8 décembre 2017

Generating a dictionary of distinct random words in C++

I am trying to generate a random collection of distinct words and write it into a txt file. As a convention, each word should be of random length between 2 and 5 characters and each line will consist of 10 words. Also the output file should have approximately a predefined size in bytes.

The desired file size is 10MB and it should have no duplicate words.

The code is as follows:

#include <bits/stdc++.h>
#include <cstdlib>

using namespace std;

//The number of bytes you want your file to be
unsigned long targetBytes = 10000000;

// Prints unique words in a file
void generateUniqueWords(char filename[])
{    
    unsigned long count = 0;

    map<string, int> mp;

    ofstream of(filename, ios::out | ios::trunc);

    unsigned int wordsPerLine = 10;

    for (unsigned long i = 1; i < ULONG_MAX; i++){

        unsigned int length = rand()%4 + 2;//for small words from 2 to 5 characters

        char word_char[length+1];

    for(unsigned int j = 0; j < length; j++){
            word_char[j] = 'a' + random()%26;;
    }
        word_char[length] = '\0';

        string word_string(reinterpret_cast<char*>(word_char),  length);

        if (mp.count(word_string) == 0){
        mp.insert(make_pair(word_string, 1));
            of.write(word_char, length);
        }else{
            continue;
        }

    count += length; 

    if (i % wordsPerLine == 0){
        of.write("\r\n", 2);
        count+=2;
    }else{
        of.write(" ", 1);
       count++;
    }

        if (count >= targetBytes) break;

    }
    of.close();
}

int main()
{
    char filename[] = "text.txt";

    generateUniqueWords(filename);
    return 0;
}

I have used g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-11) (on a CentOS virtual machine). This program works but I would like to get some insight on perhaps more efficient ways to do this that can perform faster. Of course, there is a relationship between the average length and the total file size. One cannot expect to generate 1GB collection with an average length of 4 characters (including the space and the \r\n) per word.




Aucun commentaire:

Enregistrer un commentaire