mercredi 22 mai 2019

Reading a CSV file line by line and producing streams with some lines sent with a random delay

I am reading concurrently 3 CSV file line by line and sending them to a message queue (Apache Kafka). The data rows are ordered by increasing timestamp values. I am simulating the streams by looking at the last timestamp and at the new timestamp and make a thread sleep for the difference of the two timestamps. I do this in order to simulate the production of the messages.

Now I want to have some of those messages delayed by a random amount of time and have prepared a function that put the thread to sleep for a random amount of time and chooses randomly when to perform this operation.

When I do that I put to sleep the entire reading of the CSV file by delaying all the subsequent messages that have to be produced.

Probably I am lacking experience in doing this stuff, but I don't know how to randomly put to sleep and of the messages that I am producing without delaying all the messages that have to come?




Aucun commentaire:

Enregistrer un commentaire