jeudi 30 août 2018

Is there a package for reading large data in blocks AND randomly split it into N parts in R?

everyone! I am a decent R user who already used R for several years in modeling and manipulation. Since R is an in RAM operational language, is there a package that can do both of the following:

  1. Can read the extremely large file (like 1 TB CSV) in (let's say) 256MB blocks each time.
  2. Distribute them into N (user-defined) files and save as different CSV files.

Since I am mostly working on modeling, I am really lacking of knowledge on read and split data. I knew some packages can do part of it, but not with both of it.

Again, thank you for any suggestions and help!




Aucun commentaire:

Enregistrer un commentaire