jeudi 10 octobre 2019

Is there a more efficient way to oversample data than random.sample()?

I got a big unbalanced classification problem and want to address this issue by oversampling the minor classes. (N(class 1) = 8,5mio, N(class n) = 3000)

For that purpose I want to get 100.000 sample for each of the n classes by

data_oversampled = []
for data_class_filtered in data:
    data_oversampled.append(data_class_filtered.sample(n=100000, replace=True))

where data is a list of class specific DataFrames.

That works as expected but unfortunately takes literally forever. Is there a more efficient way to do the same thing?




Aucun commentaire:

Enregistrer un commentaire