dimanche 4 octobre 2015

how to run random sampling a million times from a csv

I have a big csv looks like this

claim score
yes   1
yes   1
no    1
no    1
yes   1
...   1
...   1
...   1

score is all the same number, i need to run a lots times of random sampling for size say (1000). Then calculate the mean percentage of yes counts

the code looks like this:

#imports
import random
import numpy

TotalYes = 0
csvFile = numpy.genfromtxt("/nas/home/twu/wind/output_1.csv",delimiter=",",dtype=None)
for j in range(1,10001):
    #csv format : claim (Yes/No), value
    #read in your csv file and store in array
    #initialize random number generator
    random.seed()

#create RandomSample array
RandSamples=[]
samplesize = 1000
#Fill RandomSample array with 10000 random samples from cvs array
for i in range(1,1001):
    #for row in csvFile:
    #get a random index within csvFile[]. random num range is 0 to csv array length
    randIndex=random.randint(0,len(csvFile))
    print randIndex
    RandSamples.append(csvFile[randIndex:randIndex+1,:])
#RandSamples1=numpy.asarray(RandSamples)
#get number of 'yes' from RandomSample array
RandYesSample=[]
for i in range(0,1001):

    # check to see if current record is Yes claim or no
    if RandSamples[i:i+1,:1] == "yes":
        #yes, copy value to yes array
        RandYesSample.append (RandSamples[i:i+1,:1])

#get percent of yes in RandomSample array
PercYes = float(len(RandYesSample)) / 1000
TotalYes = TotalYes + PercYes

TotalYes = float(TotalYes) / 10000

print TotalYes  

The error I had is:

if RandSamples[i:i+1,:1] == "yes":...TypeError: list indices must be
integers, not tuple

I cannot get it working. can somebody help?




Aucun commentaire:

Enregistrer un commentaire