Author of this content has low reputation.

Using SKlearn I am getting memory errors is there anyway to use batching?

Words
53
Reading
1 min
Listen
Play
9y

#MemoryError clf.fit(X)

#500k dataset [0,0,0]

# I cannot explore the data if I cannot create a pickle so how do I process this data set in batches and get accurate results?

###################################################

df = pd.read_csv('uberunixtime.csv')

df.drop(['Base','DateTime'], 1, inplace=True)

df.convert_objects(convert_numeric=True).dtypes

df.dropna(inplace=True)

df['timeSeconds'] = df['timeSeconds']/10

X = np.array(df)

X = preprocessing.scale(X)

clf = MeanShift()

clf.fit(X)

Using SKlearn I am getting memory errors is there anyway to use bat... | Ecency