Author of this content has low reputation.
Using SKlearn I am getting memory errors is there anyway to use batching?
Words
53
Reading
1 min
Listen
Play
9y
#MemoryError clf.fit(X)
#500k dataset [0,0,0]
# I cannot explore the data if I cannot create a pickle so how do I process this data set in batches and get accurate results?
###################################################
df = pd.read_csv('uberunixtime.csv')
df.drop(['Base','DateTime'], 1, inplace=True)
df.convert_objects(convert_numeric=True).dtypes
df.dropna(inplace=True)
df['timeSeconds'] = df['timeSeconds']/10
X = np.array(df)
X = preprocessing.scale(X)
clf = MeanShift()
clf.fit(X)