I want to train the following model.
def get_uncompile_model(input_shape):
model = keras.models.Sequential()
model.add(keras.layers.Conv1D(filters=64, kernel_size=3,
padding='causal',input_shape=input_shape))
model.add(keras.layers.Bidirectional(keras.layers.LSTM(64,return_sequences=True)))
model.add(keras.layers.LSTM(52))
model.add(keras.layers.Dropout(0.2))
model.add(keras.layers.Dense(32))
model.add(keras.layers.Dropout(0.2))
model.add(keras.layers.Dense(1,activation='linear'))
model.add(keras.layers.Lambda(lambda x: x*10000))#norm.mean.numpy()
model.build()
model.summary()
return model
model=get_uncompile_model([168,6])
optimizer = keras.optimizers.Adam(0.008245186880230904)
# Set the training parameters
model.compile(loss=keras.losses.Huber(), optimizer=optimizer, metrics=['mse','mae'])
model.fit(x=train_series,
y=train_targets,
epochs=10,
steps_per_epoch=train_series.shape[0]//BATCH_SIZE,
validation_data=(val_series,val_targets),
validation_steps=BATCH_SIZE)
But when I try, no matter if used or not batches, validation data, or steps, I always get the following error
Error: Canceled future for execute_request message before replies were done at a.KernelShellFutureHandler.dispose (/home/lean-user/.openvscode-server/extensions/ms-toolsai.jupyter-2022.3.1001111913/out/node_modules/@jupyterlab/services.js:2:32353) at /home/lean-user/.openvscode-server/extensions/ms-toolsai.jupyter-2022.3.1001111913/out/node_modules/@jupyterlab/services.js:2:26572 at Map.forEach (<anonymous>) at y._clearKernelState (/home/lean-user/.openvscode-server/extensions/ms-toolsai.jupyter-2022.3.1001111913/out/node_modules/@jupyterlab/services.js:2:26557) at /home/lean-user/.openvscode-server/extensions/ms-toolsai.jupyter-2022.3.1001111913/out/node_modules/@jupyterlab/services.js:2:29000 at runMicrotasks (<anonymous>) at processTicksAndRejections (node:internal/process/task_queues:96:5)
Sometimes It is at the end of an epoch, sometimes at the middle, but never past one epoch.
I think it has something to do with the RAM or the kernel because once the error happens I have to close and reload the project. The shape of my inputs is (samples, ) + (168, 6), I have 31000 samples for the training dataset. Can someone help me, please
Fabian Kliem
I have the same issue. Any solution for this?
Louis Szeto
Hi Andre and Fabian
I believe the jupyter kernel died from depleting the RAM memory, please refer to Jared's answer here. Consider upgrading to a higher capacity research node if you need a larger scale ML. For further assistance/suggestions, please include the full snippets.
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Andres tello
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!