Share New Research 
When running live trading with GDAX from stand-alone LEAN instances, I frequently notice websocket disconnection (example logs below). The situation appears to be quite bad in us-east zones (Google Cloud instances) with disconnects happening every few minutes. Zones in us-west seem a bit better but still disconnecting every hour or so.
What are the recommended practices for running LEAN instances with GDAX in terms of geo-location / cloud zones? I found some discussions mentioning websocket provision with cloudflare (supposed to work in multiple regious right?) but no specific conclusions on achieving the websocket stability.
Example log fragments:
20180609 02:18:58.877 Trace:: 2018-06-09 02:18:58Z Isolator.ExecuteWithTimeLimit(): Used: 47 Sample: 43
20180609 02:19:59.049 Trace:: 2018-06-09 02:19:59Z Isolator.ExecuteWithTimeLimit(): Used: 47 Sample: 46
20180609 02:20:54.009 Trace:: Debug: 6/9/2018 2:20:53 AM|Fatal|WebSocket.<startReceiving>b__170_2|System.NullReferenceException: Object reference not set to an inst
ance of an object
at WebSocketSharp.WebSocket.<startReceiving>b__170_2 (System.Exception ex) [0x00001] in <1fc1e12f21eb4681aa69c484f58d0121>:0
at WebSocketSharp.Ext.ReadBytesAsync (System.IO.Stream stream, System.Int32 length, System.Action`1[T] completed, System.Action`1[T] err
or) [0x00093] in <1fc1e12f21eb4681aa69c484f58d0121>:0
at WebSocketSharp.WebSocketFrame.readPayloadDataAsync (System.IO.Stream stream, WebSocketSharp.WebSocketFrame frame, System.Action`1[T]
completed, System.Action`1[T] error) [0x00097] in <1fc1e12f21eb4681aa69c484f58d0121>:0
at WebSocketSharp.WebSocketFrame+<>c__DisplayClass82_0.<ReadFrameAsync>b__2 (WebSocketSharp.WebSocketFrame frame2) [0x00000] in <1fc1e12
f21eb4681aa69c484f58d0121>:0
at WebSocketSharp.WebSocketFrame.readMaskingKeyAsync (System.IO.Stream stream, WebSocketSharp.WebSocketFrame frame, System.Action`1[T] c
ompleted, System.Action`1[T] error) [0x0004a] in <1fc1e12f21eb4681aa69c484f58d0121>:0
at WebSocketSharp.WebSocketFrame+<>c__DisplayClass82_0.<ReadFrameAsync>b__1 (WebSocketSharp.WebSocketFrame frame1) [0x00000] in <1fc1e12
f21eb4681aa69c484f58d0121>:0
at WebSocketSharp.WebSocketFrame.readExtendedPayloadLengthAsync (System.IO.Stream stream, WebSocketSharp.WebSocketFrame frame, System.Ac
tion`1[T] completed, System.Action`1[T] error) [0x00044] in <1fc1e12f21eb4681aa69c484f58d0121>:0
at WebSocketSharp.WebSocketFrame+<>c__DisplayClass82_0.<ReadFrameAsync>b__0 (WebSocketSharp.WebSocketFrame frame) [0x00000] in <1fc1e12f
21eb4681aa69c484f58d0121>:0
at WebSocketSharp.WebSocketFrame+<>c__DisplayClass73_0.<readHeaderAsync>b__0 (System.Byte[] bytes) [0x0000c] in <1fc1e12f21eb4681aa69c48
4f58d0121>:0
at WebSocketSharp.Ext+<>c__DisplayClass54_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x0009e] in <1fc1e12f21eb4681aa69c484f58d0121
>:0
20180609 02:20:59.226 Trace:: 2018-06-09 02:20:59Z Isolator.ExecuteWithTimeLimit(): Used: 47 Sample: 40
20180609 02:21:59.281 Trace:: 2018-06-09 02:21:59Z Isolator.ExecuteWithTimeLimit(): Used: 47 Sample: 52
20180609 02:22:24.212 Trace:: Brokerage.OnMessage(): Disconnect - Code: Disconnect - Connection with server lost. This could be because of internet connectivity iss
ues.
20180609 02:22:24.212 Trace:: DefaultBrokerageMessageHandler.Handle(): Disconnected.
20180609 02:22:24.212 Trace:: DefaultBrokerageMessageHandler.Handle(): Disconnect when exchanges are open, trying to reconnect for 15 minutes.
20180609 02:22:25.725 Trace:: BaseWebsocketsBrokerage(): Reconnecting... IsConnected: False
20180609 02:22:26.432 Trace:: WebSocketWrapper.OnOpen(): Connection opened(True): wss://ws-feed.gdax.com
20180609 02:22:26.433 Trace:: Brokerage.OnMessage(): Information - Code: -1 - GDAXBrokerage.Subscribe: Sent subscribe.
20180609 02:22:27.018 Trace:: Debug: Brokerage Info: GDAXBrokerage.Subscribe: Sent subscribe.
20180609 02:22:27.311 Trace:: Brokerage.OnMessage(): Information - Code: -1 - GDAXBrokerage.Subscribe: Sent subscribe.
20180609 02:22:28.030 Trace:: Debug: Brokerage Info: GDAXBrokerage.Subscribe: Sent subscribe.
20180609 02:22:34.290 Trace:: Brokerage.OnMessage(): Reconnect - Code: Reconnect - Connection with server restored.
20180609 02:22:34.291 Trace:: DefaultBrokerageMessageHandler.Handle(): Reconnected.
20180609 02:22:59.427 Trace:: 2018-06-09 02:22:59Z Isolator.ExecuteWithTimeLimit(): Used: 48 Sample: 47
20180609 02:23:59.554 Trace:: 2018-06-09 02:23:59Z Isolator.ExecuteWithTimeLimit(): Used: 48 Sample: 41
Jared Broad
Web sockets aren't inherently stable over the internet; timeouts happen for 100 reasons between points A-B. LEAN builds in aggressive reconnection attempts which with retail-cloud platforms or home internet connections it is about the best you can do.
In QC Cloud we've invested in hardware and racked the servers in Equinix which is virtually perfect. We used to be in retail clouds but found their internet too unreliable.
"Every few minutes" is still too frequent disconnections; perhaps there's some firewall issue? Or a dynamic IP shifting around?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Pavel Paramonov
Going away from a dynamic IP was helpful, thank you Jared.
After testing various zones with a retail cloud provider, I found US-Central workable with about one disconnect every few hours.
Pavel Paramonov
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!