Share New Research 
Hello,
during live implementation of my algorithm I'm trying to get history of last 300000 SPY trade bars:
self.History(["SPY"], 300000, Resolution.Minute).loc["SPY"]I've got following error:
The history request took too long to process. History requests have a 20 second timeout.
I've checked it more precisely and 20s time frame is enough only to get 150000 trade bars. I am blocked and I think the best would be to somehow make time limit dependent on the history length request. We're going more and more in the machine learning applications, so we need to access quite big data sets.
Andrzej Fudala
If somebody is interested - below potential solution for minute resolution which splits the request.
self.splitted_history_request(self.stock, 300000, 100000) def splitted_history_request(self, stock, periods, single_request_period): security = self.Securities[stock] dates = [] dates.append(Time.GetStartTimeForTradeBars(security.Exchange.Hours, self.UtcTime, Time.OneMinute, periods, security.IsExtendedMarketHours)) for i in reversed(range(periods / single_request_period)): splitted_periods = (i + 1) * single_request_period if splitted_periods != periods: dates.append(Time.GetStartTimeForTradeBars(security.Exchange.Hours, self.UtcTime, Time.OneMinute, splitted_periods, security.IsExtendedMarketHours)) dates.append(self.UtcTime) hist = [] for i in range(len(dates[:-1])): hist.append(self.History([stock.Value], dates[i], dates[i+1], Resolution.Minute).loc[stock.Value]) hist = pd.concat(hist) return datesDan Whitnable
Agreed. It would be helpful to have an efficient History method. I'm sure the QC folks are going to mention RollingWindows, however there are two issues with that approach. First, it doesn't work easily with a dynamic universe selection. Second, RollingsWindows are not updated for stock splits in live trading (ie previous entries in the window are not adjusted for a current split). The ability to fetch historical data is a must. I would suggest that a large portion of algorithms don't need 'real time' historicaal data. Fetching data could occur overnight or have a long timeout during trading hours.
Andrzej Fudala
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!