Indicator Updates / After Hours Trading

I have a simple desire to wait until market close, get the updated daily indicators based on the closing price, and seek to buy in after hours trading. I have created the attached project with only RSI to illustrate my issues.

Daily Indicators Update @ 4AM the following day? - When using the helper function RSI(), the indicator doesn't seem to update at 4pm on market close... Or at any time later in after hours trading (even though I have enabled extended hours trading data at the second resolution when adding my security). Instead, it updates at 4am the following morning. I want to trade in the AH market with the indicator updated based on the closing price. Am I doing something wrong or is this how it works?
In extended market hours I have proven that my OnData function appears to be called randomly after 4pm rather than every second. Take a look at some data below from a similar project to the attached that was restricted to outputting a log message at the first available opportunity after 4pm. Even though the security data resoluion was set to second, the first execution of the loop return varied from 16:00:05 all the way to 16:11:00. Is there an explanation for this? I would expect every trading day where data was being fed to OnData to trigger within a second of the same time (ie. 16:00:05 or 16:00:06). Having an apparent random variance of up to 11 minutes on second data is a significant incongruity. Interestingly the apparently random times remained consistent when backtesting different overlapping date ranges. (See brief log output data showing these results posted below).
After Hours Orders - I had quite a bit of trouble with after market hours orders and I would like to understand how to place them so they can execute effectively. At first, my orders were being queued but never filled. I guessed it must be my brokerage model, so I set that to default. Then I started executing limit orders quite far outside what I understood the current aftermarket price to be. Sometimes these orders would execute, but other times they would not. If I am trading a very liquid security / ETF such as XIV, what is the best way to me to effectively place a buy order in After Market hours at the "current" price?

Please note that I am using Resolution.Second data with after hours data enabled.

Log output testing AH first OnData() call issue. The first available OnData() execution cycle after 4:00pm market close appears to be random. Additionally you can see the RSI update varies up to 2 hours from 4am - 6am every morning. I don't understand why. Please help! Thanks in advance. =)

2016-03-01 00:00:00 Launching analysis for b592bfb6498af4ca953757aa4eb5c4c7 with LEAN Engine v2.3.0.2.1318 2016-02-29 04:00:01 Algorithm warming up... 2016-03-01 00:00:00 Algorithm finished warming up. 2016-03-01 06:01:32 OnRSIUpdate() 2016-03-01 16:11:01 Time: 03/01/2016 16:11:00 2016-03-02 04:00:01 OnRSIUpdate() 2016-03-02 16:00:40 Time: 03/02/2016 16:00:39 2016-03-03 04:00:01 OnRSIUpdate() 2016-03-03 16:05:17 Time: 03/03/2016 16:05:16 2016-03-04 04:00:01 OnRSIUpdate() 2016-03-04 16:00:20 Time: 03/04/2016 16:00:19 2016-03-07 04:00:01 OnRSIUpdate() 2016-03-07 16:00:44 Time: 03/07/2016 16:00:43 2016-03-08 06:03:24 OnRSIUpdate() 2016-03-08 16:02:20 Time: 03/08/2016 16:02:19 2016-03-09 05:00:27 OnRSIUpdate() 2016-03-09 16:02:30 Time: 03/09/2016 16:02:29 2016-03-10 04:07:37 OnRSIUpdate() 2016-03-10 16:03:56 Time: 03/10/2016 16:03:55 2016-03-11 05:13:10 OnRSIUpdate() 2016-03-11 16:02:56 Time: 03/11/2016 16:02:55 2016-03-14 04:25:40 OnRSIUpdate() 2016-03-14 16:03:14 Time: 03/14/2016 16:03:13 2016-03-15 06:46:34 OnRSIUpdate() 2016-03-15 16:01:02 Time: 03/15/2016 16:01:01 2016-03-16 05:06:20 OnRSIUpdate() 2016-03-16 16:00:39 Time: 03/16/2016 16:00:38 2016-03-17 04:00:01 OnRSIUpdate() 2016-03-17 16:00:11 Time: 03/17/2016 16:00:10 2016-03-18 04:08:23 OnRSIUpdate() 2016-03-18 16:00:06 Time: 03/18/2016 16:00:05 2016-03-19 00:00:00 Algorithm Id:(b592bfb6498af4ca953757aa4eb5c4c7) completed in 9.46 seconds at 39k data points per second. Processing total of 367,160 data points.

In backtest mode, the algorithm can only determine a day change when it process a tick/bar with a new/different date. The first data in a given day is normally at 4 am.
After market is closes and enters extended hours, liquidity drops and we stop having enough data to call OnData every second (or minute). If we want a steady flow of data, we can subscribe to it with the fillforward parameter set to true:

var targetSecurity = AddSecurity(SecurityType.Equity, 
    _symbol, 
    Resolution.Second, 
    true,   // Fill Data Forward
    true);  // Extended Market Hours

Market orders are not filled when the market is closed. They are converted into MarketOnOpenOrder. In this case, we need to rely on limit orders.

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Is there any way I can get the latest value for a daily indicator at the market close so I can trade on that data in the after market hours? I suppose I could avoid using the RSI helper function mentioned above, but then I would be completely responsible for feeding the indicators myself at the prescribed time.

Alexandre Catarino INVESTOR

QuantConnect | March 2017

Upvote

1 person upvoted this

Chris Martin INVESTOR

March 2017

I simply want to get the computed daily indicator for RSI at market close. The Update method of the RelativeStrenghIndex class simply don't allow this. The Update function requires an IndicatorDataPoint and I don't see how to create the type it needs to update at the EOD. I have multiple indicators I would like to do this with. Can anyone point me in the right direction?

JayJayD INVESTOR

Hey Chris,

You can have total control of the data input to any indicator y using the Update method. And, if you want to update an indicator in a certain time, you can use Schedule events.

Update and indicator with an IndicatorDataPoint is as easy as:

_insicator.Update(new IndicatorDataPoint
{
     Time = Time,
     Value = Securities[symbol].Price
});

Check this simple example, the RSI is updated randomly and every day at 16:00.

Is this what are you looking for?

I appreciate your effort here.

Updating the indicator is what I wanted, but I much preferred relying on the helper function to initiate a dependable daily update to ensure my indicator was daily resolution. I understand it is possible to manage on my own. Looking at the source of IndicatorBase.cs, I can see that there is no protection to ensure a particular resolution for an indicator. If update is called more than once in an indicator that I intend to have a daily resolution, it would distort my value. For now I'm going to rely on the daily indicator to update a day behind as it is coded, then pull the actual calculation code out of RelativeStrengthIndex.cs and produce my own value for RSI at any point in the day without worry of corrupting the base indicator.

Ray Bohac INVESTOR

I'm doing something similar to JayJayD's example above to track 52 week high/low. I track the stocks at a lower resolution but use a daily consolidator to update the indicators to my liking. Example included also shows how to do a warmup.

This example comes from a helper class that I wrote formyself where I only track 1 symbol per instance. You could do something similar, or with slight modifications use Dictionaries or Lists to track mulptiple symbols in a single class.


// Setting it up
var dailyConsolidator = new TradeBarConsolidator(TimeSpan.FromDays(1));
dailyConsolidator.DataConsolidated += OnDataDaily;
SubscriptionManager.AddConsolidator(_symbol, dailyConsolidator);

// Warmup
var bars = _algorithm.History(_symbol, 365, Resolution.Daily);
  foreach (var bar in bars)
  {
  _52WeekLow.Update(new IndicatorDataPoint(bar.Time, bar.Low));
  _52WeekHigh.Update(new IndicatorDataPoint(bar.Time, bar.High));
  }


// Updating the indicators
private void OnDataDaily(object sender, TradeBar consolidated)
{
  _52WeekHigh.Update(consolidated.Time, consolidated.High);
  _52WeekLow.Update(consolidated.Time, consolidated.Low);
}

// using it
if ((_52WeekHigh.IsReady) && if (_52WeekLow.IsReady))
Debug(String.Format("52 Week Low: {0:0.00} High: {1:0.00}",
                _52WeekLow.Current.Value,
                _52WeekHigh.Current.Value));

2 people upvoted this

Ray,

Very informative warmup routine there. Nicely done. I'll be putting that to use as soon as I can get my daily indicator updates completed. I have the RSI updating now. I'm going to post elsewhere about updating the Stochastic Indicator. Stochastic.Update() requires an iBaseDataBar as the input rather than a simple IndicatorDataPoint. Unfortunately iBaseDataBar is an interface so you can't just instantiate and object inline like you can with the class IndicatorDataPoint. I understand that the Stochastic wasn't the HILO information and I wager it's somehwere in the Security data. I just can't find it.

For the benefit of anyone readying through this thread in the future, the Stochastic.Update() can take the TradeBar object you can pull from the TradeBars array you receive in your OnData function. I throught I attempted this on my first try but received the compiler error about the iBaseDataBar interface... I went on a long, unecessary journey through the source code. Hopefully I am better for it.

Jared Broad INVESTOR

" I went on a long, unecessary journey through the source code" :D Sorry Chris, we're working on making this easier. We made the co-pilot recently and will update it to suggest the right input here.

Scott Woods INVESTOR

June 2016

This is kind of embarressing, but I can't seem to figure out how to initalize a candle stick indictator.. any help?

Stefano Raggi INVESTOR

@Scott,
candlestick pattern indicators are quite simple to use:

1. define a variable for the indicator in your algorithm class:

private Harami _pattern = new Harami();

2. create an instance of the indicator in your Initialize method using one of the CandleStickPattern helper methods:

_pattern = CandlestickPatterns.Harami(_symbol);

3. in your OnData, test the value of the indicator value:
- if the value is 0, no pattern found at the current bar
- if the value is +1, bullish pattern found at the current bar
- if the value is -1, bearish pattern found at the current bar

Note that some patterns are only bullish, some are only bearish and some are both.

I attached a simple example algorithm for you

Hope this helps

3 people upvoted this

Thank you so much, I was making a bit of a foolish error. Your example helped me a lo, it was just what i was looking for.

Boris Sachakov INVESTOR

December 2016

@Stefano, Hi!

I followed your example above and wrote an almost identical algorithm for SPY with Closing Marubozu Candlestick pattern. I used the provided data (local Lean data). It worked and performed about 20 trades for the period of 2014 (I set this period of time).

However, when I adjusted the algorithm to receive custom data directly from Yahoo.com with Quandl Wrapper (as recommended in How Do I Import Yahoo Data? in QuantConnect University) I encountered the following problem:

The data is not getting through in OnData method past: if (_pattern == 1) or if (_pattern == -1). It is getting through OnData, but is somehow blocked by == . If I replace it by anything else, the algorithm finds 1st Closing Marubozu pattern on 01/17/2014. So the patterns are definitely there, plus we know that they are there from my earlier algorithm that I ran on Lean data.

Please, recommend the solution. Thanks in advance.

@Boris, to use Quandl data, the easiest way is to prefix the symbol (YAHOO/INDEX_SPY) and use a custom class derived from the base Quandl class.
The problem with your algorithm was that deriving from BaseData all OHLC values were the same, so no patterns could be found.

Here is the revised algorithm:

@Stefano

Thanks for the above help. However, I still want to find a way to be able to use the custom data. I was going to start to try by importing first Yahoo data with Quandl data, and then evetually to import my data with link to Dropbox (as in CustomDataNIFTYAlgorithm.cs).

Also, please, explain how is that all OHLC values were the same in my original algorithm and how to fix it. Thanks in advance.

@Boris,

when I mentioned that OHLC values were the same, I didn't mean the instances of your BaseData-derived class, but the inputs to the candlestick patterns, which are generated internally by a consolidator (in your case a BaseDataConsolidator, which only has a BaseData type as input, instead of a TradeBar). What happens is that the pattern class receives an input TradeBar with all OHLC values equal.

The solution is to derive your class from TradeBar, so a TradeBarConsolidator will be used and the candlestick pattern will get the correct input (TradeBar), as in the attached backtest:

@Stefano,

Thanks again for your help. Your above algorithm worked really well and provided me with a few lessons to learn now. I just wanted to mention that in order to run it on Visual Studio, I needed to add

using QuantConnect.Data.Market;

for TradeBar to work.

Let me study your code and see if I have any further questions. Thank you.

I have now studied your algorithm. Thank you again for providing it. I just have a couple of questions. What is the necessity for EndTime and Period methods inside of

public class CloseMar : TradeBar

when we already have TradeBar.Open and TraderBar.Close values from the 'try' block of 'BaseData Reader'. Plus algorithm seems to able to run without these 2 methods.

Looking forward to your response. Thanks in advance.

Time and EndTime are properties defined in the BaseData class and represent the start and end time of the bar, Period is defined in the TradeBar class and represents the time span between the start and the end of the bar (the default is one minute).
If we don't override them, the custom CloseMar bars will be timestamped incorrectly as one minute bars instead of daily bars.

Thanks for your help. I might have some more questions later.

I used your latest algorithm from above and substituted the www.Quandl.com... in GetSource with link to www.Dropbox.com... with my daily ES data for 2015 (as in CustomDataNiftyAlgorithm.cs). There is no AdjustedClose in this data, so I had to make appropriate (I hope) changes. I ran this algorithm, again looking for Closing Marubozu patterns. But I got nothing. I did not work. Maybe you can review and critique my work please. As always, thanks in advance.

QuantConnect | December 2016

First, please checkout the data format. You are trying to parse datetime with the "yyyy-MM-dd" while the your first column has a "yyyyMMdd" string:

cmBar.Time = DateTime
    .ParseExact(data[0], "yyyyMMdd", 
    CultureInfo.InvariantCulture);

Do you really need to override EndTime and Period? After I removed that from the code, the data flew are expected, the indicator was calculated and there were 18 trades.

I have discovered the problem myself, after multiple tries.

In DateTime.ParseExact's format portion I originally put in "yyyy-MM-dd". As soon as I changed it to "yyyyMMdd", it worked. See the algorithm and backtest. Even though the financial results were poor, but that wasn't the goal at this time. I just need first to get the parts moving correctly.

Thanks again.

@Alexandre,

I posted my last comment before I knew you posted yours. Maybe we even posted them almost at the same time. I changed the date range a little bit too. Anyway, thank you for confirming that date format was incorrect.

Concerning overriding EndTime and Period, I followed the recommendation by Stefano Raggi (see his earlier comments on this thread). I also earlier discovered, that by removing them from the code, the number of trades does not change and financial results do not change, just like you said. However, if we remove override EndTime and Period from the code, each line in the log (and each corresponding trade) are registered 24 hrs earlier. Please see 'Logs' and 'Trades' after running a backtest on QuantConnect. So the dilemma for me now: which one represents the actual time of events?

Looking forward to your response.

When we define the Period and EndTime, the results, in terms of date/time, are more accurate.

What you are saying is that we should be overriding EndTime and Period?

I want to present one more algorithm I just created and ran. It calls intraday 5-min ES data, also looking for Closing Marubozu. On 2016/01/04 it makes 2 trades. However, the time logged and the trade is at 9:56 instead of 9:55, and 10:16 instead of 10:15. The time is Chicago time. The data is every 5 minutes, for example 9:55, 10:00, 10:05; not 9:56, 10:01 or 10:06. I did not include override EndTime and Period in this last algorithm.

What seems to be the reason for this 1-min discrepancy? Thank you in advance.

Period for a bar tells LEAN when to send the bar to the algorithm. If the bar starts in Jan 1 2001 and has a 1 year period; it won't be released until Jan 1 2002. If you don't specify the period it will be released immediately in 2001. This would mean you could "look ahead" into the future and isn't desirable.

EndTime is calculated with Time+Period; custom data defaults to a 1 minute period.

All data has a period; even instantaneous point data has a period of 0s.

Thanks Jared!

Let me see how I should use your suggestion and I will get back to you.

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

414,100 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

Actions

Join QuantConnect for Free

SIGN IN