Hello my beloved community,
I need serious help that may be easy for someone moderately well endowed in Python to help me out with. I am a beginner who is new to universe selection and doesn't understand classes very well.
Questions - I have no idea to how to make OnData, Consolidated functions, Scheduled events, inclusive to my universe that is changing every day with a different amount of symbols potentially each day. I am very new
Strategy - I want this algorithm to filter by volume, and then by which of those stocks have had earnings report recently, then some sort of buy logic.
Buy Logic - The buy logic contains a consolidated data function where two variables are assigned every day. The OnData method is going to check price every minute relative to the two variables declared shortly after market open
Here is the code just in case it does not show up without a backtest attached.
#region imports
from AlgorithmImports import *
#endregion
class EarningsORB(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2023, 2, 2)
self.SetEndDate(2023, 2, 5)
self.SetCash(100000)
# Scheduled events
self.Schedule.On(self.DateRules.EveryDay(),self.TimeRules.AfterMarketOpen(60), self.RangeToggle)
self.Schedule.On(self.DateRules.EveryDay(), self.TimeRules.AfterMarketOpen(), self.UpdateUniverse)
# Universe Selection
self.AddUniverse(self.MyCoarseFilterFunction)
self.volumeBySymbol = {}
self.symbols = []
# Toggles
self.range_toggle = False
# Consolidated data
# Sixty Min Bars
sixty_min_consolidator = TradeBarConsolidator(timedelta(minutes=60))
self.SubscriptionManager.AddConsolidator(self.symbol.Symbol, sixty_min_consolidator)
sixty_min_consolidator.DataConsolidated += self.OnSixtyMinBar
self.sixtyMinBarWindow = RollingWindow[TradeBar](1)
def MyCoarseFilterFunction(self, coarse):
# Price above 10 and dollar volume above 10mil
filtered = [ x for x in coarse if x.Price > 10 and x.DollarVolume > 10000000 ]
# Adds to VolumeBySymbol Dictionary
for x in filtered:
if x.Symbol not in self.volumeBySymbol:
self.volumeBySymbol[x.Symbol] = SymbolData(x.Symbol, self)
self.volumeBySymbol[x.Symbol].Update(x.EndTime, x.Volume)
# Filter based on recent earnings report - This is where the filter logic would go, this is example code, it is not correct
# Maybe this is where the 'estimize' code is supposed to go?
self.symbols = []
for symbol in self.volumeBySymbol.keys():
last_report_date = self.Fundamentals.GetLastReport(symbol).ReportDate
if (self.Time - last_report_date).days <= 1: # Only include symbols with earnings report in the last day
self.symbols.append(symbol)
# Log
symbol_strings = [str(symbol.Value) for symbol in self.symbols]
self.Log(f"Symbols with recent earnings reports: {symbol_strings}")
return self.symbols
# Scheduled events
def UpdateUniverse(self):
# Refresh universe
self.RemoveSecurity(self.symbols)
self.AddUniverse(self.MyCoarseFilterFunction)
def RangeToggle(self):
self.range_toggle = True
# Consolidated Data
def OnSixtyMinBar(self, sender, bar):
self.sixtyMinBarWindow.Add(bar)
if not self.sixtyMinBarWindow.IsReady:
return
# VARIABLES - HIGHS - LOWS
trigger_bar = self.sixtyMinBarWindow[0]
if self.range:
self.trigger_sixty_min_open = trigger_bar.Open
self.trigger_sixty_min_close = trigger_bar.Close
self.range = False
# Buy Logic
def OnData(self, data):
# Check buy criteria every minute
for symbol in self.symbols:
# Enter long when it crosses sixty min high
if self.Securities[symbol].Price > self.trigger_sixty_min_close[symbol]: # am i supposed to add [symbol] there? Do i add [symbol] anywhere else? Like in the consolidated data function?
self.SetHoldings(symbol, 1 / len(self.symbols))
# Symbol Data class
class SymbolData:
def __init__(self, symbol, algo):
self.algo = algo
self.symbol = symbol
self.sma = SimpleMovingAverage(30)
history = algo.History(symbol, 30, Resolution.Daily)
if not history.empty:
for time, row in history.loc[symbol].iterrows():
self.sma.Update(time, row['volume'])
def Update(self, time, vol):
self.sma.Update(time, vol)
Thank you for taking the time to read this and maybe help me out as usual.
Best,
Jesse
Jesse Fleming
I forgot to mention,
I have the EPS data Estimize purchased. To add one more thing to the list of not knowing how to make universe friendly, I do not know how to add the Estimize documentation to this algorithm in the way I need. For example, do I add the Estimize lines of code in my coarse filter? Do I add it in the OnData method? I have no idea.
Nico Xenox
Hey Jesse Fleming,
If you want to sort equities after their rearings report you can add a fine selection:
This will sort all the equities after the day of the earnings report. This means you don't have to necessarily use the estimize dataset.
Also when doing scheduled events add an equity like SPY so that it works:
You can't schedule the universe selection, that will happen automatically at 00:00 so you can delete that schedule.
Also to add consolidators to each symbol you will have to put it into the symbol data, something like this:
Note that you will have to put the consolidation function into the symboldata class.
Also before being able to consolidate data of an equity, the equity has to enter the selection. This means the OnSecuritiesChanged could look something like this:
Best,
Nico
Jesse Fleming
Nico Xenox Another brilliant and timely response as usual, my friend.
So of course your code is undoubtedly working, now forgive me for being so green, but I am really struggling with adding MACD indicator to the algorithms that have universe selection in them. The warm up confuses me and updating it confuses me, I think once I start messing around with SymbolData class I am out of my comfort zone a little bit.
Does anyone know how I would go about doing this? Even just a skeleton outline of what that looks like would be so useful.
Thank you for everything you do for the community brother,
Jesse
Algo Trader
I also am struggling with this myself!
Jesse Fleming
For instance,
Why would this line of code:
result in the following error:
Best,
Jesse
Nico Xenox
Hey Jesse Fleming,
to add the MACD indicator you could do something like this:
Here's a post about the other problem you have:
Hope it helps ;)
Best,
Nico
Jesse Fleming
Nico,
with your excellent advice i have come close to completing the algorithm. I have managed to add the MACD indicator on five minute resolution which is great.
But now there is a massive warm up problem i am trying to solve. It appears that MACD values are not correct until 26 bars pass. I need the MACD indicator to be warmed up at market open so that it is giving accurate values immediately as i do not have time to not take trades for 26 bars. Can you please help me solve this one last issue?
Here is the code:
Nico Xenox you are the best,
thank you for your time,
Jesse
Nico Xenox
Hey Jesse Fleming,
I think that this is happening because it is not updating the history values as consolidated data. For this the SymbolData class probalby should be changed like this:
And this part of the code in Onsecuritieschanged can be deleted:
Hope it helps ;)
Best,
Nico
Jesse Fleming
Nico Xenox your idea of moving everything to SymbolData Class was correct and wonderful insight.
Here is the problem I am having now. the problem is a symbol appears in the watchlist almost every day, but there is only a couple trades despite the only requirement being that the price breaks the first twenty-five-minute high or low. i know Estimize is doing its job because i am logging the symbols entering the watchlist and they are all correct on the correct days.
I really should be in a trade in all of these days there is a symbol in the watchlist. After doing some logging, i noticed that the symbols we do not trade cannot get passed the check if symbol_data is None or not symbol_data.warmed_up: continue
here is the code for reference
Thank you for all of your help! I am infinitely grateful,
Best,
Jesse
Jesse Fleming
And now there is another weird thing happening where all the orders happen right at 12:20 instead of when they cross the high/low. I don't know what happened I am just telling you everything for as much context as possible. Clearly there is something seriously wrong with the way i am grabbing data.
Jesse Fleming
In regards to the 12:20 orders, for some reason when I take this block of code:
And turn it into this for this check:
But the problem is that I made that change in response to an error about mKey or oKey or something like that.
Again, the main problem is that symbols in Watchlist cannot get passed the check:
in OnData
Best,
Jesse
Nico Xenox
Hey Jesse Fleming,
it took some time but I found out what the problem was. I had to change this:
Into this:
Hope it helps ;)
Jesse Fleming
Hey Nico Xenox , first off, I really appreciate you dedicating and putting so much time into this issue. I am beyond grateful. I am afraid to say I still have the same problem.
Below, I have attached the algorithm with the lines you advised me to swap out with your new code. I have added three logs, the first two stating that symbols have been skipped if they do not pass the check, as well as a third log that would symbolize an entry in some way, if the algorithm is working the way it is supposed to.
As you can see from the backtest, there are many symbols still being skipped, and even the ones that are not skipped, still do not trigger the ‘crossed range’ log. I even lowered the range to two minutes to emphasize the potential issue.
I truly do not know what is going on, why would it skip these symbols and why would the ones it does not skip not cross the two minute high/low. If this works the way it should, it really should not skip any symbol and every range should be crossed at one point of the day.
I patiently await your response as you are my only source of help. Thank you. Someone give this guy a raise already. I hope I responded fast enough to catch you before your hard day of work is over.
Best,
Jesse
Nico Xenox
Hey Jesse Fleming,
sorry I was very busy. So I saw that you skip most symbols. Did you check when the release date actually gets released? Because currently it seems that these dates get published only a few hours/minutes before the actual date. This might be a problem that you're facing and because of that no stocks enters the “cross range”.
Best,
Nico
Jesse Fleming
Hey Nico Xenox ,
I really appreciate you getting back to me despite you being so busy. And yes I am aware that the dates are published only moments before the actual date, they are all before market open, but I do not know why that would matter? It gets added to Watchlist on the correct day, MACD is getting warmed up instantaneously, and then we begin grabbing the minute highs and lows for the range.
What makes you say there is a problem with the date getting published before market open?
How can we fix this? Thank you for everything and I am looking forward to your response. I am lost.
Best,
Jesse
Nico Xenox
Hey Jesse Fleming,
the price was wrong and so the price was the whole time the same number. Also I had to add another piece of code which requested for each security the last known price.
I also had to change this:
to this:
I also changed a little bit of code because there was a weird mechanic with the open and close market. Maybe try to debug the code to see if it should be like this.
Anyways the code now crossed the log function with **cross range**.
Here's the log:
If you're satisfied, don't forget to accept my answer 😊
Best,
Nico
Jesse Fleming
Nico Xenox ,
I am sure at this point you are as frustrated with this problem as I am. I have been banging my head against the keyboard for quite some time.
This is precisely the problem. How is it that all of these symbols had earnings and only one of them crossed the two minute range in this case? GS is the only stock that crossed its two minute range? That is not correct. You can assume all of these stocks at some point in the day cross their two minute range because that would be insane if they did not. I have also verified on TradingView that this is clearly not the case.
I took your code and added a log in the first check to understand where the symbols are being skipped over.
Here are the logs:
Nico Xenox
Hey Jesse Fleming,
you're right. I spent another few hours trying to debug the code only to find out that you were reseting the symboldata information… Thats the reason why it wasnt working. Not only were you reseting the watchlist but also the class that held all the information of indicators.
Also dont use schedules and dictionaries to flag if the exchange is open. Just use this:
This will return a bool.
Also I reached the limit of logging, but this is what I got for the first ones:
Please consider closing this discussion 😊
Best,
Nico
Jesse Fleming
Nico Xenox
You have once again out done yourself.
Derek Melchin, look at this thread when you get a chance, this guy stuck with me 18 times back and forth not giving up on this weird problem. After his long hours he came back to put in even more time to figure out this obscure issue.
Nico, thank you brother for sticking with it and helping me out.
Appreciate you man,
Best,
Jesse
Jesse Fleming
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!