Scheduled Universe Selection Model doesn't add symbols and only removes symbols

Hi Alessio,

The issue is that the algorithm above calls `AddEquity` for the entire universe during the `Initialize` method and sets up a coarse universe selection method. Since it's not currently possible to chain universe selection methods, we need to select just one universe selection technique to resolve the issue. In the attached backtest, we define the universe using the `AddEquity` method. We then create a Scheduled Event to narrow down our universe to the symbols with the largest gaps. Note that when using this technique, the algorithm is still subscribed to all of the securities in the original universe after narrowing down the universe.

In the attached backtest, we've also made the following changes:

• The insights in the `OnSecuritiesChanged` method have been removed. In a framework algorithm, insights should be returned from the `Update` method of the alpha models. 
• The calls to `RegisterIndicator` have been removed because the indicators are manually updated in the `DailyConsolidator` method.
• The Scheduled Event that narrows down the universe has been moved to occur at 10:00AM instead of 10:30AM. When using Scheduled Events with hourly data, the event should be scheduled to occur at the top of an hour instead of in the middle of the hour.

Best,
Derek Melchin

Thank You Derek!

I've seen that it's not possible to chain universe selection, but does it mean that the framework adds the initial list in the Initialize method and then, when the scheduled event is called, it doesn't remove or add symbols?
That could be the problem that is showing up that I can see with debugging that there are removed equities, but no added equities.

I've tried to follow your guidelines (moved the insights inside the update function of the alpha model, and changed the time of the scheduled event to 10.01, not 10.00, because I need the indicators to be updated before the filter is called).

I Registered the indicators because without registering them, they're not warming up before the algorithm starts.

I've attached the new algorithm based on your suggestions, but the problem remains: you can see it by inspecting self.removedSym and self.addedSym.

I can try to solve it by saving the Previous list of symbols in a variable and by checking the differences with the current list (so the alpha model can emit insights only on differences).

Best regards

Alessio

Hi Alessio,

>does it mean that the framework adds the initial list in the Initialize method and then, when the scheduled event is called, it doesn't remove or add symbols?

When the `SelectSymbols` method removes symbols from the universe, the `OnSecuritiesChanged` method is called, but all of the symbols in the `symDict` remained subscribed to for the entire backtest. See the attached backtest for reference.

>I Registered the indicators because without registering them, they're not warming up before the algorithm starts.

When we register the indicators in this algorithm, they are updated twice. They are updated automatically as new daily bars are passed to the algorithm and they are also updated when the `DailyConsolidator` is called. To have the indicators ready when the backtest starts, we just need to change

self.SetWarmUp(TimeSpan.FromDays(max([self.myStdDays, self.fastAverage, self.slowAverage])+ 1))

to

self.SetWarmUp(max([self.myStdDays, self.fastAverage, self.slowAverage])+ 1, Resolution.Daily)

>I can try to solve it by saving the Previous list of symbols in a variable and by checking the differences with the current list (so the alpha model can emit insights only on differences).

Instead, we recommend tracking which of the symbols the `OnSecuritiesChanged` method includes in the `RemovedSecurities` collection.  Then we can emit flat insights for the symbols that `SelectSymbols` has removed and emit Up/Down insights for the remaining symbols. Keep in mind that this technique will only work with a static universe. It won't work with a dynamic universe selection model.

See the attached backtest for reference.

Best,

Derek Melchin

Thank You Derek for the Explanation.

I Was reading your code and I found that You debugged also Splits and Dividends information.

if num_bars != 4:
            self.Debug(f'{num_bars} bars at {self.Time}. Splits: {data.Splits.Count}; Dividends: {data.Dividends.Count}')

Is it possible to pass these information to SymbolData through its update method? Like:

def OnData(self, data):
        '''OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
            Arguments:
                data: Slice object keyed by symbol containing the stock data
        '''
        split = {}
        
        dividend = {}
        
        for symbol, symbol_data in self.symDict.items():
            
            if data.Splits.ContainsKey(symbol):
                split[symbol] = data.Splits[symbol].SplitFactor
            else:
                split[symbol] = 0
                
            if data.Dividends.ContainsKey(symbol):
                dividend[symbol] = data.Dividends[symbol].Distribution
            else:
                dividend[symbol] = 0
                
        if not (data is not None and data.Time.time() <= time(10,0)):
            
            return
            
        for symbol, symbol_data in self.symDict.items():
            if data.ContainsKey(symbol) and data[symbol] is not None and symbol_data.IsReady:
                    symbol_data.Update(data[symbol],split[symbol],dividend[symbol]) #trying to pass splits and dividends
                    
        if self.IsWarmingUp:
            return
   ### ----------- skip some code--------------------###
   class SymbolData(object):
   ### ----------- skip some code --------------------###
   def Update(self, bar, split, dividend): #not receiving splits and dividends

Cause I'm trying but I really don't know which is the best way to get that result.

Best regards

Hi Alessio Magri,
In the above code snippet, the Splits and dividends are passed to on data method during midnight, and the trade bar is passed from 10 am to 4 pm; therefore, the update method does not receive information about split factor and dividend distribution. We recommend implementing a separate method to receive updates about splits and dividends(in the backtest Update1). Refer to the attached backtest.
Best,
Varad Kabade