Managing Universe and Trades within Algorithm Framework

 From the trade results of this algorithm, long positions are liquidated at the end of the day. You can use the scheduled event to liquidate the existing holdings.

 "Alpha Assets" chart includes all assets in the entire life of the algorithm. It doesn't reflect the current asset holdings.

Hi Jing,

Many thanks for your help.

I have a question regarding a major change between your algo and mine: you selected self.UniverseSettings.Resolution = Resolution.Minute, whereas I was using Daily. I find it hard to understand the timing of all these events.

- When is the UniverseSelectionModel called? Once a day between trading days?

- Also when calling self.Liquidate from main.py, are the stocks sold using the self.execution(ImmediateExecutionModel()) or with a different execution model? Maybe I should schedule a function within the AlphaModel (which I believe runs every minute) for end of day setting InsightDirection.Flat for all symbols in order to make sure it uses the ImmediateExecutionModel.

- I'm getting a lot of "Backtest Handled Error: The order quantity for POM cannot be calculated: the price of the security is zero". How is that possible if one of the conditions from the Universe Selection is price > 20?

Thank you and sorry for all the questions!

Best,

Emilio

UniverseSelectionModel is called every day at midnight 00:00 by default.  

self.UniverseSettings.Resolution determines the requested data resolution for symbols returned by the UniverseSelectionModel. If you use daily resolution here, you only get the daily data for those symbols. Therefore, all market orders will be converted to the MakretOnOpen() orders. When you try to place the market order x minutes before the market close, there's no minute price data because you only request the daily data, you'll get the error message like "The order quantity for XXX cannot be calculated: the price of the security is zero". 

Calling liquidate will bypass the execution model and directly sell the assets in your portfolio; we recommend using a flat insight signal or setting the period of the insight to expire before the end of the day.

Thanks Jing.

Based on your comments, I modified the ConstantAlphaModel accordingly by changing the timedelta parameter of Insight.Price to expire (become flat) 5 hours from time of emiting the insight. As I can see the insight gets created at market open 9.30 and expires at 14.30.

As you can see in the code I set self.UniverseSettings.Resolution = Resolution.Minute in the main script so I should be getting minute data for all symbols now. However, I keep getting the message "Backtest Handled Error: The order quantity for CEO cannot be calculated: the price of the security is zero". I'm no longer using Liquidate either. The only thing I'm thinking could be self.SetPortfolioConstruction(EqualWeightingPortfolioConstructionModel()), which by default has resolution=Resolution.Minute.

Final question if I may, now that I subscribed to Minute for my Universe, all my calculations in the UniverseSelectionModel are based on minute data! I'll have to consolidate, right?

Thank you so much for your help Jing,

Emilio

Hi again Jing!

So based on my previous question, my understanding was that I had to request minute data and then consolidate within the UniverseSelectionModel in order to base my selection on daily data. You can find attached my attempts to do so. I had to comment out all the lines related to the data consolidation as it was giving me errors and could not attach the algo. Error message is Runtime Error: Exception : Please register to receive data for symbol 'A RPTMYV3VC57P' using the AddSecurity() function.....

As you can see I was using something like this within the SymbolData Class:

self.priceSMA10 = SimpleMovingAverage(10)

self.Consolidator = algorithm.ResolveConsolidator(self.Symbol, resolution)

algorithm.RegisterIndicator(self.Symbol, self.priceSMA10, self.Consolidator)

and then simply this within the SelectCoarse function:

for cf in coarse:

if cf.Symbol not in self.calculations:

self.calculations[cf.Symbol] = SymbolData(algorithm, cf.Symbol, self.resolution)

Please let me know your thoughts when you have some time.

Thanks!!

Emilio

UniverseSelectionModel is set to run once at 00:00 every day by default. You don't need the consolidator here as the resolution of UniverseSelectionModel is daily.  "SelectCoarse(self, algorithm, coarse):" method is called every day. "cf.Price, cf.Volume, cf.DollarVolume" those values are all the daily value. cf.Price(raw price) and cf.AdjustedPrice(adjusted price) are all yesterday's daily price. Therefore, all indicators are updated with the daily value. 

"self.UniverseSettings.Resolution = Resolution.Minute" request the minute data for symbols returned by universe model. It means once the universeSelectionModel finish the selection based on the daily price/volume/fundamental factors, those symbols will be returned in fine universe selection

With minute resolution setting in self.UniverseSettings.Resolution, you'll get the minute data in alpha model for those 20 symbols returned in SelectFine(). 

Thanks Jing. I understood now and corrected the algo accordingly.

The attached is using self.UniverseSettings.Resolution = Resolution.Minute, but I'm still receiving the warning message 

"Backtest Handled Error: The order quantity for DOW cannot be calculated: the price of the security is zero" many many times during the backtest!

Emilio

You should add a price check in alpha model to stop sending the insight for symbols with zero price. Otherwise, you'll place orders based on insights combined with ImmediateExecutionModel().

Thanks!!

One quick question Jing please. How can I warm up those indicators within the UniverseSelectionModel so that I can at some point do paper trading live? I'm having trouble figuring that out!

Thanks again!

Emilio

You can request the history price to warm up the indicator manually with indicator.Update(). Please refer to this algorithm 

Please be wary of large requests. Once you get over about 120K data points you'll run into timeout issues. For reference, coarse data set is about 7K symbols, so even 30 days at the daily resolution on all symbols is 210K data points. 

Thanks Jing,

I'm trying the below but get TypeError : object is not callable. I thought it'd be something similar to this:

class UniverseSelectionModel(FundamentalUniverseSelectionModel):

    def __init__(self,
                 universeCount = 20,
                 filterFineData = True,
                 universeSettings = None,
                 securityInitializer = None):

        super().__init__(filterFineData, universeSettings, securityInitializer)
        self.universeCount = universeCount
        
        # Holds our coarse fundamental indicators by symbol
        self.calculations = {}

    def SelectCoarse(self, algorithm, coarse):
        
        '''Performs coarse selection based on price and volume'''
        
        # We are going to use a dictionary to refer the object that will keep the calculations
        for cf in coarse:
           if cf.Symbol not in self.calculations:
                self.calculations[cf.Symbol] = SymbolData(cf.Symbol)

          history = algorithm.History([cf.Symbol], 50)
          self.calculations[cf.Symbol].update(cf.EndTime, history, cf.Price, cf.Volume, cf.DollarVolume, cf.HasFundamentalData)

# Class used to improve readability of the coarse selection function
class SymbolData:
    def __init__(self, symbol):
        self.Symbol = symbol
        self.priceWin = RollingWindow[float](5)
        self.priceSMA10 = SimpleMovingAverage(10)
        self.priceSMA50 = SimpleMovingAverage(50)
        self.volSMA = SimpleMovingAverage(20)
        self.is_uptrend = False

    def update(self, time, history, price, volume, dollarvolume, fundamentaldata):
        self.price = price
        self.volume = volume
        self.dollarvolume = dollarvolume
        self.fundamentaldata = fundamentaldata
        
        for index, row in history.loc[self.Symbol].iterrows():
            self.priceSMA10.Update(index, row["close"])
            self.priceWin.Add(row["close"])
            self.priceSMA50.Update(index, row["close"])
            self.volSMA.Update(index, row["volume"])

        if self.priceWin.IsReady and self.volSMA.IsReady and self.priceSMA10.IsReady and self.priceSMA50.IsReady:     
            SMA10price = self.priceSMA10.Current.Value
            SMA50price = self.priceSMA50.Current.Value
            SMAvol = self.volSMA.Current.Value

            # Here we can add the criteria for our universe
            self.is_uptrend = (SMA10price > SMA50price)

Hi Jing,

So don't worry about the above question as I believe I sorted that out with the below code. Also, following your advice regarding timeout isses, I modified the code so that it first looks at the whole universe and chooses a few based on basic filters and then at the end calculates de indicators only for those securities.

However, now I keep getting this error Runtime Error: ArgumentException : This is a forward only indicator: SMA5 Input: 2015-12-05 00:00:00Z Previous: 2016-01-01 00:00:00Z

Any idea what I might be doing wrong?

Thanks a lot!

Emilio

It looks like you update the indicator with history price repeatedly. You should only request the history to initialize the indicator once for newly added symbols. For symbols already exists in the dictionary, the update method should only update the indicator with new bars.

The above code to initialize the indicator should be out of the update() method.

For example, you can create a method in SymbolData: class named WarmUpIndicator, and then pass the history price to this method and call this method each time you add a new symbol to the dictionary. Once the indicator is initialized for this symbol, this method should not be called anymore. You only need to update the indicator with the current price/volume data in update() method. Your update() method now passed parameters cf.Price, cf.Volume, cf.DollarVolume but didn't use them at all.

Brilliant thanks! Got it to work :)

Thanks Jing,

Emilio