How To Keep an Updated Option Universe

You should manage the options subscriptions at OnSecurityChanged method.
Remove the subscription from the universe selection method:

And add it to OnSecurityChanged:

Note: I haven't tested this code yet. It is a suggestion that you can work on instead of waiting for a solution.

This looks great, but I just put a debugging message at the top of the OnSecurityChanged function, and this function isn't being triggered at all.  So it ends up not adding any options to the universe.

OK yeah sorry I noticed you slightly mispelled the function name, it should be OnSecuritiesChanged.  The other part with a syntax error is this line:

The variable s isn't defined.  I figured that was supposed to be x but changing s to x resulting is no securites ever being removed from the universe.  I think I'm almost there.

Sorry, there are typos: OnSecuritiesChanged and the s for x in line 12. (I edited the original)

So the code seems to be working at expected.  I put Logs in there stating which securites the algo is trying to remove, and that code is executing, but you can see but the counter printout, every time through the OnData trading code (once per week) the number of option chains in the slice.OptionChains object keeps increasing... 

hi did you ever find the solution to stop the explosion of the ever incresing slices?

I am trading straddles on earnings with a calender from dropbox. Running it for longer then a month and all sorts of slowness happens including the backtest repeating itself 2-3 times. if i run the same backtest in one month batches no such issues and quick enough.

Hi Farhan,

Without seeing your code, there could be a number of things that are contributing to slow backtesting speeds. However, I've attached a backtest that is a version of Tim's code and helps demonstrate how options are added and removed from the Universe so that there isn't an explosion in the data Slice. As options expire, so long as the portfolio is not invested in them, they are removed from the data Slice and no longer present as an Active Security, but the attached algorithm explicitly logs this information and ensures removal by modifying the OnSecuritiesChanged() method. Hopefully, this helps, but if you post more information and your algorithm code, then we can better pinpoint the source of your speed issues.

thanks for the pointer. Ive tried removing and adding equities/options all over the place and various universe methods. I am actually looking to update the universe once a day around 1 hour before close and enter the positions. Then the next day approx 1hr after open close the positions. 

The explicit log helped a lot and ive managed to get it to manage the universe as I wanted with a few work arounds but ive now caused this problem and lost all my daily logs with any run longer then a week.

Warning: The AA equity security was set the raw price normalization mode to work with options

I do have a function to deal with this that is invoked on the intial adding the securites to the universe. So i think im still removing from universe badly.

Attached is the backtest. its a sprawling mess from experimenting but ive removed the options stratgies and tried to make it digestable.

Hi Farhan,

I ran you code a number of times and I can't reproduce the error you are encountering. These warnings aren't a breaking error so this shouldn't be the cause of any runtime trouble you're encountering, but you can consider using OptionChainProvider to add Options to the Universe instead. Additionally, I checked to see what securities were in the Universe before and after it is updated, and Symbols are being removed just fine. Is there a specific error message or stack trace error code that you're seeing?

You can see how I logged the information in the backtest I've attached, which is essentially just the bare skeleton of your code.

hi maybe i wasnt clear then. It does seem like it is working fine from a runtime POV but now my logs get filled with this

 Warning: The AA equity security was set the raw price normalization mode to work with options. 

This warning starts happening after the securites are removed.  

When the securites are initially added then the funciton set raw price normalisation mode is invoked correctly.

So if it is all fine then please can I switch of the warning message from the logs so i can use them again? 

i just tried your skeleton, it exhibits the same issue.

here is a screenshot fo the logs.

another thing is i did look at Optionchainprovider when i orignally looked at how to use options and becuase i use the greeks (removed from the code i posted) it seemed like the wrong choice according to this thread 

I have encountered, with "vol complex eg. VXX" options multiyear backtests, incorrect greeks at  certain points and this mean it is likely I will code a local option pricer to get the greeks,, so maybe I will give the optiochainprovider algorithm in combination with export to a DF and calculate the greeks myself a go.

Hi Farhan,

We apology for your late response. Firstly, OnSecuritiesChanged() method is a handler for newly added/removed securities. Therefore it is not recommended to add new securities in it, as it would break the workflow of our event handling system and cause unexpected behaviors. The direct cause of those warning messages comes from line 195 in your shared code.

The greeks should be calculated correctly based on the option price model of your choice. And you are always welcome to build customized calculation method on your own.

In addition, we are not fully supporting dynamic option universe (add new options at runtime), therefore we cannot foresee its behavior. Here and here are some information about how to add options in initialization and access their data later in your algorithm.

hi ok thanks i see. My working options backtester version (not for earnings)  uses the direction of your links and is used for a much lower number of securities and works nicely with various methods of selecting options and then making spreads. e.g deltas.

thanks for clearing it up for me wrt dynamic universe

ps. The greeks in general do calculate correctly but on multi year tests ive done there are some minor gaps. When i get back to that testing, i will capture, double check and raise an issue if it is real. -- i got sidetracked by earnings

Using Tim's code, It was able to gather a universe and then add the options in the OnSecuritiesChanged, but what I don't understand is how do you make orders using the options within that universe?

It looks like in the code, it just makes a market order of some parameters in an option chain, but not the options within the particular universe we created in the CoarseSelectionFunction.

Hi Young,

Thank you for your question. Please let me explain the OnSecuritiesChanged event. It is fired only when universe contents are changed (securities are added or removed from the algorithm). It means that all the securities in OnSecuritiesChanged event are filtered from Universe selection. In Tim's algorithm, he only does AddOption() for these securities. Then, the corresponding options are passed to the slice.OptionChains in OnData method. Therefore, only the options within that universe will be traded.

For more information on OnSecuritiesChanged or Options, please visit these two pages. Hope it helps!

Thanks for the response Daniel

I was trying to work on it a different way, but keep running into issues saying that there is a "|" or space in the value of the equity symbol

Its on line 101, where i try to addoption()

Looks like i may have posted a slightly different version of it.

Below is the revised part where it had "GOOG" as the symbol for the AddOption function.

I added security.Symbol.Value instead to add all the securities recently added to the universe, but it is showing a runtime error saying that there is a "|" or space in the value. 

What is another way of expressing the symbols of the securities so that the AddOption function will take it?

Hi Young,

Thank you for the follow-up. By running the backtest, I can replicate the error information you had. In fact, this error happened because the option will be added to the *self._changes.AddedSecurities* as well. So if we use self.AddOption(security.Symbol.Value) when security.Type is Option, the error will happen, since for method self.AddOption() the security should be the underlying rather than the option itself.

After we detect which part causes the problem, we can take advantage of one property of security objects, that is security.Type. When security.Type != SecurityType.Equity, it means that the security is Option, which we want to skip for self.AddOption(). Therefore, I add the following code snippet in your backtest to avoid this case:

 

Then the error will no longer exist! Also, I add the adjusted backtest below. Hope this helps, and we look forward to your good news!

Is using AddUniverseOptions() now the recommended way to do this?