TLDR: Trust QuantConnect, imo.

Hi Goldie Yalamanchi ,

I'm ramping up with QC, but I skimmed the code and took a look at this in CMLViz. 

At first I thought the discrepancy might be because we are using intraday data on QC (with minute resolution), but with CMLViz we only have a single snapshot of the option chain for the each day (daily resolution) 

So I backtested the strategy on OptionStack (OS), which also has daily resolution, and my OS results are closer to the results we are seeing in QuantConnect.  In my image below, you will notice that the strategy does okay and then hits a big drop in August. Just like our QC strategy. 

There could be a few reasons why CMLViz does not have this drop, but it’s hard to tell what they are doing behind the scenes.

It’s my first time using CMLViz, and the first thing i noticed is that there isn’t much granularity on how to pick strikes. Our vertical consists of a 70 delta and 80 delta, but there are times in an option chain you might not have exact deltas that match. For example You my have 69 delta  and 71 delta. but not exact ’70’. In this scenario, what strike does CMLViz pick? Do they just skip the trade? We don’t know, and that can make all the difference. 

In options backtesting platforms like OptionStack and ORATs (they both also only have daily option chain data), you specify a range. For example, in OptionStack, for the 70 delta, i put in a range of 69 - 70 delta (instead of just 70), and I specified that if there are multiple matches within this range, then select the best strike based on max return. 

In QC, we have even much more control / granularity where we have minute resolution for our option chain, and at different times of day our option chain can look very different, but we can pick exactly what we want, with custom logic.

My advice would be to trust QC, where you have more data, and more control.  However, if you really want your QC and CMLViz backtests to match, try to find out what exactly CMLViz is doing under the hood, so you can replicate it in QC.