The right way to calculate volitality

QuantConnect Community Discussions

QUANTCONNECT COMMUNITY

Join Our Discord Channel

Join QuantConnect's Discord server for real-time support, where a vibrant community of traders and developers awaits to help you with any of your QuantConnect needs.

Draft Discussions

Bookmarked Discussions

The right way to calculate volitality

Last comment by - June 2020

The right way to calculate volitality

Asking for confirmation on calculating volatility in QuantConnect using STD function, and if there's a better method.

Share New Research

Start New Discussion Sign up

SEARCH DISCUSSIONS

366,900 Quants.

Become a Quant

VOTE FOR UPCOMING FEATURES

Share your input and vote on our future direction.

LEAN Roadmap

JOIN OUR Community MAILING LIST

Create an account on QuantConnect for the latest community delivered to your inbox.

The right way to calculate volitality

Travis Fu | July 2015

I know this is sort of basic and should be well addressed, but is there a well defined method to calculate volatility?

This is how I currently do it:

var dailyReturn = ROCP(symbol, 1, Resolution.Daily);

var volitality = STD(symbol, 84).Of(dailyReturn);

I first define a variable called daily return, and calculated 84 days (trading days in 4 months) of standard deviation of it. However it is far from what I manually calculated from the prices. I know the numbers may be not exactly the name (Annualized, not annualized etc.) but the shape of the data curve is totally different. Can anyone confirm if this is the right way to calculate vol in QC? or if there is a better way to do that?

Thank you!

Author

Travis Fu

July 2015

Upvote

Jared Broad INVESTOR

QuantConnect | July 2015

There is no one way to calculate volatility from what I've seen. We used the deviation in daily performance over 252 days to calculate the vol for sharpe ratio formulas. If you have a specific case you'd like help with please share an algorithm or refer us to a project-id and we can help.

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Upvote

Michael Handschuh INVESTOR

July 2015

Hey @Travis, You'll need to give us some more details as to what method you're using to compute it manually. The code as you described will compute the standard deviation of the previous 84 days percentage change. Maybe you should be using log returns instead of ROCP?


    public class LogReturn : WindowIndicator
    {
        public LogReturn(string name, int period)
            : base(name, period)
        {
        }

        public LogReturn(int period)
            : base("LOGR" + period, period)
        {
        }

        protected override decimal ComputeNextValue(IReadOnlyWindow window, IndicatorDataPoint input)
        {
            decimal valuef = input;

            decimal value0 = !window.IsReady 
                ? window[window.Count - 1] 
                : window.MostRecentlyRemoved;

            return (decimal) Math.Log((double) (valuef/value0));
        }
    }

Upvote

Travis Fu INVESTOR

July 2015

Hi @MichaelH, Thank you very much for providing this snippet, would you pointing out a way if I want to apply this in the online version of QC? I am sure it is easy to just add this in the indicator class in Lean project on my local machine, but it would be great to know how to extend the capability online. Thank you Travis

Upvote

Michael Handschuh INVESTOR

July 2015

Hey @Travis, I've attached an example of creating your own indicator (LogReturn) and registering it for automatic updates. I've also included some code to plot the value. The curve here seems more typical of a volatility curve. I hope this helps!

Upvote

Michael Handschuh INVESTOR

July 2015

Here's a comparison of the two techniques. They seem to have similar shape in my tests. Let me know what you guys think.

Upvote

Michael Handschuh INVESTOR

July 2015

Hey @Travis, one more thing... It looks like you've named your indicators all the same thing! You pass in 'symbol' as the name of the indicator. I'm not sure how you're generating the plots, but some of the plot helper functions use the name of the indicator for the plot, so if there are multiple indicators with the same name you may end up with very unexpected results. Maybe this is why my recent share above doesn't suffer from the jitter you described. In the future it's easiest for us if you share an algorithm which demonstrates the problem, it will greatly improve our ability to help you. Thanks!

Upvote

Travis Fu INVESTOR

July 2015

Hi @MichaelH , Thank you for your replies and suggestions! However, I still have questions about your code, I see your code provided above generate nothing. I am a little confused, clearly you can see the shapes on your machine, but I couldn't see any plot, I just need to compare the plot you generated and compare with the manually calculated vol shape in my excel to confirm Thanks for pointing out the same "symbol" name, actually I have a ticker list, and I just have a loop go through them and variables generated by "symbol" will be passed to the tickers in the list individually. Greatly appreciate your help! Sorry didn't reply promptly, was busy at work, but I will work on this this weekend. Travis

Upvote

Michael Handschuh INVESTOR

July 2015

@Travis, did you clone and run the backtest to look at the plots? The chart shown in this thread is the equity chart, which is flat because I didn't place any trades. Clone the algorithm and look at the plots. There's a volatility plot and a returns plot which computes the vol using ROCP and LogReturns indicator I've previously provided.

Upvote

John Radosta INVESTOR

June 2020

There are various ways to measure volatility, it just matters if you're trying to estimate current volatility or future volatiltiy, as well as realized or implied volatility. Let's look at the indicator route first.

For current realized volatility, you could certainly use STD (which is essentially also what Bollinger Bands uses), Keltner channels which provide volatilty in absolute pricing with a moving average, or you can even go the statistical arbitrage route if you have asset cointegration, measured by the divergeance between one or more other assets (this is how Renaissance fund is rumored to do it). ATR is also another one if you want just the absolute pricing.

For predicting future realized volatility, you can do that surprisingly accurately using the GARCH method (link to paper is below), but as mentioned in many talks on YouTube, realized volatility is not what ultimately determines asset prices, implied volatility does, and there is no known accurate way to predict future implied volatility. If you find it though, well, you'd be rich :-) Sidenote: this is why options traders are always talking about implied volatility, because implied volatility is a derivative (calculus) of asset price: the underlying asset price is baked into it.

The good news is that the GARCH method has been shown to be a better predictor of future realized volatility than even the VIX, so I'd go that route if you really want to get your hands dirty.

hec.unil.ch/agoyal/docs/Garch.pdf

Upvote

Derek Melchin INVESTOR

QuantConnect | June 2020

Hi John,

Thank you for providing a research paper for users to learn more about GARCH models. I noticed the link to the paper returns a 404 error though. Here's a working link for those interested in learning more: www.hec.unil.ch/agoyal/docs/Garch.pdf

Best,
Derek Melchin

Upvote

1 person upvoted this

Tristan F INVESTOR

June 2019

Here are other thoughts to gauge the value of trading and screen out "Buy APPL in 1998" strategies. This will take a few posts, my apologies in advance.

The two primary ways that can make active trading more profitable than buy-and-hold are:

Choice of assets
Trading pattern (buy and sell decisions)

Choice of assets

Here you would want to compare the assets you selected to an appropriate benchmark. The traditional CAPM model could provide an alpha that shows the value of the chosen assets in excess of the benchmark and a risk free rate. The difficulty with this is that you would need to automate the selection of the benchmark, which is somewhat subjective and therefore difficult to do.

Since we are mainly interested in backtesting and have the benefit of 20/20 hindsight; it's very easy to select the best assets in the rear view mirror. The choice of assets is therefore not as valuable as a backtest screen, and probably doesn't follow the intent of the volume factor anyways. For now, let's put this one aside and focus on of the trading pattern.

Trading Pattern

A simple approach to look at the value of trading is to compare desired metrics between an active trading strategy and a buy-and-hold strategy with the same assets held at the average allocation from the trading model.

A simple example: we trade SPY using the traditional 200 day moving average, if SPY is above its 200MA, we buy; if it's below; we sell and buy IEF (10 year treasuries). The following backtest shows results from 1/1/2009 to 12/31/2018. From the overview, and with rf = 0; the RoMaD = 10.757%/17.3% = 62%. From the results report, the average allocation is 79% SPY and 21% IEF. I can't find the Sortino anywhere on the output, but let's assume the negative deviation is 9%; and the Sortino ratio is therefore 10.757/9 = 1.2.

Upvote

Tristan F INVESTOR

June 2019

Now we take the average allocation from the active trading strategy (79% SPY and 21% IEF) and see how things look with a buy-and-hold strategy. We get a RoMaD of 11.454%/22.1% = 52%; and an estimated Sortino of 11.454/10.5 = 1.09.

Upvote

Tristan F INVESTOR

June 2019

We now need to determine the value of active trading. We'll ignore the tax impact of turning over the portfolio since our target customers are hedge funds and not individual investors.

Active Strategy:

Annual Return: 10.76%
Negative Std Dev: 9%
Max Draw Down: 17.3%

Passive Strategy:

Annual Return: 11.45%
Negative Std Dev: 10.5%
Max Draw Down: 22.1%

The idea is to understand what risk-adjusted value is produced by active trading. The Modigliani risk-adjusted performance looks at the expected return of oe investment if it is scaled to the same risk as another investment. For example, if we look at the expected return of the active trading for the same negative standard deviation of the passive approach, we get 10.76% * 10.5% / 9% = 12.5%. In other words, if we took the same level of risk as the passive approach, we would get an expected return of 12.5% on the active approach, or a 1.05% higher return.

This represents the change in the expected return generated by the trading strategy for the same level of risk as the buy-and-hold strategy. We can do the same thing for the RoMaD; the only difference is the risk metric (max DD instead of neg SD). In the example above, we would get an expected return of 10.76% * 22.1%/17.3% = 13.74%, or a 2.29% higher expected return from the active trading strategy for the same maximum draw down.

The fitness score could conceivably be equal to the average of these two changes in expectation, in the example this would be (1.05%+2.29%)/2 = 1.67%. If this is greater than a threshold (1%?) then the algorithm makes the alpha stream, otherwise it is rejected.

Pros:

Measures the risk-adjusted value of the trading activity, and not the underlying assets
- Think of this as a “trading alpha” – the return that is generated by trading
- The growth of the underlying assets is independent from the trading alpha, so buy AAPL in 1998 would have a trading alpha of 0
Compares risks at the average level (Sortino) and the extreme (RoMaD), like the original metric
Works for longer (say monthly) or shorter term (HFT) trading strategies
- May favor more frequent trading because there are more opportunities for the trading alpha to develop if there are more trades

Cons:

Requires post-run calculations (buy-and-hold backtest)
More difficult to test systematically and therefore to know where a good threshold might be
- However, since this measures an alpha (annual return), a threshold based on judgment from QC, such as 1%, is justifiable
Doesn't look at absolute returns but rather the additional returns generated by trading

Upvote

Douglas Stridsberg INVESTOR

June 2019

Hey Tristan,

Couple of thoughts regarding your proposed approach:

Unless I'm mistaken, this is just comparing the Sortino of alpa with Sortino of benchmark (where benchmark is average holding weight of all assets), and saying the Alpha's has to be at least some ratio of the benchmark's (>1.01)?
I see this working with a very specific set of strategies - namely long-only equity strategies. What about long/short strategies or even market-neutral ones, where your benchmark might very well end up having negative returns or zero weights, meaning your score would be wildly inflated compared to a traditional long-only score? What about other asset classes?

Upvote

Tristan F INVESTOR

June 2019

Douglas,

Good points. Here are some thoughts:

The ratio of Sortinos is a bit different because the result would be dimensionless. From the prior example, the Sortinos are 1.2 for the active strategy and 1.09 for the passive. If we compared the two we would get 1.2/1.09 = 1.1. I prefer calculating a trading alpha instead because it gives you a percent return, something that is easy to interpret
I'm not sure why this couldn't be used with long-short strategies or alternative assets. For a long-short, the benchmark would have negative allocation for the short assets, i.e. you go short and stay short for the duration of the backtest. Also, I don't know how this would get wildly inflated. For this to go wild, the risk measure (e.g. max draw down) of the passive strategy would have to be much larger than that of the active strategy. The formula is simply:

trading alpha = ra * sp/sa - rp

where:

ra = annual return of active strategy
rp = annual return of passive strategy
sa = risk measure, e.g. negative standard deviation or max draw down, of active strategy
sp = risk measure of passive strategy

Upvote

Douglas Stridsberg INVESTOR

June 2019

Hi Tristan - I see now that my approximation to ratios of Sortinos isn't quite what you meant, my apologies. I can't quite see a case where the measure would get wildly inflated.

Again, however, I get the impression you're thinking in terms of equities, where indeed a long/short portfolio would typically long some companies and short others, thus ending up with a meaningful measure.

Let's say instead I'm trading a basket of G10 currencies and my active strategy is such that the average weights over the period is 0% across all currencies. rp = 0, sp = 0 - giving a measure of 0 even though the strategy might be profitable.

My point is that the concept of a benchmark is complicated when you start to look outside of equities, where assets often don't have a positive drift and instead act in a more mean-reverting fashion. That's before we even start to talk about options, which brings a wholly different set of problems.

Upvote

Jared Broad INVESTOR

QuantConnect | June 2019

Just throwing this into the discussion: our aim is to remove our human bias so ideally, the fitness function should be automated. If we need to manually construct representative passive portfolios it will induce delays and costs in the evaluation process. If "the community" on a whole has factors they believe should be considered please also sanity check it can be automated!

Upvote

Tristan F INVESTOR

June 2019

Jared,

The intent was to automate the benchmark, which could've been done relatively easily, so that an alpha that measures the value of the trading activity could be determined. However, as Douglas points out (thanks Douglas), this method is not universally applicable, so it's value may be limited.

Upvote

Tristan F INVESTOR

June 2019

Jared,

Would you consider lowering the threshold on the trade factor like you mentioned before? Maybe requiring full portfolio turnover every month to get a score of 100%? That would mean average turnover of 10% per trading day under the current metric.

Thanks.

Upvote

Li li INVESTOR

June 2019

I have a suggestion. For leading us to improve Alpha Ranking, the system should present base performance correlating with Alpha Ranking in Overview page in backtest. Such as backtest year, daily portfolio turnover, annual return, max drawdown, annual std of negative return. Now backtest year, daily portfolio turnover, annual std of negative return are absent.

Upvote

Link Liang INVESTOR

June 2019

Hi Li,

Thanks for your suggestion, we will take it into consideration.

Upvote

Aliou Ba INVESTOR

June 2020

Thank you

I have concerns with Sortino and RoMaD. My model does not do HFT but it takes on average 282 positions per day on the mini ES contract against an average of 19 in live trading. On a 5 year backtest with a resolution. .Tick, the results are abnormally high but the backtest fails because of the valuation mode of the Sortino and RoMaD due to the high volume. I found that limit orders are instantly executed if the price is hit. I also noted orders executed at prices that the ES has never reached since its creation. For those who do HFT, is a 3 month backtest enough to submit an HFT model in the Alpha stream market?

Upvote

Jack Simonson INVESTOR

June 2020

Hi Aliou,

Although we no longer use the Fitness Function proposed in the research notebook at the beginning of this thread, 3-months is, unfortunately, not a long enough backtest to submit. Alphas need to demonstrate consistent performance over a long period of time. Funds want to be able to see its performance in a variety of market regimes and a 3-month backtest isn't sufficient for this. You can find details about the Alpha submission criteria here.

Upvote

1 person upvoted this

Platform

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

366,900 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

Actions

Join QuantConnect for Free

Platform

SIGN IN

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

366,900 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

SHARE RESEARCH

SHARE DISCUSSION

SHARE ARTICLE

SHARE

Actions

Join QuantConnect for Free