Share New Research 
hi everyone,
Im looking for a way to get the short interest on specific stocks or on a universe of stocks. The data would likely come from this here:
https://www.quandl.com/data/FINRA-Financial-Industry-Regulatory-Authority
The this is, I couldnt find any example on QC's website on how to pull that kind of data.
An example would be greatly appreciated.
thanks !
Jack Simonson
Hi Stephane,
This is definitely an interesting data set! You can find a few demonstration algorithms for using Quandl data in GitHub, but I've attached a backtest below demonstrating how to import the FINRA/NASDAQ TRF Short Interest: JUST dataset from the Quandl link.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Stephane b
Hi Jack,
I just wanted to thank you for your very appreciated contribution.
I will definitly have a look at it shortly and might post my findings here (if they are worth mentioning) to help other people as well.
Thanks again!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Sudheer Kumar
Above code doesn't work anymore and gives below error, what am I missing?
Error invoking FINRA/FNSQ_JUST.QuandlFINRAData data reader. Line: QECx02,You have submitted an incorrect Dataset code. Please check your Dataset codes and try again. Error: String 'QECx02' was not recognized as a valid DateTime. (Open Stack Trace)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Ashutosh
Hi Sudheer Kumar
On Dec. 4th, 2018, Nasdaq announced it had acquired Quandl, Inc. In 2021, Quandl was replaced by Nasdaq Data Link.
QuantConnect has decided to remove Quandl from Lean, and create an independent repository to NasdaqDataLink.
You can refer to the dataset document for implementation using the Nasdaq Data Link with examples.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Tristan F
Here are other thoughts to gauge the value of trading and screen out "Buy APPL in 1998" strategies. This will take a few posts, my apologies in advance.
The two primary ways that can make active trading more profitable than buy-and-hold are:
Choice of assets
Trading pattern (buy and sell decisions)
Choice of assets
Here you would want to compare the assets you selected to an appropriate benchmark. The traditional CAPM model could provide an alpha that shows the value of the chosen assets in excess of the benchmark and a risk free rate. The difficulty with this is that you would need to automate the selection of the benchmark, which is somewhat subjective and therefore difficult to do.
Since we are mainly interested in backtesting and have the benefit of 20/20 hindsight; it's very easy to select the best assets in the rear view mirror. The choice of assets is therefore not as valuable as a backtest screen, and probably doesn't follow the intent of the volume factor anyways. For now, let's put this one aside and focus on of the trading pattern.
Trading Pattern
A simple approach to look at the value of trading is to compare desired metrics between an active trading strategy and a buy-and-hold strategy with the same assets held at the average allocation from the trading model.
A simple example: we trade SPY using the traditional 200 day moving average, if SPY is above its 200MA, we buy; if it's below; we sell and buy IEF (10 year treasuries). The following backtest shows results from 1/1/2009 to 12/31/2018. From the overview, and with rf = 0; the RoMaD = 10.757%/17.3% = 62%. From the results report, the average allocation is 79% SPY and 21% IEF. I can't find the Sortino anywhere on the output, but let's assume the negative deviation is 9%; and the Sortino ratio is therefore 10.757/9 = 1.2.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Tristan F
Now we take the average allocation from the active trading strategy (79% SPY and 21% IEF) and see how things look with a buy-and-hold strategy. We get a RoMaD of 11.454%/22.1% = 52%; and an estimated Sortino of 11.454/10.5 = 1.09.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Tristan F
We now need to determine the value of active trading. We'll ignore the tax impact of turning over the portfolio since our target customers are hedge funds and not individual investors.
Active Strategy:
- Annual Return: 10.76%
- Negative Std Dev: 9%
- Max Draw Down: 17.3%
Passive Strategy:The idea is to understand what risk-adjusted value is produced by active trading. The Modigliani risk-adjusted performance looks at the expected return of oe investment if it is scaled to the same risk as another investment. For example, if we look at the expected return of the active trading for the same negative standard deviation of the passive approach, we get 10.76% * 10.5% / 9% = 12.5%. In other words, if we took the same level of risk as the passive approach, we would get an expected return of 12.5% on the active approach, or a 1.05% higher return.
This represents the change in the expected return generated by the trading strategy for the same level of risk as the buy-and-hold strategy. We can do the same thing for the RoMaD; the only difference is the risk metric (max DD instead of neg SD). In the example above, we would get an expected return of 10.76% * 22.1%/17.3% = 13.74%, or a 2.29% higher expected return from the active trading strategy for the same maximum draw down.
The fitness score could conceivably be equal to the average of these two changes in expectation, in the example this would be (1.05%+2.29%)/2 = 1.67%. If this is greater than a threshold (1%?) then the algorithm makes the alpha stream, otherwise it is rejected.
Pros:
Measures the risk-adjusted value of the trading activity, and not the underlying assets
Think of this as a “trading alpha” – the return that is generated by trading
The growth of the underlying assets is independent from the trading alpha, so buy AAPL in 1998 would have a trading alpha of 0
Compares risks at the average level (Sortino) and the extreme (RoMaD), like the original metric
Works for longer (say monthly) or shorter term (HFT) trading strategies
May favor more frequent trading because there are more opportunities for the trading alpha to develop if there are more trades
Cons:
Requires post-run calculations (buy-and-hold backtest)
More difficult to test systematically and therefore to know where a good threshold might be
However, since this measures an alpha (annual return), a threshold based on judgment from QC, such as 1%, is justifiable
Doesn't look at absolute returns but rather the additional returns generated by trading
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Douglas Stridsberg
Hey Tristan,
Couple of thoughts regarding your proposed approach:
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Tristan F
Douglas,
Good points. Here are some thoughts:
trading alpha = ra * sp/sa - rp
where:
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Douglas Stridsberg
Hi Tristan - I see now that my approximation to ratios of Sortinos isn't quite what you meant, my apologies. I can't quite see a case where the measure would get wildly inflated.
Again, however, I get the impression you're thinking in terms of equities, where indeed a long/short portfolio would typically long some companies and short others, thus ending up with a meaningful measure.
Let's say instead I'm trading a basket of G10 currencies and my active strategy is such that the average weights over the period is 0% across all currencies. rp = 0, sp = 0 - giving a measure of 0 even though the strategy might be profitable.
My point is that the concept of a benchmark is complicated when you start to look outside of equities, where assets often don't have a positive drift and instead act in a more mean-reverting fashion. That's before we even start to talk about options, which brings a wholly different set of problems.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Just throwing this into the discussion: our aim is to remove our human bias so ideally, the fitness function should be automated. If we need to manually construct representative passive portfolios it will induce delays and costs in the evaluation process. If "the community" on a whole has factors they believe should be considered please also sanity check it can be automated!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Tristan F
Jared,
The intent was to automate the benchmark, which could've been done relatively easily, so that an alpha that measures the value of the trading activity could be determined. However, as Douglas points out (thanks Douglas), this method is not universally applicable, so it's value may be limited.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Tristan F
Jared,
Would you consider lowering the threshold on the trade factor like you mentioned before? Maybe requiring full portfolio turnover every month to get a score of 100%? That would mean average turnover of 10% per trading day under the current metric.
Thanks.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Li li
I have a suggestion. For leading us to improve Alpha Ranking, the system should present base performance correlating with Alpha Ranking in Overview page in backtest. Such as backtest year, daily portfolio turnover, annual return, max drawdown, annual std of negative return. Now backtest year, daily portfolio turnover, annual std of negative return are absent.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Link Liang
Hi Li,
Thanks for your suggestion, we will take it into consideration.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Aliou Ba
Thank you
I have concerns with Sortino and RoMaD. My model does not do HFT but it takes on average 282 positions per day on the mini ES contract against an average of 19 in live trading. On a 5 year backtest with a resolution. .Tick, the results are abnormally high but the backtest fails because of the valuation mode of the Sortino and RoMaD due to the high volume. I found that limit orders are instantly executed if the price is hit. I also noted orders executed at prices that the ES has never reached since its creation. For those who do HFT, is a 3 month backtest enough to submit an HFT model in the Alpha stream market?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jack Simonson
Hi Aliou,
Although we no longer use the Fitness Function proposed in the research notebook at the beginning of this thread, 3-months is, unfortunately, not a long enough backtest to submit. Alphas need to demonstrate consistent performance over a long period of time. Funds want to be able to see its performance in a variety of market regimes and a 3-month backtest isn't sufficient for this. You can find details about the Alpha submission criteria here.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!