Hi QC Community!
We're thrilled to announce the new Crypto Coarse Fundamental Universe feature. This is the Crypto version of Coarse Universe Selection you may have been using for Equities. This new feature allows us to select a basket of Crypto pairs by its volume or price data, which is helpful to scale our universe to liquid tokens. With further engineering, we can use the filter to select Crypto assets based on the values of technical indicators. For an example of using technical indicators in a universe selection function, see the EmaCrossUniverseSelectionAlgorithm.
Similar to Coarse Universe Selection for Equities, when we add a CryptoCoarseFunamentalUniverse to an algorithm, we provide a filter function. For example:
self.AddUniverse(CryptoCoarseFundamentalUniverse(Market.Bitfinex, self.UniverseSettings, self.UniverseSelectionFilter))
The filter function receives CryptoCoarseFundamental objects, which have the following properties:
- Open: Yesterday's open price
- High: Yesterday's high price
- Low: Yesterday's low price
- Close: Yesterday's close price
- Volume: Yesterday's trading volume on the selected Market, denominated in the base currency.
- VolumeInQuoteCurrency: Yesterday's trading volume on the selected Market, denominated in the quote currency.
- VolumeInUsd: Yesterday's trading volume on the selected Market, denominated in USD. We convert the volume into USD volume to allow fair comparison between Crypto pairs with different base currencies. In most cases, this data is available, however, it is not guaranteed since the conversion requires multiple inter-pair exchanges.
The universe consists of assets from a single exchange because it makes it easier for live trading and we can't get accurate transaction records from decentralized transactions. To analyze Crypto assets from multiple markets, add multiple universes, each with a different Market argument.
In live mode, the pipeline has a 16-hour delay, so the CryptoCoarseFundamental objects are injected into algorithms at around 16:00-16:30 UTC each day, depending on the data processing task. This is not a concern for the classic Equity coarse universe, but since the Crypto markets operate 24/7, some members may notice this delay.
To help everyone get started, we updated the documentation on dataset pages for each Crypto brokerage. To view the one-liner that adds universe selection for each Crypto brokerage, open the Dataset Market, click one of the Crypto Price datasets, and then scroll down to the Getting Started section. To see full C# and Python example algorithms that use CryptoCoarseFundamentalUniverse, on the Crypto dataset listing, click the Examples tab, or find out the C# and Python demonstration algorithms in the GH repo.
As always, let us know if you have any issues or concerns. We will look into them ASAP. Enjoy! 🚀
Best
Louis
OvidiuS
Looking forward to trying out the new feature. Keep up the great work!
Dave Mueller
Hey Louis, this is awesome, thanks! However, when I tried to clone the algorithm, I receive an error that 'CryptoCoarseFundamentalUniverse' is not defined. Any idea why this is?
Louis Szeto
Hi Dave
This function is only available in IDE 3.0, since the latest LEAN version available for IDE 2.0 doesn't include it. Please upgrade to that by navigating to Your username at top right-hand corner -> My Account -> check "QuantConnect Algorithm Environment 3.0 Beta".
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Santa24
Louis Szeto
I tried it for Market.Binance. It says that on 2021-01-01 the top5 symbols by dollar volume over 10million usd are the following. Is this plausible?
Tony GU
Hi Louis,
I got same issue as mentioned by Santa24.
I tried your demo code using Bitfinex and Binance, with recent data like SetStartDate (2022, 5, 9), but found the Symbol list generated by UniverseSelectionFilter, sorted by VolumeInUsd, is apparently different from the list I got from Bitfinex or Binance APP.
Louis Szeto
Hi Santa24 and Tony
Unlike statistics provided on the brokers’ websites, our data processing mechanism is based on all available trading pairs in the exchange, not limited to stablecoin pairs only. Also, our data is processed based on the last 24 hours from 00:00 UTC, but not the last 24 hours in real-time. Thus, the data will have a discrepancy.
As per the “weird USDT pairs” query, please be noted that USDT pairs could also be in form of “USDTxxx”, so you may want to call:
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
.ekz.
Yeah, something does feel odd here.
The list that Santa24 is seeing seems wrong. For example, I'd expect to see BTC/USDT high in this list.
For reference, here's a list of currency markets (pairs) ranked by trading volume:
https://coinranking.com/markets
Louis Szeto
Hi .ekz, Santas24 and Tony
Thank you for reporting the issue. This is due to a bug in the data processor that mistakenly took the base currency as the quote currency of the pair. It is now resolved. We're sorry for the inconvenience caused.
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jack Pizza
throws run time error about fundamental data
Tony GU
Hi Louis,
Thank you for your effort in addressing these issues. I tried again and found the issue seems still there - top USD volume crypto pair list generated by Coarse is totally different from what I got from Binance APP for the same period.
In addition, I also found:
Runtime Error: '>' not supported between instances of 'NoneType' and 'int' at filtered = [datum for datum at data if datum.VolumeInUsd > 1000000]
Tony
Santa24
Louis Szeto
I am now receiving these coins as the top15 volume. This seems highly implausible
2021-01-03 00:00:00 :2021-01-03 00:00:00 UNIVERSE=['TUSDUSDT', 'CHZUSDT', 'SOLUSDT', 'NEOUSDT', 'RENUSDT', 'ETCUSDT', 'ONTUSDT', 'GBPUSDT', 'OCEANUSDT', 'BANDUSDT', 'STXUSDT', 'ALPHAUSDT', 'SRMUSDT', 'ARPAUSDT', 'QTUMUSDT']2021-01-10 00:00:00 :2021-01-10 00:00:00 UNIVERSE=['LUNAUSDT', 'OCEANUSDT', 'RUNEUSDT', 'KSMUSDT', 'RLCUSDT', 'ROSEUSDT', 'ZRXUSDT', 'CHZUSDT', 'UNFIUSDT', 'STORJUSDT', 'TRBUSDT', 'BALUSDT', 'RIFUSDT', 'COTIUSDT', 'JSTUSDT']2021-01-17 00:00:00 :2021-01-17 00:00:00 UNIVERSE=['PAXUSDT', 'CVCUSDT', 'BATUSDT', 'INJUSDT', 'FETUSDT', 'HBARUSDT', 'ENJUSDT', 'DIAUSDT', 'TOMOUSDT', 'SKLUSDT', 'HARDUSDT', 'STXUSDT', 'LUNAUSDT', 'BZRXUSDT', 'FTMUSDT'].ekz.
Santa24 : I think that is to be expected because, on line #7, You're *filtering out* symbols that have VolumeInUsd over 10000000.
Removing that line will give you the usual high volume symbols (BTC, ETH, etc).
Santa24
.ekz. oh yes you are right - sorry about that, my mistake, So for binance this ended up working for weekly universe selection top 10 by volume:
Louis Szeto
Hi Tony Gu
As we mentioned, although most situations will have this attribute available, some pairs in some unique days will not. It is recommended to check with:
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Tony GU
Hi Louis,
Thank you for following up on this question. It works.
Tony
Tom M
This is great, thanks!
I am having trouble using a margin account with this framework. If I change the brokerage model AccountType from Cash to Margin, it still fails to execute my shorts with the message:
However, I find that if I comment out the below, short orders are accepted (but I don't want to do this, as it results in other problems with the script).
Any suggestions appreciated. Thanks!
Louis Szeto
Hi Tom
We cannot reproduce the problem, do you mind attaching some code snippets or a backtest to let us understand?
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Napoleon
Hi Louis,
I've tried to use CryptoCoarseFundamentalUniverse in backtest. However, there is error message in CloudTerminal and the message is as followed.
Backtest Handled Error: We could not fetch the requested data. This may not be valid data, or a failed download of custom data. Skipping source (/Data/crypto/binance/fundamental/coarse/20220221.csv).
May I know if this is normal? Thanks in advance.
Alexandre Catarino
Hi Napoleon,
We are aware of the issue and have contacted the data vendor to patch the missing dates of Binance data.
Mitch Christow
Hi Louis.
I was hoping you could help me out. I cloned your sample code, and then tried to refactor the universe selection into its own class. However I am running into different errors depending on how I attempt to refactor the code.. I can't attach a backtest since they are failing to compile. So I am inserting the code here.
Here is the main python class
Here is the CryptoUniverseSelectionModel class
The project does not compile since the method signature does not match and it throws the following error:
If I try it by passing the parameters in so I can match the method signature
Then the project fails to compile with the following error
Would you by any chance be able to show me how to properly refactor the code so that the Universe selection lives in its own class? Thanks for your help.
Cheers,
Mitch
Louis Szeto
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!