Research Environment
Universes
Introduction
Universe selection is the process of selecting a basket of assets to research. Dynamic universe selection increases diversification and decreases selection bias in your analysis.
Get Universe Data
Universes are data types. To get historical data for a universe, pass the universe data type to the UniverseHistory
method. The object that returns contains a universe data collection for each day. With this object, you can iterate through each day and then iterate through the universe data objects of each day to analyze the universe constituents.
For example, follow these steps to get the US Equity Fundamental data for a specific universe:
- Load the assembly files and data types in their own cell.
- Load the necessary assembly files.
- Import the
QuantConnect
package and the universe data type. - Create a
QuantBook
. - Define a universe.
- Call the
UniverseHistory
universe_history
method with the universe, a start date, and an end date. - Iterate through the Series to access the universe data.
- Iterate through the result to access the universe data.
#load "../Initialize.csx"
#load "../QuantConnect.csx"
using QuantConnect; using QuantConnect.Research; using QuantConnect.Data.Fundamental;
var qb = new QuantBook();
qb = QuantBook()
The following example defines a dynamic universe that contains the 10 Equities with the lowest PE ratios in the market. To see all the Fundamental
attributes you can use to define a filter function for a Fundamental universe, see Data Point Attributes. To create the universe, call the AddUniverse
add_universe
method with the filter function.
var universe = qb.AddUniverse(fundamentals => { return fundamentals .Where(f => !double.IsNaN(f.ValuationRatios.PERatio)) .OrderBy(f => f.ValuationRatios.PERatio) .Take(10) .Select(f => f.Symbol); });
def filter_function(fundamentals): sorted_by_pe_ratio = sorted( [f for f in fundamentals if not np.isnan(f.valuation_ratios.pe_ratio)], key=lambda fundamental: fundamental.valuation_ratios.pe_ratio ) return [fundamental.symbol for fundamental in sorted_by_pe_ratio[:10]] universe = qb.add_universe(filter_function)
var universeHistory = qb.UniverseHistory(universe, new DateTime(2023, 11, 6), new DateTime(2023, 11, 13));
universe_history = qb.universe_history(universe, datetime(2023, 11, 6), datetime(2023, 11, 13))
The end date arguments is optional. If you omit it, the method returns Fundamental
data between the start date and the current day.
The universe_history
method returns a Series where the multi-index is the universe Symbol
and the time when universe selection would occur in a backtest. Each row in the data column contains a list of Fundamental
objects. The following image shows the first 5 rows of an example Series:

To get a flat DataFrame instead of a Series, set the flatten
argument to True
.
The UniverseHistory
method returns an IEnumerable<IEnumerable<Fundamental>>
object.
universe_history = universe_history.droplevel('symbol', axis=0) for date, fundamentals in universe_history.items(): for fundamental in fundamentals: symbol = fundamental.symbol price = fundamental.price if fundamental.has_fundamental_data: pe_ratio = fundamental.valuation_ratios.pe_ratio
foreach (var fundamentals in universeHistory) { foreach (Fundamental fundamental in fundamentals) { var date = fundamental.Time; var symbol = fundamental.Symbol; var price = fundamental.Price; if (fundamental.HasFundamentalData) { var peRatio = fundamental.ValuationRatios.PERatio; } } }
Available Universes
To get universe data for other types of universes, you usually just need to replace Fundamental
in the preceding code snippets with the universe data type.
The following table shows the datasets that support universe selection and their respective data type.
For more information, about universe selection with these datasets and the data points you can use in the filter function, see the dataset's documentation.
Dataset Name | Universe Type(s) | Documentation |
---|---|---|
US ETF Constituents | ETFConstituentUniverse | Learn more |
US Equity Option Universe | OptionUniverse | Learn more |
US Index Option Universe | OptionUniverse | Learn more |
Binance Crypto Price Data | CryptoUniverse | Learn more |
Binance US Crypto Price Data | CryptoUniverse | Learn more |
Bitfinex Crypto Price Data | CryptoUniverse | Learn more |
Bybit Crypto Price Data | CryptoUniverse | Learn more |
Coinbase Crypto Price Data | CryptoUniverse | Learn more |
Kraken Crypto Price Data | CryptoUniverse | Learn more |
Brain Language Metrics on Company Filings | BrainCompanyFilingLanguageMetricsUniverse | ">Learn more |
Brain ML Stock Ranking | BrainStockRankingUniverse | ">Learn more |
Brain Sentiment Indicator | BrainSentimentIndicatorUniverse | ">Learn more |
Crypto Market Cap | CoinGeckoUniverse | ">Learn more |
CNBC Trading | QuiverCNBCsUniverse | ">Learn more |
Corporate Lobbying | QuiverLobbyingUniverse | ">Learn more |
Insider Trading | QuiverInsiderTradingUniverse | ">Learn more |
US Congress Trading | QuiverQuantCongressUniverse | ">Learn more |
US Government Contracts | QuiverGovernmentContractUniverse | ">Learn more |
WallStreetBets | QuiverWallStreetBetsUniverse | ">Learn more |
Corporate Buybacks |
| ">Learn more |
To get universe data for Futures and Options, use the FutureHistory
future_history
and OptionHistory
option_history
methods, respectively.
Examples
The following examples demonstrate some common practices for universe research.
Example 1: Top-Minus-Bottom PE Ratio
The below example studies the top-minus-bottom PE Ratio universe, in which the top 10 PE Ratio stocks are brought, and the bottom 10 are sold in equal weighting daily. We carry out a mini-backtest to analyze its performance.
// Load the required assembly files and data types. #load "../Initialize.csx" #load "../QuantConnect.csx" using QuantConnect; using QuantConnect.Data; using QuantConnect.Data.Market; using QuantConnect.Algorithm; using QuantConnect.Research; using System; using MathNet.Numerics.Distributions; // Instantiate the QuantBook instance for researching. var qb = new QuantBook(); // Set start and end dates of the research to avoid look-ahead bias. var start = new DateTime(2021, 1, 1); var end = new DateTime(2021, 4, 1); // Request data for research purposes. // We are interested in the most liquid primary stocks. var universe = qb.AddUniverse( (datum) => datum.OrderByDescending(x => x.DollarVolume) .Take(100) .Select(x => x.Symbol) ); // Historical data call for the data to be compared and tested. var universeHistory = qb.UniverseHistory(universe, start, end); // Process the historical data to generate a signal and return it for research. var bottomPeRatioDict = new Dictionary<DateTime, List<Symbol>>(); var topPeRatioDict = new Dictionary<DateTime, List<Symbol>>(); foreach (FundamentalUniverse fundamentals in universeHistory) { var sortedByPeRatio = fundamentals.OrderBy(x => (x as Fundamental).ValuationRatios.PERatio).ToList(); var bottomPeRatio = sortedByPeRatio.Take(10) .Select(x => x.Symbol) .ToList(); var topPeRatio = sortedByPeRatio.TakeLast(10) .Select(x => x.Symbol) .ToList(); // Study 10 stocks with the top and bottom PE Ratios. var _time = fundamentals.First().Time; bottomPeRatioDict[_time.Date] = bottomPeRatio; topPeRatioDict[_time.Date] = topPeRatio; } // Extract symbols from both dictionaries and remove duplicates var allSymbols = bottomPeRatioDict.Values .SelectMany(symbols => symbols) .Concat(topPeRatioDict.Values.SelectMany(symbols => symbols)) .Distinct() .ToList(); // All symbols' daily prices are for return comparison. var history = qb.History<TradeBar>(allSymbols, start, end, Resolution.Daily).ToList(); // Iterate the history to backtest the top minus bottom performance. var time = new List<DateTime>() { start }; var equity = new List<decimal>() { 1m }; for (int i = 0; i < history.Count - 2; i++) { var bar = history[i]; var nextBar = history[i+1]; var timeStamp = bar.Values.First().EndTime; var bottomReturn = bottomPeRatioDict[timeStamp.Date].Sum(x => (nextBar[x].Close - bar[x].Close) / bar[x].Close * -0.1m); var topReturn = topPeRatioDict[timeStamp.Date].Sum(x => (nextBar[x].Close - bar[x].Close) / bar[x].Close * 0.1m); // Calculate the cumulative return. time.Add(nextBar.Values.First().EndTime); equity.Add((bottomReturn + topReturn + 1m) * equity[^1]); } // Create line chart of the equity curve. var chart = Chart2D.Chart.Line<DateTime, decimal, string>( time, equity ); // Create a Layout as the plot settings. LinearAxis xAxis = new LinearAxis(); xAxis.SetValue("title", "Time"); LinearAxis yAxis = new LinearAxis(); yAxis.SetValue("title", "Equity"); Title title = Title.init($"Equity by Time of Top-Minus-Bottom Universe"); Layout layout = new Layout(); layout.SetValue("xaxis", xAxis); layout.SetValue("yaxis", yAxis); layout.SetValue("title", title); // Assign the Layout to the chart. chart.WithLayout(layout); // Display the plot. HTML(GenericChart.toChartHTML(chart))
# Instantiate the QuantBook instance for researching. qb = QuantBook() # Set start and end dates of the research to avoid look-ahead bias. start = datetime(2021, 1, 1) end = datetime(2021, 4, 1) qb.set_start_date(end) # Request data for research purposes. # We are interested in the most liquid primary stocks. universe = qb.add_universe( lambda datum: [x.symbol for x in sorted( [f for f in datum], key=lambda f: f.dollar_volume )[-100:]] ) # Historical data call for the data to be compared and tested. universe_history = qb.universe_history(universe, start, end).droplevel([0]) # All symbols' daily prices are for return comparison. history = qb.history( [x.symbol for x in set( e for datum in universe_history.values.flatten() for e in datum )], start, end, Resolution.DAILY ).close.unstack(0) # Change the index format for matching. universe_history.index = universe_history.index.date history.index = history.index.date # Process the historical data to generate a signal and return it for research. bottom_liquid_universe_history = universe_history.apply( lambda datum: [x.symbol for x in sorted( datum, key=lambda d: d.dollar_volume )[:10]] ) top_liquid_universe_history = universe_history.apply( lambda datum: [x.symbol for x in sorted( datum, key=lambda d: d.dollar_volume, reverse=True )[:10]] ) ret = history.pct_change().dropna() # Keep the entries that have common indices for a fair comparison. common_index = list(set(ret.index).intersection(universe_history.index)) ret = ret.loc[common_index].apply( lambda x: x[top_liquid_universe_history.loc[x.name]].sum()*0.1\ - x[bottom_liquid_universe_history.loc[x.name]].sum()*0.1, axis=1 ) # Plot the data for visualization. It is easier to study the pattern. fig = plt.figure(figsize=(8, 8)) ax = fig.add_subplot() # Use line plot to present time series. ax.plot(ret.sort_index()) # Set axis and titles to explain the plot. ax.set_xlabel("Time") ax.set_ylabel("Return") ax.set_title("Top Minus Bottom Liquidity Return") plt.show()