Short Availability
Key Concepts
Introduction
To short a security, you need to borrow shares from another investor or organization that owns the shares. A shortable provider is a model that tracks the number of shares that are available for you to borrow. A shortable provider can make your backtest results more realistic because it doesn't let you place a short trade if there are no shares available for you to borrow.
Set Providers
The brokerage model of your algorithm automatically sets the settlement model for each security, but you can override it. To manually set the shortable provider of a security, call the set_shortable_provider
method on the Security
object.
def initialize(self) -> None: security = self.add_equity("SPY") # Get shortable stocks and quantity from local disk security.set_shortable_provider(LocalDiskShortableProvider("axos"))
You can also set the shortable provider in a security initializer. If your algorithm has a universe, use the security initializer technique. In order to initialize single security subscriptions with the security initializer, call set_security_initializer
before you create the subscriptions.
def initialize(self) -> None: # Set the security initializer before requesting data to apply to all requested securities afterwards self.set_security_initializer(self.custom_security_initializer) self.add_equity("SPY") def custom_security_initializer(self, security: Security) -> None: security.set_shortable_provider(LocalDiskShortableProvider("axos"))
If you call the set_security_initializer
method, it overwrites the default security initializer. The default security initializer uses the security-level reality models of the brokerage model to set the following reality models of each security:
To extend upon the default security initializer instead of overwriting it, create a custom BrokerageModelSecurityInitializer
.
class BrokerageModelExampleAlgorithm(QCAlgorithm): def initialize(self) -> None: # In the Initialize method, set the security initializer to seed initial the prices and models of assets. self.set_security_initializer(MySecurityInitializer(self.brokerage_model, FuncSecuritySeeder(self.get_last_known_prices))) # Outside of the algorithm class class MySecurityInitializer(BrokerageModelSecurityInitializer): def __init__(self, brokerage_model: IBrokerageModel, security_seeder: ISecuritySeeder) -> None: super().__init__(brokerage_model, security_seeder) def initialize(self, security: Security) -> None: # First, call the superclass definition. # This method sets the reality models of each security using the default reality models of the brokerage model. super().initialize(security) # Next, overwrite some of the reality models security.set_shortable_provider(LocalDiskShortableProvider("axos"))
To view all the pre-built shortable providers, see Supported Providers.
Default Behavior
The brokerage model of your algorithm automatically sets the shortable provider for each security. The default brokerage model is the DefaultBrokerageModel
, which uses the NullShortableProvider.
Provider Structure
Shortable providers must implement the IShortableProvider
interface.
Shortable providers that implement the IShortableProvider
interface must implement the fee_rate
, rebate_rate
, and shortable_quantity
methods.
These methods receives a Symbol and the local time of the algorithm.
The fee_rate
method returns the borrow fee rate.
The rebate_rate
method returns the borrow rebate rate.
The shortable_quantity
returns the shortable quantity.
class CustomShortableProviderExampleAlgorithm(QCAlgorithm): def initialize(self) -> None: security = self.add_equity("SPY") # Provide the selected security with your broker's shortable information security.set_shortable_provider(MyShortableProvider()) # Define the custom shortable provider class MyShortableProvider(NullShortableProvider): def fee_rate(self, symbol: Symbol, local_time: datetime) -> float: return 0.0025 def rebate_rate(self, symbol: Symbol, local_time: datetime) -> float: return 0.0507 def shortable_quantity(self, symbol: Symbol, local_time: datetime) -> float: return 10000
For a full example algorithm, see this backtest .
Borrowing Costs
In live trading, your brokerage charges an interest fee when you borrow shares to short a security. In backtesting, we don't currently simulate short interest fees. You can evaluate the borrowing cost with the fee_rate
and rebate_rate
methods.
To get valid borrowing rates, use the InteractiveBrokersShortableProvider.
Examples
The following examples demonstrate common practices for using a shortable provider.
Example 1: Sizing By Shortable Quantity
The following algorithm trades arbitrage of class A and class C price divergence of Google stocks using cointegration analysis. However, the position sizing may be limited by the available quantity to be shorted by the brokerage, breaking the paired trade. Thus, we need to make use of the
shortable_quantity
method of the shortable provider to calculate the quantity.
from sklearn.linear_model import LinearRegression from statsmodels.tsa.stattools import adfuller class ShortableProviderAlgorithm(QCAlgorithm): # The threshold that the spread/residual of the cointegrated series triggers a trade. threshold = 0 # Store the coefficient and intercept of the cointegrated series for calculating the spread of a new data point. coefficients = [0, 0] def initialize(self) -> None: self.set_start_date(2019, 1, 1) self.set_end_date(2023, 1, 1) # Subscribe to 2 classes of Google stocks to trade their price divergence. goog1 = self.add_equity("GOOGL", Resolution.DAILY) # Class A goog2 = self.add_equity("GOOG", Resolution.DAILY) # Class C self.goog1 = goog1.symbol self.goog2 = goog2.symbol # Set the shortable providers to simulate the available shares to trade realistically. goog1.set_shortable_provider(InteractiveBrokersShortableProvider()) goog2.set_shortable_provider(InteractiveBrokersShortableProvider()) # Use rolling windows to save the price data for cointegration analysis. goog1.window = RollingWindow[float](252) goog2.window = RollingWindow[float](252) # Warm up the rolling windows. history = self.history[TradeBar]([self.goog1, self.goog2], 252, Resolution.DAILY) for bar in history: goog1.window.add(bar[self.goog1].close) goog2.window.add(bar[self.goog2].close) # Adjust the cointegration factor between the 2 classes' monthly price series. self.schedule.on( self.date_rules.month_start(), self.time_rules.at(0, 1), self.calculate_cointegration ) self.calculate_cointegration() def on_data(self, slice: Slice) -> None: bar1 = slice.bars.get(self.goog1) bar2 = slice.bars.get(self.goog2) if bar1 and bar2: # Update the rolling windows for cointegration analysis. self.securities[self.goog1].window.add(bar1.close) self.securities[self.goog2].window.add(bar2.close) # Calculate the current cointegrated series spread. residual = self.coefficients[0] * bar2.close + self.coefficients[1] - bar1.close quantity = self.calculate_order_quantity(self.goog1, 0.2) # If the residual is lower than the negative threshold, it means class A's price is much higher than it should be compared to class C. # We sell class A and buy class C to bet on their price convergence. if residual < -self.threshold and not self.portfolio[self.goog1].is_short: # Since it will be limited by shortable quantity, we max{shortable, 20% of portfolio value}. shortable_quantity = self.securities[self.goog1].shortable_provider.shortable_quantity(self.goog1, bar1.end_time) quantity = min(quantity, shortable_quantity) self.market_order(self.goog1, -quantity) self.market_order(self.goog2, quantity * self.coefficients[0]) # If the residual is higher than the threshold, it means class A price is much lower than what it should be compared to class C. # We buy class A and sell class C to bet on their price convergence. elif residual > self.threshold and not self.portfolio[self.goog1].is_long: # Since it will be limited by shortable quantity, we max{shortable, 20% of portfolio value}. shortable_quantity = self.securities[self.goog2].shortable_provider.shortable_quantity(self.goog2, bar2.end_time) quantity = min(quantity, shortable_quantity) self.market_order(self.goog1, quantity) self.market_order(self.goog2, -quantity * self.coefficients[0]) # Close positions of the price are converged. elif (self.portfolio[self.goog1].is_short and residual > 0) or (self.portfolio[self.goog1].is_long and residual < 0): self.liquidate() def calculate_cointegration(self) -> None: # Lag direction is unimportant; it is just a sign flip in the linear regression, so we don't need to flip the window order. y = np.array(list(self.securities[self.goog1].window)).reshape(-1, 1) x = np.array(list(self.securities[self.goog2].window)).reshape(-1, 1) # Perform Linear Regression on both price series to investigate their relationship. lr = LinearRegression().fit(x, y) slope = lr.coef_[0] intercept = lr.intercept_ # Calculate the residuals series to check if it is stationary, meaning if the 2 price series move together. residuals = y - (intercept + slope * x) # Check if the residuals are stationary using the augmented Dickey-Fuller test. # Reject the null hypothesis of a unit root is present if test statistic <= -3.45 (approximate α=0.05 for n=250) # This means no unit root exists for the difference series, and the residuals are stationary. adf_reject = adfuller(residuals)[0] <= -3.45 if adf_reject: # If cointegrated, update the positional sizing ratio and the spread threshold of the trade trigger. self.coefficients = [slope, intercept] self.threshold = 2 * np.std(residuals) else: # If not cointegrated, liquidate and set the size to zeros for no positions. self.liquidate() self.coefficients = [0, 0] self.threshold = 100000000 # An arbitrarily large number that the class A price will never reach.
Other Examples
For more examples, see the following algorithms: