Indicators

You need to subscribe to some market data in order to calculate indicator values.

var qb = new QuantBook();
var symbol = qb.AddEquity("SPY").Symbol;

qb = QuantBook()
symbol = qb.add_equity("SPY").symbol

You need to subscribe to some market data in order to calculate a timeseries of indicator values.

Follow these steps to create an indicator timeseries:

1. Get some historical data.

// Request historical trading data with the daily resolution.
var history = qb.History(symbol, 70, Resolution.Daily);

# Request historical trading data with the daily resolution.
history = qb.history[TradeBar](symbol, 70, Resolution.DAILY)

2. Define a custom indicator class that inherits from the Indicator superclass.
3. Define a custom indicator class. Note the PythonIndicator superclass inheritance, Value attribute, and Updateupdate method are mandatory.

In this tutorial, create an ExpectedShortfallPercent indicator that uses Monte Carlo to calculate the expected shortfall of returns. Use the WindowIndicator superclass instead of Indicator for using a period of historical data stored in a RollingWindow.

In this tutorial, create an ExpectedShortfallPercent indicator that uses Monte Carlo to calculate the expected shortfall of returns.

public class ExpectedShortfallPercent : WindowIndicator<IndicatorDataPoint>, IIndicatorWarmUpPeriodProvider
{
    private decimal _alpha;
    
    // Set up IndicatorDataPoint attributes for the indicator.
    public IndicatorBase<IndicatorDataPoint> ValueAtRisk { get; }

    // Set up the WarmUpPeriod attribute to provide implementation of the IIndicatorWarmUpPeriodProvider interface.
    public override int WarmUpPeriod => Period;

    // Set up the constructor.
    // period: The lookback period for return distribution.
    // alpha: Alpha level of VaR cutoff.
    public ExpectedShortfallPercent(int period, decimal alpha)
        : base("ES", period)
    {
        _alpha = alpha;
        ValueAtRisk = new Identity("ES_VaR");
    }

    // Override the IsReady method to set up the flag of the Indicator and its IndicatorDataPoint attributes are ready.
    public override bool IsReady => ValueAtRisk.IsReady;

    // Mandatory: Override the ComputeNextValue method to calculate the indictor value.
    protected override decimal ComputeNextValue(IReadOnlyWindow<IndicatorDataPoint> window, IndicatorDataPoint input)
    {
        if (Samples < 2)
            return 0m;
                
        var n = Math.Min(Period, Samples);
        var cutoff = (int) Math.Ceiling(n * _alpha);
        
        var samples = new List<decimal>();
        for (int i = 0; i < window.Count - 1; i++)
        {
            samples.Add( (window[i] - window[i+1]) / window[i+1] );
        }
        var lowest = samples.OrderBy(x => x).Take(cutoff);
        
        ValueAtRisk.Update(input.Time, lowest.Last());
        return lowest.Average();
    }
}

class ExpectedShortfallPercent(PythonIndicator):
    import math, numpy as np
    
    def __init__(self, period, alpha):
        self.value = None   # Attribute represents the indicator value
        self.value_at_risk = None
        
        self.alpha = alpha
        
        self._window = RollingWindow[float](period)
    
    # Override the IsReady attribute to flag all attributes values are ready.
    @property
    def is_ready(self) -> bool:
        return self.value and self.value_at_risk
    
    # Method to update the indicator values. Note that it only receives 1 IBaseData object (Tick, TradeBar, QuoteBar) argument.
    def update(self, input: BaseData) -> bool:
        count = self._window.Count
        
        self._window.Add(input.Close)
        
        # Update the Value and other attributes as the indicator current value.
        if count >= 2:
            cutoff = math.ceil(self.alpha * count)
            
            ret = [ (self._window[i] - self._window[i+1]) / self._window[i+1] for i in range(count-1) ]
            lowest = sorted(ret)[:cutoff]
            
            self.value = np.mean(lowest)
            self.value_at_risk = lowest[-1]
            
        # return a boolean to indicate IsReady.
        return count >= 2

4. Initialize a new instance of the custom indicator.

var es = new ExpectedShortfallPercent(50, 0.05m);

custom = ExpectedShortfallPercent(50, 0.05)

5. Create a RollingWindow for each attribute of the indicator to hold their values.

In this example, save 20 data points.

// Create a window dictionary to store RollingWindow objects.
var window = new Dictionary<string, RollingWindow<decimal>>();
// Store the RollingWindow objects, index by key is the property of the indicator.
var time = new RollingWindow<DateTime>(20);
window["expectedshortfall"] = new RollingWindow<decimal>(20);
window["valueatrisk"] = new RollingWindow<decimal>(20);

# Create a window dictionary to store RollingWindow objects.
window = {}
# Store the RollingWindow objects, index by key is the property of the indicator.
window['time'] = RollingWindow[DateTime](20)
window['expectedshortfall'] = RollingWindow[float](20)
window['valueatrisk'] = RollingWindow[float](20)

6. Attach a handler method to the indicator that updates the RollingWindow objects.

// Define an update function to add the indicator values to the RollingWindow object.
es.Updated += (sender, updated) =>
{
    var indicator = (ExpectedShortfallPercent) sender;
    time.Add(updated.EndTime);
    window["expectedshortfall"].Add(updated);
    window["valueatrisk"].Add(indicator.ValueAtRisk.Current);
};

When the indicator receives new data, the preceding handler method adds the new IndicatorDataPoint values into the respective RollingWindow.

7. Iterate through the historical market data and update the indicator.

foreach(var bar in history){
    es.Update(bar.EndTime, bar.Close);
}

for bar in history:
    custom.update(bar)
    
    # The Updated event handler is not available for custom indicator in Python, RollingWindows are needed to be updated in here.
    if custom.is_ready:
        window['time'].add(bar.end_time)
        window['expectedshortfall'].add(custom.value)
        window['valueatrisk'].add(custom.value_at_risk)

8. Display the data.

Console.WriteLine($"time,{string.Join(',', window.Select(kvp => kvp.Key))}");
foreach (var i in Enumerable.Range(0, 5).Reverse())
{
    var data = string.Join(", ", window.Select(kvp => Math.Round(kvp.Value[i],6)));
    Console.WriteLine($"{time[i]:yyyyMMdd}, {data}");
}


9. Populate a DataFrame with the data in the RollingWindow objects.

custom_dataframe = pd.DataFrame(window).set_index('time')

Jupyter Notebooks don't currently support libraries to plot historical data, but we are working on adding the functionality. Until the functionality is added, use Python to plot custom indicators.

Follow these steps to plot the indicator values:

1. Call the plot method.

custom_dataframe.plot()

2. Show the plot.

plt.show()

The following examples demonstrate some common practices for researching with combining indicators.

Example 1: Expected Shortfall

The following example demonstrates researching using a custom-constructed Expected Shortfall indicator. Expected Shortfall refers to the average of the N% worst-case return.

// Load the assembly files and data types in their own cell.
#load "../Initialize.csx"

// Load the necessary assembly files.
#load "../QuantConnect.csx"
#r "../Plotly.NET.dll"
#r "../Plotly.NET.Interactive.dll"

// Import the QuantConnect, Plotly.NET, and Accord packages for calculation and plotting.
using QuantConnect;
using QuantConnect.Indicators;
using QuantConnect.Research;
            
using Plotly.NET;
using Plotly.NET.Interactive;
using Plotly.NET.LayoutObjects;

// Instantiate the QuantBook instance for researching.
var qb = new QuantBook();
// Request SPY data to work with the indicator.
var symbol = qb.AddEquity("SPY").Symbol;

// Request historical trading data with the daily resolution.
var history = qb.History(symbol, 70, Resolution.Daily);

public class ExpectedShortfallPercent : WindowIndicator<IndicatorDataPoint>, IIndicatorWarmUpPeriodProvider
{
    private decimal _alpha;
    
    // Set up IndicatorDataPoint attributes for the indicator.
    public IndicatorBase<IndicatorDataPoint> ValueAtRisk { get; }

    // Set up the WarmUpPeriod attribute to provide implementation of the IIndicatorWarmUpPeriodProvider interface.
    public override int WarmUpPeriod => Period;

    // Set up the constructor.
    // period: The lookback period for return distribution.
    // alpha: Alpha level of VaR cutoff.
    public ExpectedShortfallPercent(int period, decimal alpha)
        : base("ES", period)
    {
        _alpha = alpha;
        ValueAtRisk = new Identity("ES_VaR");
    }

    // Override the IsReady method to set up the flag of the Indicator and its IndicatorDataPoint attributes are ready.
    public override bool IsReady => ValueAtRisk.IsReady;

    // Mandatory: Override the ComputeNextValue method to calculate the indictor value.
    protected override decimal ComputeNextValue(IReadOnlyWindow<IndicatorDataPoint> window, IndicatorDataPoint input)
    {
        if (Samples < 2)
            return 0m;
                
        var n = Math.Min(Period, Samples);
        var cutoff = (int) Math.Ceiling(n * _alpha);
        
        var samples = new List<decimal>();
        for (int i = 0; i < window.Count - 1; i++)
        {
            samples.Add( (window[i] - window[i+1]) / window[i+1] );
        }
        var lowest = samples.OrderBy(x => x).Take(cutoff);
        
        ValueAtRisk.Update(input.Time, lowest.Last());
        return lowest.Average();
    }
}

// Create the expected shortfall indicator to study.
var es = new ExpectedShortfallPercent(50, 0.05m);

// Create a window dictionary to store RollingWindow objects.
var window = new Dictionary<string, RollingWindow<decimal>>();
// Store the RollingWindow objects, index by key is the property of the indicator.
var time = new RollingWindow<DateTime>(20);
window["expectedshortfall"] = new RollingWindow<decimal>(20);
window["valueatrisk"] = new RollingWindow<decimal>(20);

// Define an update function to add the indicator values to the RollingWindow object.
es.Updated += (sender, updated) =>
{
    var indicator = (ExpectedShortfallPercent) sender;
    time.Add(updated.EndTime);
    window["expectedshortfall"].Add(updated);
    window["valueatrisk"].Add(indicator.ValueAtRisk.Current);
};

// Iterate through the historical market data and update the indicator.
foreach(var bar in history){
    es.Update(bar.EndTime, bar.Close);
}

// Create line chart of the Expected Shortfall Percent.
var chart = Chart2D.Chart.Line<DateTime, decimal, string>(
    time,
    window["expectedshortfall"]
);

// Create a Layout as the plot settings.
LinearAxis xAxis = new LinearAxis();
xAxis.SetValue("title", "Time");
LinearAxis yAxis = new LinearAxis();
yAxis.SetValue("title", "%");
Title title = Title.init($"Expected Shortfall Percent of {symbol}");

Layout layout = new Layout();
layout.SetValue("xaxis", xAxis);
layout.SetValue("yaxis", yAxis);
layout.SetValue("title", title);
// Assign the Layout to the chart.
chart.WithLayout(layout);

// Display the plot.
HTML(GenericChart.toChartHTML(chart))

# Instantiate the QuantBook instance for researching.
qb = QuantBook()
# Request SPY data to work with the indicator.
symbol = qb.add_equity("SPY").symbol

# Request historical trading data with the daily resolution.
history = qb.history[TradeBar](symbol, 70, Resolution.DAILY)

class ExpectedShortfallPercent(PythonIndicator):
    import math, numpy as np
    
    def __init__(self, period, alpha):
        self.value = None   # Attribute represents the indicator value
        self.value_at_risk = None
        
        self.alpha = alpha
        
        self._window = RollingWindow[float](period)
    
    # Override the IsReady attribute to flag all attributes values are ready.
    @property
    def is_ready(self) -> bool:
        return self.value and self.value_at_risk
    
    # Method to update the indicator values. Note that it only receives 1 IBaseData object (Tick, TradeBar, QuoteBar) argument.
    def update(self, input: BaseData) -> bool:
        count = self._window.Count
        
        self._window.Add(input.Close)
        
        # Update the Value and other attributes as the indicator current value.
        if count >= 2:
            cutoff = math.ceil(self.alpha * count)
            
            ret = [ (self._window[i] - self._window[i+1]) / self._window[i+1] for i in range(count-1) ]
            lowest = sorted(ret)[:cutoff]
            
            self.value = np.mean(lowest)
            self.value_at_risk = lowest[-1]
            
        # return a boolean to indicate IsReady.
        return count >= 2
    
# Initialize a new instance of the custom indicator.
custom = ExpectedShortfallPercent(50, 0.05)

# Create a window dictionary to store RollingWindow objects.
window = {}
# Store the RollingWindow objects, index by key is the property of the indicator.
window['time'] = RollingWindow[DateTime](20)
window['expectedshortfall'] = RollingWindow[float](20)
window['valueatrisk'] = RollingWindow[float](20)

# Iterate through the historical market data and update the indicator.
for bar in history:
    custom.update(bar)
    
    # The Updated event handler is not available for custom indicator in Python, RollingWindows are needed to be updated in here.
    if custom.is_ready:
        window['time'].add(bar.end_time)
        window['expectedshortfall'].add(custom.value)
        window['valueatrisk'].add(custom.value_at_risk)
    
# Populate a DataFrame with the data in the RollingWindow objects.
custom_dataframe = pd.DataFrame(window).set_index('time')

# Create line chart of the Expected Shortfall Percent.
custom_dataframe.plot(title=f"Expected Shortfall Percent of {symbol}", xlabel="Time", ylabel="%")
plt.show()