Hey everyone!

We know that Alpha Streams can be a bit confusing to new users, and even for experienced users the acceptance/rejection criteria can still be a bit fuzzy. Right now, we're seeing an 80-90% rejection rate, so we've outlined nine of the most common reasons we reject Alpha Streams submissions. Hopefully, this answers some questions you had or clears up any confusion out there. If you have more questions for us, post in the comments below!

  1. Failure to use required API
    • All Alpha Streams submissions are now required to have at least 5-years of backtesting history, which should cover the most recent 5-years up to the present. The results between a submission that doesn't use five years and our test can differ substantially, and so we will reject it so the author can continue to test their model.
    • All Alpha Streams submissions must use the Alpha Streams Brokerage Model. This model is built to model the specific fees, borrowing costs, and other constraints on trading that funds will experience when trading a licensed Alpha. We're continuing to add functionality to this model, so expect some changes in the future.
    • Submissions that use Indicators or Consolidators must use either the SetWarmup or History API. SetWarmup should be used for any Indicators that are specifically assigned to a symbol or registered with the algorithm in the Initialize() method. Any Indicators initialized elsewhere (i.e., in OnSecuritiesChanged) will need to use the History API to warm them up. Using the History or SetWarmup API will ensure that the algorithm is trading under the correct circumstances as soon as it is deployed and will regain state immediately in the case of a restart.
  2. Insights
    • Insights are predictions about the price movement of specific securities and are required in all Alpha Streams submissions. For classic algorithms, one of the Insight constructors must be used and emitted using the EmitInsights API. An Insight must be emitted before any orders are placed. Insights provide funds with information about the predictive power of your models and give them insight into why the orders they see are being placed. For Framework-style algorithms, Insights need to be generated in the Update() method of the Alpha Model. These will then be used in the Portfolio Construction model to build a portfolio.
    • Insights must contain information about the security, the duration of the anticipated price movement, and the direction. Additionally, it is extremely valuable for the Insights to have one or more of the magnitude, confidence, or weighting arguments. These arguments give funds an idea of what the expected movement of a security's price will be, which predictions are particularly relevant to determine optimal capital allocation, and ones that they can act upon with confidence.
  3. Daily Data and Intraday Events
    • We aggregate our Daily data and then passed through the algorithm at 00:00 UTC the day after (i.e., the Daily resolution TradeBar for 2019-10-22 will pass through the algorithm at 2019-10-23 00:00). When using daily data, any operations performed during an intraday event will be using stale data. Using daily data can lead to unexpected results and trades, as well as unrealistic performance. All orders are placed as Market On Open orders, which is often not what people expect when they use Daily data. Not fully understanding Daily data can cause confusion about the behavior of an algorithm and lead to performance that can't be replicated by a fund licensing the Alpha. In general, it is best to use Minute or Hour resolution and a Scheduled Event to perform daily operations to achieve realistic performance.
  4. Open-Source IP
    • We provide the community with lots of example algorithms. The goal of this is to demonstrate how to use the API correctly, incorporate new data sources into their existing algorithms, implement our recommended best practices for Alpha Streams, and more. However, we cannot accept any copies or near-copies of our demonstration algorithms into Alpha Streams. We love to see that our work inspired someone, but each submission must provide sufficient originality that the work can indeed be called the author's own.
  5. External Data Sources
    • When a fund licenses an Alpha, we need to guarantee them that it will perform without error and be of the highest quality possible. To do this, we can't accept any submissions that use an external data source that is not built into Lean (i.e., dropbox files). If we don't directly support the data, then we are unable to guarantee its integrity and that it will be maintained faithfully. We do our best to provide an extensive data library, and we are always working on adding new data sources.
  6. Overfitting
    • Overfitting will doom an algorithm in live trading. To prevent this and try to boost the quality of submissions, we can't accept any Alphas that obviously overfit to data. Overfitting can manifest itself in countless ways, but the most common things we see are
      • Coding of indicator parameters that work for certain hand-picked assets but perhaps not for any others.
      • Using thresholds for indicator values that are hard-coded and have no fundamental theory behind their value.
      • Look-ahead bias, such as hard-coding specific dates to perform specific actions. This can only be done if there is prior knowledge of an event. Look-ahead bias might boost the backtest, but it does not mean the model is fundamentally valuable and can sustain performance during future outliers/extreme events
      • Selection bias, such as picking stocks known ahead of time to perform exceptionally well during specific periods.
        • ETF baskets are common ways to trade, but the more algorithmic your selection process can be (such as using the Coarse/Fine universe selection), the more valuable your model is.
        • Trading just SPY won't provide much value for a fund, but a model that trades various securities, especially across sectors, has substantial value.
    • For more information on common overfitting mistakes, please see our documentation section and this blog post we made about overfitting and the research process.
  7. Slow Recovery
    • Algorithms that fail to recover from a drawdown within 6-months cannot be accepted. Funds understand that all algorithms experience drawdowns, and the smaller, the better, but the recovery time is especially important. An algorithm that is in a sustained drawdown for more than 6-months most likely won't be traded by a fund and would likely be dropped if this occurs in live trading.
  8. Profitability
    • Alphas need to be profitable over the 5+ year backtest. PnL is not the best metric for success or the value of an algorithm, but accepted Alphas must still demonstrate the ability to generate a profit.
  9. Infrequent Insight Generation
    • Algorithms don't need to trade daily or intraday, but the maximum holding period for funds is usually a matter of days or weeks. Anything longer than monthly-rebalancing can't be accepted. Additionally, High-Frequency Trading algorithms might perform in backtesting, but the reality of live borrowing costs means that almost all HFT algorithms fail in live trading. For the competition, algorithms must emit Insights daily or near-daily basis -- for any week in the backtest, there must be at least 5 Insights.