Optimization

Follow these steps to run a local optimization:

1. Set up your local data for all the data required by your project.
2. Convert your project to use project parameters instead of constants for all values that must be optimized.
3. Open a terminal in the organization workspace that contains the project.
4. Run lean optimize "<projectName>" to start optimizing the project in . / <projectName>. This command starts an interactive wizard which lets you configure the optimizer.

   $ lean optimize "My Project"
   Select the optimization strategy to use:
   1) Grid Search
   2) Euler Search
   Enter an option:

5. Enter the number of the optimization strategy to use. You can either choose for Grid Search, which runs through all possible combinations of parameters, or for Euler Search, which performs an Euler-like which gradually works towards smaller optimizations.

   $ lean optimize "My Project"
   Select the optimization strategy to use:
   1) Grid Search
   2) Euler Search
   Enter an option: 1

6. Enter the number of the optimization target to use. The target specifies what statistic you want to optimize and whether you want to minimize or maximize it.

   $ lean optimize "My Project"
   Select an optimization target:
   1) Sharpe Ratio (min)
   2) Sharpe Ratio (max)
   3) Compounding Annual Return (min)
   4) Compounding Annual Return (max)
   5) Probabilistic Sharpe Ratio (min)
   6) Probabilistic Sharpe Ratio (max)
   7) Drawdown (min)
   8) Drawdown (max)
   Enter an option: 2

7. For each parameter, enter whether you want to optimize it and what its values can be.

   $ lean optimize "My Project"
   Should the 'ema-fast' parameter be optimized? [Y/n]: y
   Minimum value for 'ema-fast': 5
   Maximum value for 'ema-fast': 10
   Step size for 'ema-fast' [1.0]: 1
   Should the 'ema-medium' parameter be optimized? [Y/n]: y
   Minimum value for 'ema-medium': 25
   Maximum value for 'ema-medium': 30
   Step size for 'ema-medium' [1.0]: 1
   Should the 'ema-slow' parameter be optimized? [Y/n]: y
   Minimum value for 'ema-slow': 45
   Maximum value for 'ema-slow': 50
   Step size for 'ema-slow' [1.0]: 1

8. Enter the constraints of the optimization. An example optimization is "Drawdown <= 0.25", which discards all parameter combinations resulting in a drawdown higher than 25%.

   $ lean optimize "My Project"
   Current constraints: None
   Do you want to add a constraint? [y/N]: y
   Select a constraint target:
   1) Sharpe Ratio
   2) Compounding Annual Return
   3) Probabilistic Sharpe Ratio
   4) Drawdown
   Enter an option: 4
   Select a constraint operator (<value> will be asked after this):
   1) Less than <value>
   2) Less than or equal to <value>
   3) Greater than <value>
   4) Greater than or equal to <value>
   5) Equal to <value>
   6) Not equal to <value>
   Enter an option: 2
   Set the <value> for the selected operator: 0.25
   Current constraints: TotalPerformance.PortfolioStatistics.Drawdown <= 0.25
   Do you want to add a constraint? [y/N]: n

   After configuring the constraints the optimizer starts running.
9. View the results in the terminal after the optimizer finished. The logs contains the optimal parameter combination.

   $ lean optimize "My Project"
   20220223 18:26:20.000 TRACE:: Program.Main(): Exiting Lean...
   20220223 18:26:20.079 TRACE:: LeanOptimizer.TriggerOnEndEvent(OID 2313bbba-9c71-4b9e-8b91-c70cc117b0c7): Optimization has ended. Result for Target: ['TotalPerformance'].['PortfolioStatistics'].['SharpeRatio'] at: 3.6205: was reached using ParameterSet: (ema-slow:47,ema-medium:26,ema-fast:5) backtestId '21c30000-dc5a-4dec-b75a-da5b1796ccba'. Constraints: (['TotalPerformance'].['PortfolioStatistics'].['Drawdown'] 'LessOrEqual' 0.25)
   Optimal parameters: ema-slow: 47, ema-medium: 26, ema-fast: 5
   Successfully optimized 'My Project' and stored the output in 'My Project/optimizations/2021-03-24_00-22-15'
   

10. View the individual backtest results in the <project> / optimizations / <timestamp> directory. Results are stored in JSON files and can be analyzed in a local research environment. You can save results to a different directory by providing the --output <path> option in step 4.

    $ lean optimize "My Project" --output "My Project/custom-output"
    20220223 18:28:20.000 TRACE:: Program.Main(): Exiting Lean...
    20220223 18:28:20.079 TRACE:: LeanOptimizer.TriggerOnEndEvent(OID 1ac5e638-aae0-4aa9-80d4-02c51bb7b84d): Optimization has ended. Result for Target: ['TotalPerformance'].['PortfolioStatistics'].['SharpeRatio'] at: 3.6205: was reached using ParameterSet: (ema-slow:47,ema-medium:26,ema-fast:5) backtestId 'e2aa3abf-bb60-4e91-a281-59c882ada62f'. Constraints: (['TotalPerformance'].['PortfolioStatistics'].['Drawdown'] 'LessOrEqual' 0.25)
    Optimal parameters: ema-slow: 47, ema-medium: 26, ema-fast: 5
    Successfully optimized 'My Project' and stored the output in 'My Project/custom-output'

By default, local optimizations run in the LEAN engine in the quantconnect/lean Docker image. This Docker image contains all the libraries available on QuantConnect, meaning your algorithm also has access to those libraries. If the specified project is a C# project it is first compiled using the same Docker image. See Project Libraries to learn how to use project libraries, and Custom Docker Images to learn how to build and use custom Docker images.

When you run a local optimization, the default data provider is your local machine. To use data from QuantConnect Cloud, a brokerage, or a third-party data provider, include the --data-provider-historical option. To view what brokerages and third-party data providers are available, see the reference page for this command.

When you run an optimization in QuantConnect Cloud, it uses the data from the Dataset Market. Follow these steps to run a cloud optimization:

1. Log in to the CLI if you haven't done so already.
2. Convert your project to use project parameters instead of constants for all values that must be optimized.
3. Open a terminal in the organization workspace that contains the project.
4. Run lean cloud optimize "<projectName>" --push to push . / <projectName> to the cloud and start optimizing the project in the cloud.

   $ lean cloud optimize "My Project" --push
   [1/1] Pushing 'My Project'
   Successfully updated cloud file 'My Project/main.py'
   Started compiling project 'My Project'
   Successfully compiled project 'My Project'
   Select an optimization target:
   1) Sharpe Ratio (min)
   2) Sharpe Ratio (max)
   3) Compounding Annual Return (min)
   4) Compounding Annual Return (max)
   5) Probabilistic Sharpe Ratio (min)
   6) Probabilistic Sharpe Ratio (max)
   7) Drawdown (min)
   8) Drawdown (max)
   Enter an option:

5. Enter the number of the optimization target to use. The target specifies what statistic you want to optimize and whether you want to minimize or maximize it.

   $ lean cloud optimize "My Project" --push
   Select an optimization target:
   1) Sharpe Ratio (min)
   2) Sharpe Ratio (max)
   3) Compounding Annual Return (min)
   4) Compounding Annual Return (max)
   5) Probabilistic Sharpe Ratio (min)
   6) Probabilistic Sharpe Ratio (max)
   7) Drawdown (min)
   8) Drawdown (max)
   Enter an option: 2

6. For each parameter, enter whether you want to optimize it and what its values can be.

   $ lean cloud optimize "My Project" --push
   Should the 'ema-fast' parameter be optimized? [Y/n]: y
   Minimum value for 'ema-fast': 1
   Maximum value for 'ema-fast': 10
   Step size for 'ema-fast' [1.0]: 1
   Should the 'ema-slow' parameter be optimized? [Y/n]: y
   Minimum value for 'ema-slow': 21
   Maximum value for 'ema-slow': 30
   Step size for 'ema-slow' [1.0]: 1

7. Enter the constraints of the optimization. An example optimization is "Drawdown <= 0.25", which discards all parameter combinations resulting in a drawdown higher than 25%.

   $ lean cloud optimize "My Project" --push
   Current constraints: None
   Do you want to add a constraint? [y/N]: y
   Select a constraint target:
   1) Sharpe Ratio
   2) Compounding Annual Return
   3) Probabilistic Sharpe Ratio
   4) Drawdown
   Enter an option: 4
   Select a constraint operator (<value> will be asked after this):
   1) Less than <value>
   2) Less than or equal to <value>
   3) Greater than <value>
   4) Greater than or equal to <value>
   5) Equal to <value>
   6) Not equal to <value>
   Enter an option: 2
   Set the <value> for the selected operator: 0.25
   Current constraints: TotalPerformance.PortfolioStatistics.Drawdown <= 0.25
   Do you want to add a constraint? [y/N]: n

8. Enter the number of the optimization node type to use.

   $ lean cloud optimize "My Project" --push
   Select the optimization node type:
   1) O2-8 (2 cores, 8 GB RAM) @ $0.15 per hour
   2) O4-12 (4 cores, 12 GB RAM) @ $0.30 per hour
   3) O8-16 (8 cores, 16 GB RAM) @ $0.60 per hour
   Enter an option: 2

9. Enter the number of nodes that should run in parallel.

   $ lean cloud optimize "My Project" --push
   How many nodes should run in parallel (1-12) [6]: 10

10. Confirm the given input to start the optimizer.

    $ lean cloud optimize "My Project" --push
    Estimated number of backtests: 100
    Estimated batch time: 8 minutes
    Estimated batch cost: $0.38
    Organization balance: 173,368 QCC ($1,733.68)
    Do you want to start the optimization on the selected node type? [Y/n]: y
    

11. Inspect the optimal parameter combination and the full statistics of the backtest that ran with this combination at the bottom of the logs when the optimizer has finished.