Datasets

When running LEAN locally using the CLI, you can already use all of the features explained in the Importing Data page of the LEAN documentation. However, sometimes you may not want to upload your file to a cloud storage service like Dropbox. You can follow these steps to convert your custom data class (the class extending BaseDataPythonData) to retrieve data from a local file instead of a remote one:

1. Copy the data file that you want to use to the data directory in your organization workspace.
2. Open the source file containing your custom data class in an editor.
3. Update your GetSourceget_source method to load data from the local file in your data directory. Make sure you only use forward slashes. Backward slashes as path separators don't work. For the Weather example in the LEAN documentation, that is done like this:

   using System;
   using System.IO;
   using QuantConnect.Data;
   
   namespace QuantConnect.Algorithm.CSharp
   {
       public class Weather : BaseData
       {
           public override SubscriptionDataSource GetSource(SubscriptionDataConfig config, DateTime date, bool isLive)
           {
               // Old:
               // var source = "https://www.dropbox.com/s/8v6z949n25hyk9o/custom_weather_data.csv?dl=1";
               // return new SubscriptionDataSource(source, SubscriptionTransportMedium.RemoteFile);
   
               // New:
               // Replace custom_weather_data.csv with the path to your data file in the data directory
               var source = Path.Combine(Globals.DataFolder, "custom_weather_data.csv");
               return new SubscriptionDataSource(source, SubscriptionTransportMedium.LocalFile);
           }
       }
   }

   import os
   
   from QuantConnect import Globals, SubscriptionTransportMedium
   from QuantConnect.Data import SubscriptionDataSource
   from QuantConnect.Python import PythonData
   
   
   class Weather(PythonData):
       def get_source(self, config: SubscriptionDataConfig, date: datetime, is_live: bool) -> SubscriptionDataSource:
           # Old:
           # source = "https://www.dropbox.com/s/8v6z949n25hyk9o/custom_weather_data.csv?dl=1"
           # return SubscriptionDataSource(source, SubscriptionTransportMedium.REMOTE_FILE)
   
           # New:
           # Replace custom_weather_data.csv with the path to your data file in the data directory
           source = os.path.join(Globals.DATA_FOLDER, "custom_weather_data.csv")
           return SubscriptionDataSource(source, SubscriptionTransportMedium.LOCAL_FILE)

4. Save the source file.