Volume data is missing on those Forex & CFD product

Volume in FOREX is a tricky concept. There is no centralized market place like in equities, so the volumes are only local to each market maker. Sometimes this is the interbank rate - with interbank volumes (FXCM), and sometimes its the market-maker's internal trading volume (like with Oanda). We don't have CFD's yet; and we default to using FXCM's market data at the moment which comes with a bid/ask spread only. TradeBar's are created using the midpoints on that bid ask spread. Through the API you can ask for Oanda's market data as well. We're waiting on Oanda to give us daily updates for their data, but once that is available we'll publish how to access Oanda spreads.

Jared Why don't you guys use Integral's TrueFX free historical data? Or LMAX?

As we integrate with each brokerage we'll add their specific data. Each market maker has their own spreads so its important to use the right data for backtesting. For now we only integrate with FXCM and Oanda brokerages -- so only offer their data.

Given the availability of tick data from e.g. FXCM, you should be able to calculate a very rough "volume" yourself, by simply counting the number of ticks that have come in during any given [minute/hour/day]. I'm looking into how to do this and will share if I come across anything that seems doable.

Hi Douglas, thank you for looking into it.
Since that conversation, we have implemented a custom data type for FxcmVolume and a downloader to fetch and convert the data. Please take a look into it and check whether it does what you are trying to do.
However, this data is not available in QuantConnect. You can only use it for local R&D.

Very interesting, thank you! Did not spot this before.

Hi Alexandre,

Are there any plans to bring this into QuantConnect? I can call upon it in the research enviroment, however the data does only up to November 2017.

Hi Ralph Clayton, we don't have plans to add this data.
However, since you can get this data with the downloader and FXCM data is free, you can run a Jupyter notebook locally.

Hi Alexandre, Douglas, and others--

Sorry, i'm still learning.

How can the downloader this to say--Download EURUSD 1h or 1m resolution volume for as much historical data as is available? 

What do I change in LEAN to run it and download these? I'm familiar with the API data downloader, but not with this.

You will need to compile the Toolbox project, then execute Lean/Toolbox/bin/Debug/QuantConnect.ToolBox.exe and follow the instructions at Toolbox readme.

Thanks Alexandre,

I've gotten it to launch, but whenever I enter anything, even "--help", it immediately exits.

Is it working in the last lean build? I just downloaded that version

Andre Stevens, I have just launched the downloader successfully:

Please note that you need to place a config.json file in Lean\Toolbox\bin\Debug with the FXCM credentials.

Hi, sorry for bothering again but I'm still not able to get the data and im unsure why my last post isn't visible (maybe too long, so I cut the debug paste a bit).

I copy+pasted my config.json into the lean/toolbox/bin/debug (default FXCM credentials that are already there?) as suggested.

The launcher opens, but whenever I type anything even like --help, it exits immediately.

I also tried pasting your commend --app fvdl --tickers=EURUSD --from-date=20180101-00:00:00 --resolution=Daily

Can someone help me out? Here's the end of the output from debug, thoug there's not a lot of information here.

'QuantConnect.ToolBox.exe' (CLR v4.0.30319: QuantConnect.ToolBox.exe): Loaded 'C:\Users\Andre\Downloads\Lean-masterRetry3\ToolBox\bin\Debug\Microsoft.Extensions.CommandLineUtils.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
The program '[27084] QuantConnect.ToolBox.exe' has exited with code 0 (0x0).

Even can someone upload the 1m EURUSD volume to dropbox or something and share it (if that isn't against the FXCM TOS of course)

Regards,

Andre

Hi Andre Stevens, I don't think that these messages are error messages.
It is odd that you didn't get this message:
Config.Get(): Configuration key not found. Key: data-directory - Using default value: ../../../Data
But if data-directory is defined, it shouldn't.

default FXCM credentials that are already there?
No, you have to use your FXCM credentials.

> Hi Ralph Clayton, we don't have plans to add this data.

Care to elaborate on this? Jared's absolutely correct about fx volume being tricky in that it's specific to particular market/broker, but FXCM does provide the volume data over their api; believe IB does the same.

Sorry Basher, we don't have plans to add this data yet.
Since FXCM provides volume data over their API, users can rely on the Importing Custom Data feature to import this information into the algorithm.