Using Google Trends to Predict Markets

Actually, this would be the accurate backtest. 

Super Interesting! Thanks for sharing!

Really cool! We should try and make this one a LEAN custom data type to avoid any potential look-ahead bias. If the API is fairly robust we can make it available to the whole community with one line of code:

That would be amazing!

Interesting to see this! I wrote a paper on this many years ago actually: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2502508

I think having Google Trends data would be interesting on the QC side.

I remember looking into this a while back and noticed the Google trends data has a look ahead bias. If I remember correctly, the numbers are % of popularity. So if suddenly a topic becomes way more popular than in the past, previous values will change to reflect this new popularity. Backtesting this data, the %'s might have been different at the actual time of the hypothetical trades.

So for example, bitcoin became very popular in 2017, like the picture below. However, if we were back in 2012, those little spikes would have been the 100% mark at the time.

Post above is true but irrelevant -- interesting & out of the box. Thanks for the share OP

Tocht I would say it is quite relevant. Let's say you're building an interface to access this Google Trend data - not only would you need to pass a parameter for the date range of the data (i.e. get me data for Nov 1 2012) but you would also need a second parameter to specify the "simulated date of access".

Looking at the graph above, if you got Nov 2012 data as of today, it would be around 5-10. However, if you got the Nov 2012 data as of, say, Dec 2012, then it would probably be in the ~100 range.

This means thought needs to be put into how to design the data interface. You might be able to get away with simply scaling the data up by the maximum value attained prior to "simulated date of access" - but if Google's scaling is more complex, you may need to somehow store a copy of the data for each "simulated date of access".

Exactly. I don't know if the scaling is just (current value) / (max historical value) or something more complex. And I don't think they give you the raw values.

Another thing to be mindful of are the actual search terms used. For example now we all know bitcoin was super popular in 2017, but if we were to start trading a strategy with bitcoin as a keyword in 2010, we wouldn't know bitcoin would be popular. So a systematic way of choosing keywords needs to be implemented. 

In the example above, would we have known to use the word debt prior to the GFC?

Thank you for the post, I agree that adding that data to lean would be excellent. Is there an idea when this might be available?

For now I guess using the pyinterface of trend data is the way to go?

Hi Guys, I am keen on recreating the above algo without the discussed look ahead bias. I have ~180 google trends CSV files. Would anyone know the best way of getting theses into arrays within QC?

Sorry to hijack the thread, I have created a separate post here: 

Was wondering if anyone ever went through with this plan. I am not the greatest programmer and the world so I am about to manually convert all of the data from 2004 onward and adjust the value of each day by working my way down the intervals google provides and adjusting all values in each particular dataset. If someone already did this or automated this procedure, I would very much appreciate it if you could share some insight on it or just tell me to stop wasting my time lol.

Hi Samuel,

I'd recommend using pandas to automate the data adjustments of historical CSV files. To increase your skills with pandas, checkout our Introduction to Financial Python tutorials.

Best,
Derek Melchin