Centralized Execution

Hi Rubio,

• “I'm looking to build a setup where i decouple the strategy target generation from the target execution.”: yes, this is Alpha and Execution in the Algo Framework.
• “My ideal setup is to have multiple strategies running independently (and concurrently)”: yes, but only if each strategy is inside its own Algo. Well, you can combine multiple Alphas in the same Algo. So it depends on the strategies you are talking about, how complex they are each and thus how fast/slow it would become if all integrated into the same Algo…
• “each generating its own targets on it own schedule and frequency and one centralized process aggregating all strategies targets, matching them internally and sending the aggregated target to the broker.”: if you do multiple Alphas in one Algo, this can be done with how the framework is designed. If you do multiple Algos, then they need to run on separate nodes, so you need to code the glue between them.

Fred

Thanks Fred

We'll do the necessary glue. 

Is a setup like the one below is feasible ? if yes how many nodes are required ?

here i assume :

• Each strategy is inside its own algo, they read the RT feed, Alphas, post orders and get fills from the “RT Persistence” module.
• “RT Alpha Server” module receives RT feed from the broker, computes Alphas and writes them to the “RT Persistence” module.
• The Centralized Execution module reads targets from “RT Persistence” module, match them internally, sends orders to the broker and writes back fills.
• The “RT Persistence” module could be a Redis server for example, that periodically persists into a DB.

I'm quite new to QC, i find it extremely flexible and I'm exploring if we can build our trading operations around it, so all your help is much appreciated.

Rubio

Hi Rubio,

😊

This looks like free corporate software architecture design…

I'll do it! 😊

Have a look first at the Algorithm Framework - this is how, more or less, you would or could model your Algo 1, Algo 2, Algo 3. This certainly has an impact on everything else. https://www.quantconnect.com/docs/algorithm-framework/overview

Now, software architecture is both a science and an art (based on experience and preferences). Maybe the Alpha Server is not doing what I think it does but I would not see it has getting input from the Broker but from the Algos. And then its output would be into the COM and yes, the COM would be bidir with the Broker. So overall, the flow would be a bit more linear. Thus I would opt with using a lightweight technology in the implementation with which fanning out the data to multiple “servers” for scaling up when necessary (load balancing), for redundancy (robustness), and the likes, is very very easy. Some of those technos also give you persistence for free, so no DBs on the side, while still being able to query information. One trick I can also give you is during option analysis and integration testing (start of project and during/end of it), make sure that your entire platform, composed of all selected technos (for the option under study or the selected option, during development), can saturate the network bandwidth (if you choose to distribute this in the classic sense). You want to confirm that the network is your slowest link, at least under fake processing, as it should be.

But, I don't see why your architecture would not be possible. It's rare I see one which seems not possible at all 😊.

Wrt the number of nodes, you would need one trading node per Algo. The other nodes would be yours.

Now, I bet you could also talk with QC's officials as they do deal with professional trading firms and come to agreements I am not aware of 😊.

Fred

Hi Fred

I always found it challenging to combine the strategies into one algo (different universes, different schedules …) so i opted for an architecture like this.

The “RT Alpha Server” is facing the Broker because I assumed the broker is the source of the real time data but indeed it could any real time data provider.

Thanks for your comments, I'll reach out to QC for more details.

Cheers

Rubio

Hi Rubio,

We are, I believe, saying the same thing but we are not understanding us well. 😊 The story of my life - especially right now with teleworking 😊!

1- Dividing strategies in separate algos is the way to go in my opinion also. Just so we know we agree here.

2- If I understand properly, your RT Alpha Server should not use the data feed but should receive trading signals from your algos, that's what I meant. Thus, for me, the “alphas” are another word for “trading signals”, as in, “buy with some confidence, magnitude, etc” or “sell with some confidence, etc”. So your algos use the data feed to derive those signals, give them to your Alpha Server, which acts upon them - while considering all of them as a whole, not just in a dumb one-after-one fashion or else, your entire architecture makes no sense 😊, and to do the actual trading, the “COM” comes into play…

I linked the Algo Framework doc because that architecture is basically exactly the same. The only difference is that you want something to handle multiple Algos at once and make correlations, etc, across alphas (trading signals) from those multiple Algos…

Again, as mentioned, that can already be done with the current Algo Framework - you can integrate multiple AlphaModels in the same Algo. And you can use PortfolioConstructionModels that indeed consider all alphas received over a given period, etc. It only depends on your scale here. Integrating everything into one Algo can stay distinct and manageable on the long run BUT at some point, you can also have too many alphas tracking too many assets, sending too many signals, etc…. You might need a trading node with more RAM and CPU power than what's offered standard… Not discussing potential issues with single points of failure: if your algo crashes, ALL your strategies stop, if you loose the connection with broker, ALL strategies stop trading, etc, etc.

Fred

Hi Rubio Manhattan ,QuantConnect/Lean does not allow your proposed architecture at the moment. We have had some demand for something similar (e.g. here) that we "solved" with Algorithm Framework where instead of different algorithms, there is only one with different Alphas as Fred has brilliantly explained.

Best regards,Alex