combine existing file reader with live data feed

[bournejt]

If historical data is saved as csv files and live feed is from websocket, can we build a PushPullInputAdapter utilizing existing CSVReader? Right now, I can think of 2 solutions:

1. Build a unified data service with one interface so we only need one adapter.
2. Manually read in csv files in the PushPullInputAdapter and switch to websocket feed for the live part.

Both solutions loses some flexibility of existing tools. Is there any way for me to just combine CSVReader with a live websocket reader to form a new PushPullInputAdapter?

[AdamGlustein]

You can build a PushPullInputAdapter that replays data from csv and then, once the data is exhausted, switches to a websocket source. You can do this purely from Python - unfortunately, I don't see an easy way to programatically extend the existing websocket push adapter or the csv pull adapter to get this behaviour, so you'd need to likely duplicate the code for your purposes.

[bournejt]

@AdamGlustein Thanks for confirming my understanding. I think I'll go with unified data service interface.

[robambalu]

I dont see any easy way to share the adapter code to accomplish this. The more sane approach to this is to write a node that merges the two streams. Ideally you would have some sort of sequence # that allows you to stitch them properly