-
Notifications
You must be signed in to change notification settings - Fork 17
Open
Labels
UIdata-layerreal-time and historical data processing and storagereal-time and historical data processing and storageintegrationexternal stack and/or lib augmentationsexternal stack and/or lib augmentationstsdbtime series db stufftime series db stuff
Description
with #308 landing shortly we'll have daemon support for storing large time series as part of a trader's normal toolset. one of the main utilities i'm sure most would expect is the ability to edit data sets in a way that's both intuitive and simple.
UX ideas
- allow user selection of datums on a chart and export to a repl (likely
ipythonshell) which has all the user's computational tools as well as a standard set of tsdb mgmt APIs for "fixing" data that was either written wrong or for overlaying data sets on a whim - the mgmt APIs will likely want to include:
- timeseries diffing and duplication detection (the latter for when bad writes happen due to bugs or mistakes)
- this SO
numbasolution is probably handy: https://stackoverflow.com/a/71854903
- this SO
- tsdb write/delete ability and "reload" functionality, possibly by context-menu style right click
- export to a format suitable for network transport?
- gap detection for attempting to find missing history frame segments
- timeseries diffing and duplication detection (the latter for when bad writes happen due to bugs or mistakes)
- a repl (prolly
ipython) which wraps all this in a suitable env (with some extensions?) for doing all the above easily from cli (so likely we'll needIPythonintegration 😎 goodboy/tractor#306)
main holdups
-
marketstoredoesn't seem to have any obvious way to delete series segments (https://github.com/alpacahq/marketstore/issues/264) - it also has no way to query granularity (https://github.com/alpacahq/marketstore/issues/207) though we are not really planning to ever require dowsampling more then 1m period
Probably more to come as this gets researched more.
Metadata
Metadata
Assignees
Labels
UIdata-layerreal-time and historical data processing and storagereal-time and historical data processing and storageintegrationexternal stack and/or lib augmentationsexternal stack and/or lib augmentationstsdbtime series db stufftime series db stuff