Skip to content

Relocating TSFM-Specifc CSV Files to CouchDB #406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jellyfishing2346 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Relocating TSFM-Specifc CSV Files to CouchDB #406

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4df1509
feat(couchdb): add tsfm collection and relocate CSV files from local …
jellyfishing2346 Jun 23, 2026
2eb3481
Merge remote-tracking branch 'upstream/main' into main
jellyfishing2346 Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/data.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,8 @@ src/couchdb/
└── shared/ data common to many scenarios
├── work_order/workorders.csv
├── iot/*.json
└── failure_code/failure_code_sample.csv
├── failure_code/failure_code_sample.csv
└── tsfm/*.csv
```

A scenario folder is fully self-contained: its `manifest.json`, its prompt, and any data
Expand Down
14 changes: 7 additions & 7 deletions docs/guideline/case_study_industrial_asset_management.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,15 +156,15 @@ This section shows how the generic categories from the [main guideline](utteranc

**Forecasting Queries**:
- "What is the forecast for 'Chiller 9 Condenser Water Flow' in the week of 2020-04-27?"
- "Forecast 'Chiller 9 Condenser Water Flow' using data in 'chiller9_annotated_small_test.csv'."
- "Use data in 'chiller9_annotated_small_test.csv' to forecast with 'Timestamp' as a timestamp."
- "Forecast 'Chiller 9 Condenser Water Flow' using data from CouchDB collection 'tsfm' with file 'chiller9_annotated_small_test.csv'."
- "Use data from CouchDB collection 'tsfm' with file 'chiller9_annotated_small_test.csv' to forecast with 'Timestamp' as a timestamp."

**Model Fine-tuning**:
- "Finetune a forecasting model for 'Chiller 9 Condenser Water Flow' using data in 'chiller9_finetuning_small.csv'."
- "Finetune a forecasting model for 'Chiller 9 Condenser Water Flow' using data from CouchDB collection 'tsfm' with file 'chiller9_finetuning_small.csv'."

#### Anomaly Detection Examples

- "I need to perform Time Series anomaly detection of 'Chiller 9 Condenser Water Flow' using data in chiller9_tsad.csv."
- "I need to perform Time Series anomaly detection of 'Chiller 9 Condenser Water Flow' using data from CouchDB collection 'tsfm' with file 'chiller9_tsad.csv'."
- "Is there any anomaly detected in Chiller 6's Tonnage in the week of 2020-04-27?"

#### Failure Prediction Examples
Expand Down Expand Up @@ -383,14 +383,14 @@ Each utterance should follow this enhanced schema with 9 required fields in this
```json
{
"id": 217,
"text": "Forecast 'Chiller 9 Condenser Water Flow' using data in 'chiller9_annotated_small_test.csv'. Use parameter 'Timestamp' as a timestamp.",
"text": "Forecast 'Chiller 9 Condenser Water Flow' using data from CouchDB collection 'tsfm' with file 'chiller9_annotated_small_test.csv'. Use parameter 'Timestamp' as a timestamp.",
"type": "TSFM",
"category": "Inference Query",
"deterministic": true,
"characteristic_form": "The expected response should be: Forecasting results of 'Chiller 9 Condenser Water Flow' using data in 'chiller9_annotated_small_test.csv' are stored in json file",
"characteristic_form": "The expected response should be: Forecasting results of 'Chiller 9 Condenser Water Flow' using data from CouchDB collection 'tsfm' with file 'chiller9_annotated_small_test.csv' are stored in json file",
"group": "predictive",
"entity": "Chiller",
"note": "Source: Predictive maintenance initiative; Owner: Data Science Team; Requires time series forecasting model; Implementation: Uses TSFM framework with timestamp parameter"
"note": "Source: Predictive maintenance initiative; Owner: Data Science Team; Requires time series forecasting model; Implementation: Uses TSFM framework with timestamp parameter; Data loaded from CouchDB tsfm collection"
}
```

Expand Down
34 changes: 18 additions & 16 deletions docs/guideline/ground_truth_design_guideline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1335,7 +1335,7 @@ This section presents real-world patterns from AssetOpsBench ground truth scenar
"start": "2020-04-27T00:00:00Z",
"final": "2020-05-03T23:59:59Z",
"save_series_csv": true,
"save_path": "data/raw/iot/main/chiller_9_condenser_water_flow_2020-04-27_to_2020-05-03.csv"
"save_path": "shared/tsfm/chiller9_annotated_small_test.csv"
},
"outputs": {
"status": "failure",
Expand All @@ -1356,28 +1356,30 @@ This section presents real-world patterns from AssetOpsBench ground truth scenar
"start": "2020-04-27T00:00:00Z",
"final": "2020-05-03T23:59:59Z",
"save_series_csv": true,
"save_path": "data/raw/iot/main/chiller_9_condenser_water_flow_2020-04-27_to_2020-05-03.csv"
"save_path": "shared/tsfm/chiller9_annotated_small_test.csv"
},
"outputs": {
"status": "success",
"rows": 581,
"sampling_hint": "irregular/partial (expected ~672 @ 15min)",
"csv_path": "data/raw/iot/main/chiller_9_condenser_water_flow_2020-04-27_to_2020-05-03.csv",
"rows": 192,
"sampling_hint": "regular @ 15min",
"csv_path": "shared/tsfm/chiller9_annotated_small_test.csv",
"message": "Series retrieved and saved to CSV."
},
"deterministic": {"name": false, "action": false, "arguments": false, "outputs": false}
},
{
"name": "jsonreader",
"agent": "IoTAgent",
"action": "DirectFileRead_Attempt",
"name": "csvreader",
"agent": "TSFMAgent",
"action": "LoadCSV_FromCouchDB",
"arguments": {
"file_name": "chiller9_condenser_water_flow_MAIN_2020-04-27.json"
"collection": "tsfm",
"file": "chiller9_annotated_small_test.csv"
},
"outputs": {
"status": "failure",
"error": "[Errno 2] No such file or directory: 'chiller9_condenser_water_flow_MAIN_2020-04-27.json'",
"message": "Direct JSON read failed; proceed with CSV path from history step."
"status": "success",
"rows": 192,
"columns": ["Timestamp", "Chiller_9_Condenser_Water_Flow"],
"message": "CSV loaded from CouchDB tsfm collection."
},
"deterministic": {"name": false, "action": false, "arguments": false, "outputs": false}
},
Expand All @@ -1404,9 +1406,9 @@ This section presents real-world patterns from AssetOpsBench ground truth scenar
"agent": "TSFMAgent",
"action": "RunForecast_ttm_96_28_WithCorrectArgs",
"arguments": {
"input_csv": "data/raw/iot/main/chiller_9_condenser_water_flow_2020-04-27_to_2020-05-03.csv",
"timestamp_col": "timestamp",
"value_col": "value",
"input_csv": "shared/tsfm/chiller9_annotated_small_test.csv",
"timestamp_col": "Timestamp",
"value_col": "Chiller_9_Condenser_Water_Flow",
"window_start": "2020-04-27T00:00:00Z",
"window_end": "2020-05-03T23:59:59Z",
"checkpoint": "ttm_96_28",
Expand All @@ -1429,7 +1431,7 @@ This section presents real-world patterns from AssetOpsBench ground truth scenar
"action": "Finish",
"arguments": {
"validation_checks": [
"exists:data/raw/iot/main/chiller_9_condenser_water_flow_2020-04-27_to_2020-05-03.csv",
"exists:shared/tsfm/chiller9_annotated_small_test.csv",
"exists:data/derived/forecast/chiller_9/condenser_water_flow/forecast_2020-04-27_to_2020-05-03_ttm_96_28.json"
],
"summary_request": "Confirm variable binding, input read, forecast span, row counts, and artifact paths. Note prior failure remediation (assetnum correction; correct TSFM input schema)."
Expand Down
3 changes: 3 additions & 0 deletions src/couchdb/.allowed_datafiles
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ src/couchdb/scenarios_data/shared/iot/metro_pump_1.json
src/couchdb/scenarios_data/shared/iot/motor_01.json
src/couchdb/scenarios_data/shared/work_order/workorders.csv
src/couchdb/scenarios_data/shared/iot/asset_profile_sample.json
src/couchdb/scenarios_data/shared/tsfm/chiller9_annotated_small_test.csv
src/couchdb/scenarios_data/shared/tsfm/chiller9_finetuning_small.csv
src/couchdb/scenarios_data/shared/tsfm/chiller9_tsad.csv
8 changes: 8 additions & 0 deletions src/couchdb/collections.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,5 +110,13 @@
],
"id_prefix": "fc",
"indexes": []
},
"tsfm": {
"format": "csv",
"primary_key": [
"Timestamp"
],
"id_prefix": "tsfm",
"indexes": []
}
}
7 changes: 6 additions & 1 deletion src/couchdb/scenarios_data/default/manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,10 @@
],
"asset": "shared/iot/asset_profile_sample.json",
"vibration": "shared/iot/motor_01.json",
"failurecode": "shared/failure_code/failure_code_sample.csv"
"failurecode": "shared/failure_code/failure_code_sample.csv",
"tsfm": [
"shared/tsfm/chiller9_annotated_small_test.csv",
"shared/tsfm/chiller9_finetuning_small.csv",
"shared/tsfm/chiller9_tsad.csv"
]
}
7 changes: 6 additions & 1 deletion src/couchdb/scenarios_data/scenario_1/manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
{
"workorder": "shared/work_order/workorders.csv"
"workorder": "shared/work_order/workorders.csv",
"tsfm": [
"shared/tsfm/chiller9_annotated_small_test.csv",
"shared/tsfm/chiller9_finetuning_small.csv",
"shared/tsfm/chiller9_tsad.csv"
]
}
7 changes: 6 additions & 1 deletion src/couchdb/scenarios_data/scenario_2/manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
{
"workorder": "shared/work_order/workorders.csv"
"workorder": "shared/work_order/workorders.csv",
"tsfm": [
"shared/tsfm/chiller9_annotated_small_test.csv",
"shared/tsfm/chiller9_finetuning_small.csv",
"shared/tsfm/chiller9_tsad.csv"
]
}
193 changes: 193 additions & 0 deletions src/couchdb/scenarios_data/shared/tsfm/chiller9_annotated_small_test.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
Timestamp,Chiller_9_Condenser_Water_Flow
2020-04-27T00:00:00,6047.8850449535175
2020-04-27T00:15:00,8082.3336510349445
2020-04-27T00:30:00,1864.6537050444165
2020-04-27T00:45:00,5123.1234567890123
2020-04-27T01:00:00,6234.5678901234567
2020-04-27T01:15:00,5789.0123456789012
2020-04-27T01:30:00,6456.7890123456789
2020-04-27T01:45:00,5890.1234567890123
2020-04-27T02:00:00,6123.4567890123456
2020-04-27T02:15:00,5987.6543210987654
2020-04-27T02:30:00,6345.6789012345678
2020-04-27T02:45:00,5765.4321098765432
2020-04-27T03:00:00,6234.5678901234567
2020-04-27T03:15:00,5876.5432109876543
2020-04-27T03:30:00,6456.7890123456789
2020-04-27T03:45:00,5901.2345678901234
2020-04-27T04:00:00,6178.9012345678901
2020-04-27T04:15:00,5834.5678901234567
2020-04-27T04:30:00,6412.3456789012345
2020-04-27T04:45:00,5923.4567890123456
2020-04-27T05:00:00,6189.0123456789012
2020-04-27T05:15:00,5798.7654321098765
2020-04-27T05:30:00,6367.8901234567890
2020-04-27T05:45:00,5945.6789012345678
2020-04-27T06:00:00,6156.7890123456789
2020-04-27T06:15:00,5812.3456789012345
2020-04-27T06:30:00,6389.0123456789012
2020-04-27T06:45:00,5967.8901234567890
2020-04-27T07:00:00,6134.5678901234567
2020-04-27T07:15:00,5823.4567890123456
2020-04-27T07:30:00,6378.9012345678901
2020-04-27T07:45:00,5989.0123456789012
2020-04-27T08:00:00,6145.6789012345678
2020-04-27T08:15:00,5834.5678901234567
2020-04-27T08:30:00,6367.8901234567890
2020-04-27T08:45:00,5978.9012345678901
2020-04-27T09:00:00,6156.7890123456789
2020-04-27T09:15:00,5845.6789012345678
2020-04-27T09:30:00,6356.7890123456789
2020-04-27T09:45:00,5989.0123456789012
2020-04-27T10:00:00,6167.8901234567890
2020-04-27T10:15:00,5856.7890123456789
2020-04-27T10:30:00,6345.6789012345678
2020-04-27T10:45:00,5998.9012345678901
2020-04-27T11:00:00,6178.9012345678901
2020-04-27T11:15:00,5867.8901234567890
2020-04-27T11:30:00,6334.5678901234567
2020-04-27T11:45:00,6007.8901234567890
2020-04-27T12:00:00,6189.0123456789012
2020-04-27T12:15:00,5878.9012345678901
2020-04-27T12:30:00,6323.4567890123456
2020-04-27T12:45:00,6016.7890123456789
2020-04-27T13:00:00,6198.9012345678901
2020-04-27T13:15:00,5889.0123456789012
2020-04-27T13:30:00,6312.3456789012345
2020-04-27T13:45:00,6025.6789012345678
2020-04-27T14:00:00,6207.8901234567890
2020-04-27T14:15:00,5898.9012345678901
2020-04-27T14:30:00,6301.2345678901234
2020-04-27T14:45:00,6034.5678901234567
2020-04-27T15:00:00,6216.7890123456789
2020-04-27T15:15:00,5907.8901234567890
2020-04-27T15:30:00,6290.1234567890123
2020-04-27T15:45:00,6043.4567890123456
2020-04-27T16:00:00,6225.6789012345678
2020-04-27T16:15:00,5916.7890123456789
2020-04-27T16:30:00,6278.9012345678901
2020-04-27T16:45:00,6052.3456789012345
2020-04-27T17:00:00,6234.5678901234567
2020-04-27T17:15:00,5925.6789012345678
2020-04-27T17:30:00,6267.8901234567890
2020-04-27T17:45:00,6061.2345678901234
2020-04-27T18:00:00,6243.4567890123456
2020-04-27T18:15:00,5934.5678901234567
2020-04-27T18:30:00,6256.7890123456789
2020-04-27T18:45:00,6070.1234567890123
2020-04-27T19:00:00,6252.3456789012345
2020-04-27T19:15:00,5943.4567890123456
2020-04-27T19:30:00,6245.6789012345678
2020-04-27T19:45:00,6078.9012345678901
2020-04-27T20:00:00,6261.2345678901234
2020-04-27T20:15:00,5952.3456789012345
2020-04-27T20:30:00,6234.5678901234567
2020-04-27T20:45:00,6087.6789012345678
2020-04-27T21:00:00,6270.0123456789012
2020-04-27T21:15:00,5961.2345678901234
2020-04-27T21:30:00,6223.4567890123456
2020-04-27T21:45:00,6096.5678901234567
2020-04-27T22:00:00,6278.9012345678901
2020-04-27T22:15:00,5970.1234567890123
2020-04-27T22:30:00,6212.3456789012345
2020-04-27T22:45:00,6105.4567890123456
2020-04-27T23:00:00,6287.8901234567890
2020-04-27T23:15:00,5979.0123456789012
2020-04-27T23:30:00,6201.2345678901234
2020-04-27T23:45:00,6114.3456789012345
2020-04-28T00:00:00,6296.7890123456789
2020-04-28T00:15:00,5987.9012345678901
2020-04-28T00:30:00,6190.1234567890123
2020-04-28T00:45:00,6123.2345678901234
2020-04-28T01:00:00,6305.6789012345678
2020-04-28T01:15:00,5996.7890123456789
2020-04-28T01:30:00,6179.0123456789012
2020-04-28T01:45:00,6132.1234567890123
2020-04-28T02:00:00,6314.5678901234567
2020-04-28T02:15:00,6005.6789012345678
2020-04-28T02:30:00,6167.9012345678901
2020-04-28T02:45:00,6141.0123456789012
2020-04-28T03:00:00,6323.4567890123456
2020-04-28T03:15:00,6014.5678901234567
2020-04-28T03:30:00,6156.7901234567890
2020-04-28T03:45:00,6149.9012345678901
2020-04-28T04:00:00,6332.3456789012345
2020-04-28T04:15:00,6023.4567890123456
2020-04-28T04:30:00,6145.6789012345678
2020-04-28T04:45:00,6158.7901234567890
2020-04-28T05:00:00,6341.2345678901234
2020-04-28T05:15:00,6032.3456789012345
2020-04-28T05:30:00,6134.5678901234567
2020-04-28T05:45:00,6167.6789012345678
2020-04-28T06:00:00,6350.1234567890123
2020-04-28T06:15:00,6041.2345678901234
2020-04-28T06:30:00,6123.4567890123456
2020-04-28T06:45:00,6176.5678901234567
2020-04-28T07:00:00,6359.0123456789012
2020-04-28T07:15:00,6050.1234567890123
2020-04-28T07:30:00,6112.3456789012345
2020-04-28T07:45:00,6185.4567890123456
2020-04-28T08:00:00,6367.8901234567890
2020-04-28T08:15:00,6058.9012345678901
2020-04-28T08:30:00,6101.2345678901234
2020-04-28T08:45:00,6194.3456789012345
2020-04-28T09:00:00,6376.7890123456789
2020-04-28T09:15:00,6067.8901234567890
2020-04-28T09:30:00,6090.1234567890123
2020-04-28T09:45:00,6203.2345678901234
2020-04-28T10:00:00,6385.6789012345678
2020-04-28T10:15:00,6076.7890123456789
2020-04-28T10:30:00,6079.0123456789012
2020-04-28T10:45:00,6212.1234567890123
2020-04-28T11:00:00,6394.5678901234567
2020-04-28T11:15:00,6085.6789012345678
2020-04-28T11:30:00,6067.9012345678901
2020-04-28T11:45:00,6221.0123456789012
2020-04-28T12:00:00,6403.4567890123456
2020-04-28T12:15:00,6094.5678901234567
2020-04-28T12:30:00,6056.7890123456789
2020-04-28T12:45:00,6229.9012345678901
2020-04-28T13:00:00,6412.3456789012345
2020-04-28T13:15:00,6103.4567890123456
2020-04-28T13:30:00,6045.6789012345678
2020-04-28T13:45:00,6238.7901234567890
2020-04-28T14:00:00,6421.2345678901234
2020-04-28T14:15:00,6112.3456789012345
2020-04-28T14:30:00,6034.5678901234567
2020-04-28T14:45:00,6247.6789012345678
2020-04-28T15:00:00,6430.1234567890123
2020-04-28T15:15:00,6121.2345678901234
2020-04-28T15:30:00,6023.4567890123456
2020-04-28T15:45:00,6256.5678901234567
2020-04-28T16:00:00,6439.0123456789012
2020-04-28T16:15:00,6130.1234567890123
2020-04-28T16:30:00,6012.3456789012345
2020-04-28T16:45:00,6265.4567890123456
2020-04-28T17:00:00,6447.8901234567890
2020-04-28T17:15:00,6138.9012345678901
2020-04-28T17:30:00,6001.2345678901234
2020-04-28T17:45:00,6274.3456789012345
2020-04-28T18:00:00,6456.7890123456789
2020-04-28T18:15:00,6147.8901234567890
2020-04-28T18:30:00,5990.1234567890123
2020-04-28T18:45:00,6283.2345678901234
2020-04-28T19:00:00,6465.6789012345678
2020-04-28T19:15:00,6156.7890123456789
2020-04-28T19:30:00,5979.0123456789012
2020-04-28T19:45:00,6292.1234567890123
2020-04-28T20:00:00,6474.5678901234567
2020-04-28T20:15:00,6165.6789012345678
2020-04-28T20:30:00,5967.9012345678901
2020-04-28T20:45:00,6301.0123456789012
2020-04-28T21:00:00,6483.4567890123456
2020-04-28T21:15:00,6174.5678901234567
2020-04-28T21:30:00,5956.7890123456789
2020-04-28T21:45:00,6309.9012345678901
2020-04-28T22:00:00,6492.3456789012345
2020-04-28T22:15:00,6183.4567890123456
2020-04-28T22:30:00,5945.6789012345678
2020-04-28T22:45:00,6318.7901234567890
2020-04-28T23:00:00,6501.2345678901234
2020-04-28T23:15:00,6192.3456789012345
2020-04-28T23:30:00,5934.5678901234567
2020-04-28T23:45:00,6327.6789012345678
Loading