Psse parser consolidation#21
Open
m-bossart wants to merge 13 commits into
Open
Conversation
Resolve the raw file's REV up front to select v30's section order and dtypes, make default population version-aware, and widen the psse.jl converter's source_version branches to accept "30". Also fixes a v30 section-ordering bug, injects a few defaults for columns v30 omits, and handles v30 records that lack a NAME field or reference buses outside the parsed system.
v30 bus records carry fixed-shunt admittance directly (GL/BL), which the PowerModels converter never read. Also fix the v30 MULTI-TERMINAL DC NDCLN dtype, which inherited a MET field not present in v29/v30 raw files, blocking REV-less v29 files from resolving to v30 tables.
Replace the comma-only tokenizer with a quote-aware scanner that treats a comma with optional surrounding whitespace, or a run of whitespace, as the field separator. Also raise a DataFormatError with the real message on a parse failure instead of discarding it.
There was a problem hiding this comment.
Pull request overview
This PR consolidates PSSE parsing onto the PowerModelsData-based path by removing the legacy PowerFlowData-based wrapper, while extending the PTI/PSSE parser to properly handle v29/v30 layout and fields.
Changes:
- Removed the PowerFlowDataNetwork parsing pathway (code, dependency, and docs) to enforce a single parsing path.
- Added PTI v30 (and v29) section ordering, dtypes, defaults, and tokenizer improvements to support legacy RAW files in the PowerModelsData pipeline.
- Added v30-focused tests and fixtures, including validation for bus shunts (GL/BL) and clearer unsupported-version errors.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/test_parse_v30.jl | Adds test coverage for v30 parsing behavior, tokenizer rules, and error handling. |
| test/fixtures/v31_header.raw | Adds an unsupported-version fixture to validate error behavior. |
| test/fixtures/v30_bus_shunt.raw | Adds a fixture to validate v30 bus GL/BL-to-shunt handling. |
| src/PowerFlowFileParser.jl | Removes PowerFlowData integration and the PowerFlowDataNetwork export/include. |
| src/powerflowdata_data.jl | Deletes the PowerFlowDataNetwork wrapper type and constructor. |
| src/pm_io/pti.jl | Adds v30 section/dtype/default handling and improves PTI parsing/tokenization + version resolution. |
| src/pm_io/psse.jl | Extends PowerModels conversion logic to handle v30 field differences and bus-embedded shunts. |
| Project.toml | Removes the PowerFlowData dependency and bumps package version. |
| docs/src/tutorials/quickstart.md | Removes the “Alternative Parser” section for PowerFlowDataNetwork. |
| docs/src/explanation/data_structs.md | Removes PowerFlowDataNetwork documentation. |
| docs/src/explanation/arch_design.md | Updates architecture narrative to reflect the single PowerModels dictionary pathway. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
1947
to
1951
| line = strip(line_comment[1]) | ||
| comment = length(line_comment) > 1 ? strip(line_comment[2]) : "" | ||
|
|
||
| elements = split(line, _split_string) | ||
| elements = _split_fields(line) | ||
|
|
hannahchubin
left a comment
hannahchubin
left a comment
Collaborator
There was a problem hiding this comment.
Previously, I ran into trouble when I used the UUID "bed98974-b02e-5e2f-9ee0-a103f5c450dd" for PowerFlowFileParser, on my branch I'm using the UUID I suggested
| Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595" | ||
| InfrastructureSystems = "2cd47ed4-ca9b-11e9-27f2-ab636a7671f1" | ||
| Logging = "56ddb016-857b-54e1-b83d-db4d58db5568" | ||
| PowerFlowFileParser = "bed98974-b02e-5e2f-9ee0-a103f5c450dd" |
Collaborator
There was a problem hiding this comment.
Suggested change
| PowerFlowFileParser = "bed98974-b02e-5e2f-9ee0-a103f5c450dd" | |
| PowerFlowFileParser = "a1ee5d7c-0b6f-11f1-9fcb-79237b21bc78" |
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #20