PerfFlowAspect Reader + CPU/Mem Usage - #144
Closed
spencer-gre wants to merge 59 commits into
Closed
Conversation
- add node object to dataframe
- reorganize data files - restore ts and dur into frame
…a. Renamed and moved pfw test file to the correct folder
- add node object to dataframe
This commit follows remarks that llnl#144 will supersede llnl#101, and specifically prepares for the integration of both an array reader for PerfFlowAspect Array Trace Files (already present) and PerfFlowAspect Object Trace Files (to be added). Additionally removes .json files in the pytest data directory as no longer needed.
Missed two in the prior commit. Refer there for full explanation.
Contributor
Author
|
b1461b5 and faf48b2 address the following points of the roadmap:
|
This branch cleans up conflicts/redundancies introduced after object reader added. Specifically, JSON files were reintroduced, and references to the original perfflowaspect_reader was being used in the graphframe (and not perfflowaspect_array_reader)
Added test for us to s conversion
Contributor
Author
slabasan
reviewed
Jul 23, 2026
slabasan
reviewed
Jul 23, 2026
slabasan
reviewed
Jul 23, 2026
spencer-gre
marked this pull request as ready for review
July 24, 2026 20:18
This was referenced Jul 25, 2026
slabasan
added a commit
that referenced
this pull request
Jul 25, 2026
* perf_reader initial commit * graph updated * add front-level reader API, add PFA reader example - add node object to dataframe * graph updated * added pfw testfile with one iteration of laghos output * pfw file processor added * pfw to literal converter * sample json * pfw folder processor * finalized pfw to caliper reader * cleaning up code * update example for pfa reader * add perfflowaspect unit test - reorganize data files - restore ts and dur into frame * add AMS MPI pfa unit test (concat pfw files per rank), flake, black * restore kwargs in from_hdf * update comment * read cpu/mem usage * updates to pfa reader, prepare tests * forgot this test file * tests for cpu-mem * fix tests * Read All PFW Files The PerfFlowAspect Reader used to require that .pfw files (PerfFlowAspect Traces) conform to JSON standard. However, PerfFlowAspect does not generate files that are consistent with JSON. This commit modifies a PerfFlowAspect trace file that does not conform to JSON standard - by adding the closing brace and removing the trailing comma. * Cleanup/Prep for Object Reader This commit follows remarks that #144 will supersede #101, and specifically prepares for the integration of both an array reader for PerfFlowAspect Array Trace Files (already present) and PerfFlowAspect Object Trace Files (to be added). Additionally removes .json files in the pytest data directory as no longer needed. * Removing PFW JSON Files Missed two in the prior commit. Refer there for full explanation. * perf_reader initial commit * graph updated * graph updated * added pfw testfile with one iteration of laghos output * pfw file processor added * pfw to literal converter * sample json * pfw folder processor * finalized pfw to caliper reader * cleaning up code * add perfflowaspect unit test - reorganize data files - restore ts and dur into frame * add AMS MPI pfa unit test (concat pfw files per rank), flake, black * Add perfflowaspect object reader and update related tests and scripts * updated the test file. add extra fields for the time unit and matedata. Renamed and moved pfw test file to the correct folder * added us to s converter. fixed object reader tests * Cleanup after merge This branch cleans up conflicts/redundancies introduced after object reader added. Specifically, JSON files were reintroduced, and references to the original perfflowaspect_reader was being used in the graphframe (and not perfflowaspect_array_reader) * code format * pushed * nearly working single pfa reader * wrapping up pfa reader * remove changes to install * clean up and patch hatchet reader * update tests * restore hpctoolkit and timemory readers * fix license issues * flake and black format --------- Co-authored-by: r-yin <ryin2@bu.edu> Co-authored-by: Spencer Greene <ssgreene@wpi.edu> Co-authored-by: loudsun1997 <zhiwei.yang1997@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improves upon #101 adding ability to read in a PerfFlowAspect log with cpu and memory usage.