Skip to content

OpenMP support of tensor processing. - #565

Merged
highperformancecoder merged 7 commits into
masterfrom
feature-enable-openmp
Oct 6, 2025
Merged

highperformancecoder merged 7 commits into
masterfrom
feature-enable-openmp

Conversation

@highperformancecoder

@highperformancecoder highperformancecoder commented Sep 30, 2025 •

Copy link
Copy Markdown
Owner

This change is Reviewable

Summary by CodeRabbit

  • New Features

    • Optional OpenMP support for parallel execution when enabled.
    • Faster tensor evaluations, numerical integration, and ledger evaluations on larger datasets with automatic fallback to sequential mode.
    • Improved thread safety during parallel updates to prevent race conditions.
  • Chores

    • Build system updated to accept an OPENMP flag and link appropriate libraries.
  • Tests

    • Test build updated to support OpenMP configuration.
    • Removed obsolete test coverage integration.

@coderabbitai

coderabbitai Bot commented Sep 30, 2025 •

Copy link
Copy Markdown
Contributor

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Adds conditional OpenMP support in build scripts and parallelizes selected compute paths in tensor ops, Runge–Kutta evaluation, and Godley evaluation. Introduces atomic/atomic_ref guards, conditional thresholds (>20) for parallel loops, error aggregation in parallel blocks, and a redoCalc flag enabling conditional RavelTensor recomputation.

Changes

Cohort / File(s) Summary
Build: OpenMP flags
Makefile, test/Makefile
Adds conditional OPENMP blocks to append -fopenmp to compiler and linker flags; removes an AEGIS-related tcl-cov block in tests.
Tensor ops parallelization & control
engine/minskyTensorOps.cc
Adds OpenMP atomics to GeneralTensorOp specializations (all, runningSum, runningProduct); parallelizes TensorEval::eval copy for rhsSize > 20 with shared error capture; enhances RavelTensor to conditionally recompute via redoCalc loop; adds public member bool redoCalc=false.
Runge–Kutta selective parallelism
engine/rungeKutta.cc
Adds OpenMP-gated parallel init of stock vars and tensor integration for sizes > 20; retains sequential fallback; includes commented prospective parallel loop for equation eval; preserves error propagation.
Godley evaluation parallelization
engine/evalGodley.cc
Parallelizes init and accumulation loops with size thresholds; uses atomic_ref for thread-safe sv updates under OpenMP; keeps non-OpenMP path unchanged.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Th as Threads (_OPENMP)
  participant TE as TensorEval::eval
  participant RHS as rhs tensor
  participant RES as result tensor
  note over TE,RHS: Conditional parallel copy if rhsSize > 20
  alt _OPENMP and rhsSize > 20
    Th->>RES: for i in [0..n): RES[i] = RHS[i]
    Note over Th,RES: Capture first thrown error into shared threadErrMsg
    Th-->>TE: if threadErrMsg set -> throw runtime_error
  else
    TE->>RES: Sequential copy loop
  end
Loading
sequenceDiagram
  autonumber
  participant RT as RavelTensor
  participant Arg as Underlying arg
  note over RT,Arg: Conditional recomputation when Arg timestamp newer
  alt redoCalc true (single thread section)
    RT->>RT: Recompute hypercubes and chain
    RT->>RT: Reset redoCalc=false
  else
    RT->>RT: Use existing chain
  end
Loading
sequenceDiagram
  autonumber
  actor Th as Threads (_OPENMP)
  participant EG as EvalGodley::eval
  participant SV as sv vector
  participant IDX as initIdx / sidx
  Note over EG,SV: Initialize SV[initIdx]=0 (size threshold)
  alt _OPENMP and |initIdx| > 20
    Th->>SV: parallel for i in initIdx: SV[i]=0
  else
    EG->>SV: sequential init
  end
  Note over EG,SV: Accumulate over sidx into SV with atomic
  alt _OPENMP and |sidx| > 20
    Th->>SV: parallel for k in sidx: atomic_add(SV[...], ...)
  else
    EG->>SV: sequential accumulation
  end
Loading
sequenceDiagram
  autonumber
  actor Th as Threads (_OPENMP)
  participant RK as RungeKutta::evalEquations
  participant ST as stock vars
  participant TNS as tensor inputs
  Note over RK,ST: Initialize stocks (thresholded)
  alt _OPENMP and |ST| > 20
    Th->>ST: parallel init
  else
    RK->>ST: sequential init
  end
  Note over RK,TNS: Element-wise tensor integration (thresholded)
  alt _OPENMP and |TNS| > 20
    Th->>TNS: parallel integrate elements
  else
    RK->>TNS: sequential integration
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45–60 minutes

Poem

I thump my paws at threads that race,
With atomics set, they keep their place.
Ravel rewinds, then runs anew,
Godley sums in parallel too.
Makefiles whisper: “-fopenmp!”
I hop through loops—fast, carefree. 🐇⚙️

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title accurately and concisely reflects the main purpose of the pull request, which is adding OpenMP support for tensor processing across the codebase, without including extraneous details or vague terms.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature-enable-openmp

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6db0598 and fcf09c4.

📒 Files selected for processing (1)
  • Makefile (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
🔇 Additional comments (1)
Makefile (1)

218-221: OpenMP flags are now wired correctly.

Appending -fopenmp to both FLAGS and LIBS ensures the option propagates to every compile and link step once OPENMP is defined. 👍


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 864d192 and 02515a3.

📒 Files selected for processing (4)
  • Makefile (2 hunks)
  • RavelCAPI (1 hunks)
  • engine/minskyTensorOps.cc (2 hunks)
  • engine/rungeKutta.cc (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
engine/minskyTensorOps.cc (1)
engine/variableValue.cc (2)
  • i (95-99)
  • i (95-95)
engine/rungeKutta.cc (2)
model/group.cc (1)
  • flow (58-58)
schema/schema3.cc (1)
  • flow (866-866)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
🔇 Additional comments (2)
Makefile (1)

31-31: OpenMP flag never reaches the compile commands.

Line 31 now forwards OPENMP=$(OPENMP) to sub-makes, but this Makefile itself never consumes the variable. If I build with make OPENMP=-fopenmp, the compilation lines still omit -fopenmp, so _OPENMP remains undefined and the new parallel code paths stay disabled (only the link step sees -fopenmp via the clang-specific block on Line 216). Please append $(OPENMP) to the compile and link flags—e.g. add an ifdef OPENMP section that does FLAGS += $(OPENMP) and reuses the same variable for LIBS—so the setting actually enables OpenMP in this TU. request_verification

engine/rungeKutta.cc (1)

256-260: OpenMP parallel loop breaks exception propagation.

Line 258 wraps equations[i]->eval(...) in an OpenMP parallel region. Upstream (RungeKutta::RKfunction, Lines 50‑63) explicitly relies on evalEquations throwing so it can surface solver errors via threadErrMsg. Under OpenMP, any exception escaping a parallel region triggers std::terminate, so scenarios that used to report “Invalid arithmetic operation detected”, “integral not wired”, etc., will now hard-abort the process. Please keep this loop sequential or add per-iteration try/catch that records the first failure and aborts the parallel work without letting the exception cross the region (e.g. with an atomic flag + early exit). Until that’s in place this change is unsafe. raise_critical_issue

Comment thread engine/minskyTensorOps.cc Outdated
Comment on lines +1523 to +1534
auto chainBack=chain.back(); // stash a reference to the current chain, in case it gets updated below
v=(*chainBack)[i];
if (m_timestamp<chain.back()->timestamp())
{ // update hypercube if argument has changed
const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
chain=ravel::createRavelChain(ravel.getState(), arg);
m_timestamp=Timestamp::clock::now();
}
#ifdef _OPENMP
#pragma omp critical(RavelTensor_updateChain)
if (m_timestamp<chain.back()->timestamp()) // test again once lock has been obtained
#endif
{ // update hypercube if argument has changed
const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
chain=ravel::createRavelChain(ravel.getState(), arg);
m_timestamp=Timestamp::clock::now();
}

@coderabbitai coderabbitai Bot Sep 30, 2025 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Synchronize all reads of chain/m_timestamp

We still read chain.back() and m_timestamp outside the RavelTensor_updateChain critical section while the write happens inside it. That’s a data race: another thread can be in the critical block assigning a new chain vector (or updating m_timestamp) while this thread reads, yielding undefined behaviour. Pull the chain access and timestamp check inside the same critical section (or protect them with atomics) so every read and write is synchronized.

Apply this diff to guard all accesses consistently:

-        {
-          auto chainBack=chain.back(); // stash a reference to the current chain, in case it gets updated below
-          v=(*chainBack)[i];
-          if (m_timestamp<chain.back()->timestamp())
-#ifdef _OPENMP 
-#pragma omp critical(RavelTensor_updateChain)
-            if (m_timestamp<chain.back()->timestamp()) // test again once lock has been obtained
-#endif
-              { // update hypercube if argument has changed
-                const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
-                chain=ravel::createRavelChain(ravel.getState(), arg);
-                m_timestamp=Timestamp::clock::now();
-              }
-        }
+        {
+          TensorPtr chainBack;
+#ifdef _OPENMP
+#pragma omp critical(RavelTensor_updateChain)
+#endif
+          {
+            if (m_timestamp<chain.back()->timestamp())
+              {
+                const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
+                chain=ravel::createRavelChain(ravel.getState(), arg);
+                m_timestamp=chain.back()->timestamp();
+              }
+            chainBack=chain.back();
+          }
+          v=(*chainBack)[i];
+        }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
auto chainBack=chain.back(); // stash a reference to the current chain, in case it gets updated below
v=(*chainBack)[i];
if (m_timestamp<chain.back()->timestamp())
{ // update hypercube if argument has changed
const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
chain=ravel::createRavelChain(ravel.getState(), arg);
m_timestamp=Timestamp::clock::now();
}
#ifdef _OPENMP
#pragma omp critical(RavelTensor_updateChain)
if (m_timestamp<chain.back()->timestamp()) // test again once lock has been obtained
#endif
{ // update hypercube if argument has changed
const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
chain=ravel::createRavelChain(ravel.getState(), arg);
m_timestamp=Timestamp::clock::now();
}
{
TensorPtr chainBack;
#ifdef _OPENMP
#pragma omp critical(RavelTensor_updateChain)
#endif
{
// all shared reads/writes now inside the critical section
if (m_timestamp < chain.back()->timestamp())
{
const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
chain = ravel::createRavelChain(ravel.getState(), arg);
m_timestamp = chain.back()->timestamp();
}
chainBack = chain.back();
}
v = (*chainBack)[i];
}
🤖 Prompt for AI Agents
In engine/minskyTensorOps.cc around lines 1523 to 1534, reads of chain and
m_timestamp occur outside the RavelTensor_updateChain critical section while
writes occur inside, causing a data race; move the reads (chain.back(), v
assignment and the m_timestamp check) into the same omp
critical(RavelTensor_updateChain) block (or make chain/m_timestamp
atomic/lock-protected) so that the timestamp comparison and any use of chain
happen only under the same lock used for updates, then re-check condition inside
the lock and perform populateHypercube/createRavelChain/update of m_timestamp
while holding the lock.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your suggestion serialises the loop. As additional context, chain is a vector of shared_ptrs. Reading chain.back() into chainBack is atomic, and all that happens in that potentially the value of an old chain is returned, rather than the updated chain.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@highperformancecoder, you're absolutely right that the suggested fix would serialize all calls to operator[], which defeats the purpose of OpenMP parallelization. My apologies for that oversight.

However, there is still a subtle race condition. While copying a shared_ptr is thread-safe, reading from a std::vector while another thread reassigns that vector is undefined behavior according to the C++ standard. The line chain=ravel::createRavelChain(...) inside the critical section replaces the entire vector, which can corrupt reads from threads outside the critical section calling chain.back().

Better alternatives that preserve parallelism:

  1. Accept stale reads with proper barriers (if old chain values are acceptable):
auto chainBack = chain.back(); // relaxed read
v = (*chainBack)[i];
// Then do the timestamp check & update as you have it

Document that concurrent reads may use stale chain data, which seems acceptable based on your comment.

  1. Use std::atomic<Timestamp> for the timestamp and keep chain updates inside the critical section as-is. This ensures the timestamp comparison is race-free without serializing the data access.

  2. Reader-writer lock (e.g., std::shared_mutex) to allow concurrent reads while serializing updates only.

Would option 1 (accepting occasional stale reads with the current update mechanism) work for your use case? If the old chain returning correct-enough results is acceptable, you might just need to add a comment documenting this behavior.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm - maybe a reader-writer lock would be a better approach on dereferencing chain.back().

The problem is the calculation should be redone if ever the chain is rebuilt. In the sequential case, the chain is only ever rebuilt if the first iteration causes a timestamp update.

Possibly I need to do a barrier, and check for chain update, and redo the calculation if there's an update...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@highperformancecoder, you've identified the core issue precisely. Let me outline the three main approaches:

1. Reader-Writer Lock (Recommended)

This is the cleanest solution for preserving correctness while allowing parallel reads:

double operator[](size_t i) const override {
  double v=0;
  if (!chain.empty())
    {
      std::shared_lock<std::shared_mutex> readLock(chainMutex);
      auto chainBack=chain.back();
      auto needsUpdate = m_timestamp < chainBack->timestamp();
      readLock.unlock();
      
      if (needsUpdate)
        {
          std::unique_lock<std::shared_mutex> writeLock(chainMutex);
          // Double-check after acquiring exclusive lock
          if (m_timestamp < chain.back()->timestamp())
            {
              const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
              chain=ravel::createRavelChain(ravel.getState(), arg);
              m_timestamp=Timestamp::clock::now();
            }
          chainBack=chain.back();
        }
      v=(*chainBack)[i];
    }
  return v;
}

Pros: Clean semantics, all threads see consistent chain state
Cons: Requires C++17's std::shared_mutex

2. Barrier + Recompute Pattern

Check if chain changed after parallel region completes, redo if necessary:

In TensorEval::eval, wrap the parallel loop:

bool chainChanged;
do {
  chainChanged = false;
  auto initialTimestamp = rhs_as_ravel_tensor->captureTimestamp();
  
#pragma omp parallel for
  for (size_t i=0; i<rhsSize; ++i) {
    auto v=(*rhs)[i];
    result[i]=v;
  }
  
  if (rhs_as_ravel_tensor->timestampChanged(initialTimestamp))
    chainChanged = true;
} while(chainChanged);

Pros: Ensures all results come from same chain version
Cons: Complex, potential for wasted work if chain updates frequently

3. Single-Threaded Pre-Check (Simplest)

Before the parallel loop in TensorEval::eval, force chain update in a single-threaded context:

// Ensure chain is current before parallel execution
if (auto rt = dynamic_cast<RavelTensor*>(rhs.get()))
  rt->ensureChainCurrent();

#pragma omp parallel for if(rhsSize>20)
for (size_t i=0; i<rhsSize; ++i) {
  ...
}

Pros: Simple, no mid-iteration updates
Cons: Small serialization point; chain could still update during iteration if timestamps drift

Given your performance requirements and the codebase patterns, I'd recommend Option 1 (reader-writer lock) as it provides clear correctness guarantees while keeping reads concurrent. The overhead of shared_lock is minimal when no writes occur.

@highperformancecoder

Copy link
Copy Markdown
Owner Author

Actionable comments posted: 1
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro
📥 Commits

📒 Files selected for processing (4)

🧰 Additional context used

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)

🔇 Additional comments (2)

Makefile (1)> 31-31: OpenMP flag never reaches the compile commands.

Line 31 now forwards OPENMP=$(OPENMP) to sub-makes, but this Makefile itself never consumes the variable. If I build with make OPENMP=-fopenmp, the compilation lines still omit -fopenmp, so _OPENMP remains undefined and the new parallel code paths stay disabled (only the link step sees -fopenmp via the clang-specific block on Line 216). Please append $(OPENMP) to the compile and link flags—e.g. add an ifdef OPENMP section that does FLAGS += $(OPENMP) and reuses the same variable for LIBS—so the setting actually enables OpenMP in this TU. request_verification

Thanks - that was left in from an earlier experiment. The dependent libs no longer depend on OpenMP, so that flag can be removed.

engine/rungeKutta.cc (1)> 256-260: OpenMP parallel loop breaks exception propagation.

Line 258 wraps equations[i]->eval(...) in an OpenMP parallel region. Upstream (RungeKutta::RKfunction, Lines 50‑63) explicitly relies on evalEquations throwing so it can surface solver errors via threadErrMsg. Under OpenMP, any exception escaping a parallel region triggers std::terminate, so scenarios that used to report “Invalid arithmetic operation detected”, “integral not wired”, etc., will now hard-abort the process. Please keep this loop sequential or add per-iteration try/catch that records the first failure and aborts the parallel work without letting the exception cross the region (e.g. with an atomic flag + early exit). Until that’s in place this change is unsafe. raise_critical_issue

This is very important. Thanks for pointing that out.

Copilot AI review requested due to automatic review settings October 5, 2025 09:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds conditional OpenMP-based parallelism to heavy evaluation paths (tensor ops, Godley table accumulation, and integration loops) and propagates OpenMP flags in the build to improve multi-core performance on large models.

  • Introduces OpenMP pragmas for selected loops with size-based thresholds.
  • Updates build/test Makefiles to add OpenMP flags; bumps RavelCAPI submodule.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
engine/rungeKutta.cc Adds optional OpenMP loop parallelism and error aggregation for equation evaluation, zeroing, and element-wise integration.
engine/minskyTensorOps.cc Adds OpenMP atomics in tensor reductions/scans; modifies Ravel::operator[] logic for recalculation.
engine/evalGodley.cc Parallelizes initialization and accumulation; attempts to use atomic_ref for concurrent accumulation.
test/Makefile Adds OPENMP gating for -fopenmp in FLAGS for tests.
Makefile Adds -fopenmp to LIBS when using clang++.
RavelCAPI Updates submodule pointer.
Comments suppressed due to low confidence (1)

engine/rungeKutta.cc:1

  • The comment likely intended 'rhs is independent of result' (or of the loop index). Please clarify to avoid confusion.
/*

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread engine/evalGodley.cc
Comment on lines +101 to +109
#ifdef _OPENMP
size_t sidxSize=sidx.size();
#pragma omp parallel for if(initIdxSize>20)
for (size_t i=0; i<sidxSize; ++i)
atomic_ref(sv[sidx[i]]) += fv[fidx[i]] * m[i];
#else
for (size_t i=0; i<sidx.size(); ++i)
sv[sidx[i]] += fv[fidx[i]] * m[i];
#endif

Copilot AI Oct 5, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two issues: (1) The if-clause on the pragma uses initIdxSize instead of sidxSize, so the parallelization condition is incorrect. (2) atomic_ref is used as if it were a function; it should be std::atomic_ref with fetch_add (and requires ). Suggested fix: use if(sidxSize>20) and replace the body with std::atomic_ref ref(sv[sidx[i]]); ref.fetch_add(fv[fidx[i]] * m[i], std::memory_order_relaxed);

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

atomic_ref "used as afunction" is actually instantiating an unnamed temporary variable, and the += operator is overloaded to call fetch_add. So equivalent code, and idiomatic C++.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the incorrect #pragma - thanks.

Comment thread engine/minskyTensorOps.cc Outdated
Comment on lines +1520 to +1542
double v=0;
if (!chain.empty())
bool redoCalc=false;
do
{
v=(*chain.back())[i];
if (m_timestamp<chain.back()->timestamp())
{ // update hypercube if argument has changed
const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
chain=ravel::createRavelChain(ravel.getState(), arg);
m_timestamp=Timestamp::clock::now();
}
if (!chain.empty())
v=(*chain.back())[i];

Copilot AI Oct 5, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chain.back() is accessed unconditionally at line 1543, which is undefined behavior when chain is empty. Additionally, v is computed twice when chain is non-empty. Guard the early return when chain is empty and remove the duplicate evaluation, e.g., if (chain.empty()) return 0; auto chainBack = chain.back(); v = (*chainBack)[i];

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your're right - there does need to be a barrier before the single region, as well as after.

Comment thread engine/minskyTensorOps.cc Outdated
Comment on lines +1545 to +1552
#ifdef _OPENMP
#pragma omp single copyprivate(redoCalc)
#endif
{
redoCalc=m_timestamp<chain.back()->timestamp();
if (redoCalc)
{ // update hypercube if argument has changed
const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());

Copilot AI Oct 5, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omp single is only meaningful inside a parallel region; here it is outside any #pragma omp parallel and will be ignored or warn. If the intent is to synchronize an update across threads, either protect the mutation with a mutex/critical section or place this block inside an explicit parallel region with single (or critical) and adjust variable scoping accordingly.

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is called from within a parallel region mostly. If not, then I would expect the pragmas to be nops.

Comment thread engine/minskyTensorOps.cc Outdated
mutable vector<TensorPtr> chain;
TensorPtr arg;
mutable Timestamp m_timestamp;
bool redoCalc=false;

Copilot AI Oct 5, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Member redoCalc is unused and shadowed by the local redoCalc in operator[]. Remove the member or use it consistently to avoid confusion.

Suggested change
bool redoCalc=false;

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking that up.

Comment thread engine/rungeKutta.cc Outdated
Comment thread engine/rungeKutta.cc Outdated
{
equations[i]->eval(flow.data(), flow.size(), vars);
}
catch (std::exception& e)

Copilot AI Oct 5, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catch exceptions by const reference: catch (const std::exception& e) to avoid unintended modifications and allow binding to temporaries.

Suggested change
catch (std::exception& e)
catch (const std::exception& e)

Copilot uses AI. Check for mistakes.
Comment thread test/Makefile
endif

ifdef OPENMP
FLAGS+=-fopenmp

Copilot AI Oct 5, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some toolchains, linking requires -fopenmp as well. Consider also adding LIBS+=-fopenmp (or LDFLAGS) under the same OPENMP guard to avoid link errors.

Suggested change
FLAGS+=-fopenmp
FLAGS+=-fopenmp
LIBS+=-fopenmp

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
engine/minskyTensorOps.cc (1)

1536-1557: Chain updates still race with parallel readers

The new omp single block updates chain while the surrounding code keeps calling chain.back() outside any synchronization. One thread can still reassign chain inside the single region while another thread is executing chain.back() (or chain.empty()) in this same method, which is undefined behaviour on std::vector. This is the same race called out in the earlier review—writes are protected but reads are not.

Please guard both the read and the possible rebuild with the same critical/lock so every access is synchronized. For example:

-      bool redoCalc=false;
-      do
-        {
-          if (!chain.empty())
-            v=(*chain.back())[i];
-              
-          auto chainBack=chain.back(); // stash a reference to the current chain, in case it gets updated below
-          v=(*chainBack)[i];
-#ifdef _OPENMP
-#pragma omp single copyprivate(redoCalc)
-#endif
-          {
-            redoCalc=m_timestamp<chain.back()->timestamp();
-            if (redoCalc)
-              { // update hypercube if argument has changed
-                const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
-                chain=ravel::createRavelChain(ravel.getState(), arg);
-                m_timestamp=Timestamp::clock::now();
-              }
-          }
-        }
-      while (redoCalc);
+      bool redoCalc=false;
+      do
+        {
+#ifdef _OPENMP
+#pragma omp critical(RavelTensor_updateChain)
+#endif
+          {
+            if (chain.empty())
+              return v;
+            auto chainBack=chain.back();
+            redoCalc=m_timestamp<chainBack->timestamp();
+            if (redoCalc)
+              {
+                const_cast<Ravel&>(ravel).populateHypercube(arg->hypercube());
+                chain=ravel::createRavelChain(ravel.getState(), arg);
+                m_timestamp=Timestamp::clock::now();
+                chainBack=chain.back();
+              }
+            v=(*chainBack)[i];
+          }
+        }
+      while (redoCalc);

Any equivalent scheme that synchronizes every read with the write (e.g., std::shared_mutex) would work.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 02515a3 and 5a77fe1.

📒 Files selected for processing (5)
  • Makefile (1 hunks)
  • engine/evalGodley.cc (1 hunks)
  • engine/minskyTensorOps.cc (4 hunks)
  • engine/rungeKutta.cc (2 hunks)
  • test/Makefile (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • Makefile
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04

Comment thread engine/evalGodley.cc
Comment on lines +102 to +105
size_t sidxSize=sidx.size();
#pragma omp parallel for if(initIdxSize>20)
for (size_t i=0; i<sidxSize; ++i)
atomic_ref(sv[sidx[i]]) += fv[fidx[i]] * m[i];

@coderabbitai coderabbitai Bot Oct 5, 2025 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Fix the atomic_ref instantiation

atomic_ref is a class template; atomic_ref(...) += ...; is ill-formed, so the OpenMP build will not compile. Please instantiate a std::atomic_ref<double> (and include <atomic> with these OpenMP additions).

-#pragma omp parallel for if(initIdxSize>20)
-    for (size_t i=0; i<sidxSize; ++i)
-      atomic_ref(sv[sidx[i]]) += fv[fidx[i]] * m[i];
+#pragma omp parallel for if(initIdxSize>20)
+    for (size_t i=0; i<sidxSize; ++i)
+      {
+        std::atomic_ref<double> cell{sv[sidx[i]]};
+        cell += fv[fidx[i]] * m[i];
+      }

Remember to add #include <atomic> with the other includes.

🤖 Prompt for AI Agents
In engine/evalGodley.cc around lines 102 to 105, replace the ill-formed use of
atomic_ref(...) += ... with a properly instantiated std::atomic_ref<double>; add
#include <atomic> to the includes. Inside the OpenMP loop create a local
std::atomic_ref<double> (e.g. std::atomic_ref<double> ar(sv[sidx[i]]);) and
perform the addition via that atomic_ref (ar += fv[fidx[i]] * m[i]; or
ar.fetch_add(...)); this fixes the template instantiation and ensures
thread-safe atomic updates.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does compile - it makes use of a new C++17 feature called class template argument deduction

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@highperformancecoder, you're absolutely right! I apologize for the incorrect suggestion. The atomic_ref(sv[sidx[i]]) syntax is indeed valid with class template argument deduction (CTAD), which allows the compiler to deduce the template parameter <double> from the constructor argument.

Just a minor clarification: std::atomic_ref was introduced in C++20 rather than C++17, though CTAD itself is a C++17 feature.

Since you've confirmed it compiles successfully, the code is fine as-is. Just ensure that <atomic> is included (either directly or transitively through one of the headers) for atomic_ref to be available.

Thank you for the correction!

Comment thread engine/evalGodley.cc Outdated
highperformancecoder and others added 3 commits October 6, 2025 09:19
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between db3757e and 6db0598.

📒 Files selected for processing (5)
  • Makefile (1 hunks)
  • engine/evalGodley.cc (1 hunks)
  • engine/minskyTensorOps.cc (3 hunks)
  • engine/rungeKutta.cc (2 hunks)
  • test/Makefile (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • Makefile
🧰 Additional context used
🧬 Code graph analysis (1)
engine/minskyTensorOps.cc (2)
model/operation.cc (4)
  • x (48-48)
  • x (48-48)
  • x (54-54)
  • x (54-54)
engine/variableValue.cc (2)
  • i (95-99)
  • i (95-95)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
  • GitHub Check: Analyze (javascript, 22.04)
  • GitHub Check: ubuntu-20.04
  • GitHub Check: ubuntu-20.04
🔇 Additional comments (6)
engine/minskyTensorOps.cc (5)

350-355: LGTM: Correct atomic operation for thread safety.

The #pragma omp atomic correctly protects the x*=(y>0.5) operation from data races when multiple threads update the same accumulator in the reduction operation.


361-366: LGTM: Correct atomic operation for thread safety.

The #pragma omp atomic correctly protects the x+=y operation from data races during the running sum computation.


372-377: LGTM: Correct atomic operation for thread safety.

The #pragma omp atomic correctly protects the x*=y operation from data races during the running product computation.


1713-1734: LGTM: Correct exception handling pattern for OpenMP.

The parallel loop properly captures exceptions into errMsg using a critical section (lines 1728-1731) and rethrows after the parallel region completes (lines 1733-1734). This prevents exceptions from escaping the parallel region, which would otherwise cause std::terminate.


1536-1557: OpenMP pragmas execute only within the #pragma omp parallel for in TensorEval::eval; no additional synchronization is required.

engine/rungeKutta.cc (1)

298-305: Remove exception-wrapping suggestion for the OpenMP loop The loop body uses only operator[] and basic arithmetic (all effectively noexcept), so a try-catch is unnecessary.

Likely an incorrect or invalid review comment.

Comment thread test/Makefile
Comment on lines +29 to +32
ifdef OPENMP
FLAGS+=-fopenmp
LIBS+=-fopenmp
endif

@coderabbitai coderabbitai Bot Oct 5, 2025 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Don’t hard‑code -fopenmp; respect the caller’s flag.

ifdef OPENMP now unconditionally adds -fopenmp to both FLAGS and LIBS. That breaks toolchains that need a different switch (e.g., -qopenmp, -mp, clang’s -Xpreprocessor -fopenmp + -lomp) because they’ll still inherit this hard-coded GNU flag and fail to compile or link. Please append the user-provided $(OPENMP) value (and, if needed, a separate $(OPENMP_LIBS)) instead of forcing -fopenmp, so downstream make invocations can supply the appropriate options.

🤖 Prompt for AI Agents
In test/Makefile around lines 29 to 32, the Makefile currently hard-codes
-fopenmp into FLAGS and LIBS when OPENMP is defined; instead, respect
caller-provided flags by appending the variable values. Change the logic so that
if OPENMP is set you append $(OPENMP) to FLAGS (and optionally append
$(OPENMP_LIBS) to LIBS) rather than writing -fopenmp; ensure existing behavior
remains when callers export OPENMP or OPENMP_LIBS (so downstream toolchains can
pass their preferred compile/link switches).

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not how I intend to use it. The OPENMP flag is defined or not to enable OpenMP or not. We're currently supporting gcc and clang (or dpcpp, which is based on clang) only - gcc ignores -fopenmp on linking and clang requires it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@highperformancecoder, understood! Given that you're supporting gcc and clang/dpcpp exclusively, and -fopenmp works correctly for both (gcc ignoring it on link, clang requiring it), your boolean switch approach is perfectly reasonable. Thank you for clarifying the design intent.

@highperformancecoder
highperformancecoder merged commit 861e1c6 into master Oct 6, 2025
7 checks passed
@highperformancecoder
highperformancecoder deleted the feature-enable-openmp branch January 25, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants