While collecting min-tree from executors, the Orchestrator has a logic that copies and sorts processed task for returning result.
but it’s too time consuming.
|
let all_tree_results = join_all(all_tree_responses).await; |
|
|
|
// Aggregate results from all workers in order |
|
let mut ordered_tree_results = vec![None; self.entry_csvs.len()]; |
|
for result in all_tree_results { |
|
let (index, worker_results) = result.unwrap(); |
|
let start = index * entries_per_executor; |
|
for (i, res) in worker_results.iter().enumerate() { |
|
ordered_tree_results[start + i] = Some(res.clone()); |
|
} |
|
} |
We can improve this part use rayon
While collecting min-tree from executors, the Orchestrator has a logic that copies and sorts processed task for returning result.
but it’s too time consuming.
summa-aggregation/src/orchestrator/mod.rs
Lines 224 to 234 in 7dba825
We can improve this part use
rayon