Skip to content

Enhance error handling in DelegatingMultiSinkOutputCommitter #1606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
psainics wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Enhance error handling in DelegatingMultiSinkOutputCommitter #1606

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 27 additions & 6 deletions src/main/java/io/cdap/plugin/gcp/bigquery/sink/DelegatingMultiSinkOutputCommitter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
import org.apache.hadoop.mapreduce.JobStatus;
import org.apache.hadoop.mapreduce.OutputCommitter;
import org.apache.hadoop.mapreduce.TaskAttemptContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.util.HashMap;
Expand All @@ -35,6 +37,7 @@
* Delegated instances are supplied along with a schema, which is used to configure the commit operation.
*/
public class DelegatingMultiSinkOutputCommitter extends OutputCommitter {
private static final Logger LOG = LoggerFactory.getLogger(DelegatingMultiSinkOutputCommitter.class);
private final Map<String, OutputCommitter> committerMap;
private final Map<String, Schema> schemaMap;
private final String projectName;
Expand Down Expand Up @@ -99,18 +102,28 @@ public boolean needsTaskCommit(TaskAttemptContext taskAttemptContext) throws IOE
@Override
public void commitTask(TaskAttemptContext taskAttemptContext) throws IOException {
for (String tableName : committerMap.keySet()) {
configureContext(taskAttemptContext, tableName);

committerMap.get(tableName).commitTask(taskAttemptContext);
try {
configureContext(taskAttemptContext, tableName);
committerMap.get(tableName).commitTask(taskAttemptContext);
} catch (IOException e) {
LOG.warn("BigQuery multi-sink table '{}' failed during task commit. Reason: {}",
tableName, getFailureReason(e), e);
throw e;
}
}
Comment on lines 104 to 113
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In a multi-sink scenario, failing fast on the first error prevents other sinks from attempting to commit their data, which can lead to partial and inconsistent job states.

It is better to attempt the commit for all sinks and collect any exceptions using addSuppressed, similar to the implementation in abortTask and abortJob. Additionally, since a commit failure is a fatal event for the task, LOG.error is more appropriate than LOG.warn.

    IOException ioe = null;
    for (String tableName : committerMap.keySet()) {
      try {
        configureContext(taskAttemptContext, tableName);
        committerMap.get(tableName).commitTask(taskAttemptContext);
      } catch (IOException e) {
        LOG.error("BigQuery multi-sink table '{}' failed during task commit. Reason: {}",
                  tableName, getFailureReason(e), e);
        if (ioe == null) {
          ioe = e;
        } else {
          ioe.addSuppressed(e);
        }
      }
    }
    if (ioe != null) {
      throw ioe;
    }

}

@Override
public void commitJob(JobContext jobContext) throws IOException {
for (String tableName : committerMap.keySet()) {
configureContext(jobContext, tableName);

committerMap.get(tableName).commitJob(jobContext);
try {
configureContext(jobContext, tableName);
committerMap.get(tableName).commitJob(jobContext);
} catch (IOException e) {
LOG.warn("BigQuery multi-sink table '{}' failed during job commit. Reason: {}",
tableName, getFailureReason(e), e);
throw e;
Copy link
Copy Markdown
Contributor

@itsankit-google itsankit-google Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logging and re-throwing an error is considered an anti-pattern.

Isn't this error propagated upwards and logged somewhere in pipeline run?

}
}
Comment on lines 118 to 127
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Similar to commitTask, commitJob should attempt to commit all sinks rather than failing fast. This ensures that as much data as possible is finalized and provides a complete picture of which sinks failed. Using LOG.error is also recommended here as job commit failures are critical.

    IOException ioe = null;
    for (String tableName : committerMap.keySet()) {
      try {
        configureContext(jobContext, tableName);
        committerMap.get(tableName).commitJob(jobContext);
      } catch (IOException e) {
        LOG.error("BigQuery multi-sink table '{}' failed during job commit. Reason: {}",
                  tableName, getFailureReason(e), e);
        if (ioe == null) {
          ioe = e;
        } else {
          ioe.addSuppressed(e);
        }
      }
    }
    if (ioe != null) {
      throw ioe;
    }

}

Expand Down Expand Up @@ -168,4 +181,12 @@ public void configureContext(JobContext context, String tableName) throws IOExce
gcsPath,
fields);
}

private String getFailureReason(IOException exception) {
Throwable rootCause = exception;
while (rootCause.getCause() != null) {
rootCause = rootCause.getCause();
}
return rootCause.getMessage() == null ? exception.getMessage() : rootCause.getMessage();
}
Comment on lines +185 to +191
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current implementation of getFailureReason might return null if both the root cause and the wrapper exception have no message. It also lacks a check for circular references in the exception chain.

A more robust approach is to fall back to rootCause.toString() when the message is null, which will at least provide the exception class name.

Suggested change
private String getFailureReason(IOException exception) {
Throwable rootCause = exception;
while (rootCause.getCause() != null) {
rootCause = rootCause.getCause();
}
return rootCause.getMessage() == null ? exception.getMessage() : rootCause.getMessage();
}
private String getFailureReason(IOException exception) {
Throwable rootCause = exception;
while (rootCause.getCause() != null && rootCause.getCause() != rootCause) {
rootCause = rootCause.getCause();
}
return rootCause.getMessage() != null ? rootCause.getMessage() : rootCause.toString();
}

}
Loading