Skip to content

Sink for BigQuery #86

@Ifechukwudaniel

Description

@Ifechukwudaniel

This issue tracks the implementation of a Sink that writes data to Google BigQuery.

The goal is to allow the SDK to persist decoded on-chain data directly into BigQuery tables for analysis, indexing, and integration with other Google Cloud tools.

✅ Tasks

  • Implement Sink interface for BigQuery
  • Support dynamic schema generation based on table.Columns
  • Support Flush() that inserts rows to BigQuery using streaming inserts
  • Handle dataset/table creation if they don’t exist
  • Ensure batching is efficient and avoids quota limits
  • Add configuration options:
    • projectID
    • datasetID
    • tablePrefix or tableName
  • Add authentication support using Application Default Credentials or service account
  • Write unit tests for the sink
  • Write integration test using test project or emulator (if available)
  • Add usage example in documentation

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions