All issues

Search results

Replace pydoop to read (WARC) data from HDFS

Status: Open.

#53 In commoncrawl/cc-pyspark;

· sebastian-nagel opened on Sep 19, 2025

Abstract process_record to Separate Content Extraction Step for Reusability and Testing

Status: Open.

#48 In commoncrawl/cc-pyspark;

· silentninja opened on Dec 30, 2024

Use simdjson to read WAT payloads

enhancement

Status: Open.

#41 In commoncrawl/cc-pyspark;

· sebastian-nagel opened on Mar 7, 2023

Test and update examples to work with ARC files of the 2008 - 2012 crawls

enhancement

Status: Open.

#20 In commoncrawl/cc-pyspark;

· sebastian-nagel opened on Oct 14, 2020