Improve pdfoxide parser with text preprocessing

[rager306]

Implemented 'preprocess_text' in 'pdfoxide_parser.py' to handle smashed columns (amounts, user IDs) caused by draw-order text extraction. This fixes parsing failures for pdfoxide on standard bank statements, improving benchmark success rate from 0% to 66.7%.

---

PR created automatically by Jules for task 7756069123368046916 started by @rager306

Summary by CodeRabbit

• Bug Fixes
  • Improved PDF text extraction reliability by better handling layout issues involving financial amounts, user identifiers, and date-time information during parsing.

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[coderabbitai]

Walkthrough

A new public preprocessing function is added to the pdfoxide_parser module to sanitize extracted PDF text by inserting newlines around key boundaries (amounts, user IDs, date-time markers). This preprocessing step is integrated into the PDF parsing flow to improve downstream text extraction robustness without modifying the parser's signature or return structure.

Changes

Cohort / File(s)	Summary
PDF Text Preprocessing pdfparser/pdfoxide_parser.py	Adds new preprocess_text(text: str) -> str function to sanitize and split PDF-extracted text. Moves re module import to module scope and integrates preprocessing into the parsing flow by applying it to each page's text before aggregation and downstream parsing.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: introducing text preprocessing to improve the pdfoxide parser, directly addressing the core objective of fixing parsing failures.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/pdfoxide-parsing-7756069123368046916

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@pdfparser/pdfoxide_parser.py`:
- Around line 113-116: The fallback empty-string is currently inside the comment
and not applied; change the assignment so doc.extract_text(page_num) is OR'd
with "" before the type-ignore comment (e.g., assign page_text =
doc.extract_text(page_num) or ""  # type: ignore[attr-defined]) so page_text is
never None prior to calling preprocess_text(page_text) and concatenating into
all_text; update the statement that sets page_text in pdfoxide_parser.py
accordingly.

🧹 Nitpick comments (2)

pdfparser/pdfoxide_parser.py (2)

39-40: Consider a single-pass approach for chained amounts.

Applying the same regex twice works but is fragile for longer chains (4+ amounts). A re.findall + rejoin or a more targeted pattern might be more robust.

That said, given the 66.7% benchmark improvement, the current approach is pragmatic for the known input patterns.

---

42-45: Potential false positives on non-amount decimals.

The pattern (\d[\d,]*\.\d{2}) will match any 2-decimal-place number, potentially splitting text like "Rate 2.50% applied" incorrectly. If this becomes an issue, consider anchoring to expected contexts or adding negative lookbehind for percentage signs.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Bug: or "" is inside the comment, not in the expression.

The fallback or "" appears after the # type: ignore comment, making it ineffective. If extract_text returns None, page_text will be None.

While preprocess_text handles None via if not text, this is likely unintentional and could mask issues. The intent seems to be a fallback empty string.

🐛 Proposed fix

-            page_text = doc.extract_text(page_num)  # type: ignore[attr-defined] or ""
+            page_text = doc.extract_text(page_num) or ""  # type: ignore[attr-defined]

🤖 Prompt for AI Agents

In `@pdfparser/pdfoxide_parser.py` around lines 113 - 116, The fallback
empty-string is currently inside the comment and not applied; change the
assignment so doc.extract_text(page_num) is OR'd with "" before the type-ignore
comment (e.g., assign page_text = doc.extract_text(page_num) or ""  # type:
ignore[attr-defined]) so page_text is never None prior to calling
preprocess_text(page_text) and concatenating into all_text; update the statement
that sets page_text in pdfoxide_parser.py accordingly.