Add is_valid_url row-level check - #1457
Open
ghanse wants to merge 4 commits into
Open
Conversation
Adds a row-level check that validates URL format against the RFC 3986 §4.3 absolute-URI grammar: scheme ":" hier-part with an optional "?" query and "#" fragment. A scheme is required, so relative references such as "/path" or "example.com" are rejected, and reserved characters must be percent-encoded to be accepted inside a path, query, or fragment. Follows the existing DQPattern approach used by is_valid_email and is_valid_ipv4_address, so the check itself is a thin _matches_pattern call. Any syntactically valid scheme is accepted, which keeps non-network URLs such as s3://, ftp://, mailto: and urn: valid alongside http:// and https://. Two consequences are documented in the docstring and the reference table: the check validates URL *syntax* and not safety, so script-bearing schemes (javascript:alert(1)) and inline data: payloads pass and it must not be used to sanitize untrusted input; and RFC 3986 permits an empty host, so file:///path passes. The pattern is anchored with \A...\z rather than ^...$ so that a value with a trailing newline is rejected under Java regex semantics, consistent with the fix in issue #1440. Every repetition in the pattern is over alternatives with disjoint first characters, so there is no catastrophic backtracking; this was verified against pathological multi-kilobyte inputs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1457 +/- ##
==========================================
- Coverage 92.51% 92.44% -0.08%
==========================================
Files 141 141
Lines 13577 13591 +14
Branches 151 151
==========================================
+ Hits 12561 12564 +3
- Misses 946 955 +9
- Partials 70 72 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
❌ 956/957 passed, 1 failed, 50 skipped, 8h21m15s total ❌ TestDataContractIntegration.test_generate_rules_with_text_processing: litellm.exceptions.RateLimitError: litellm.RateLimitError: DatabricksException - {"error_code":"REQUEST_LIMIT_EXCEEDED","message":"REQUEST_LIMIT_EXCEEDED: Exceeded workspace input tokens per minute rate limit for databricks-claude-sonnet-4-5. Work with your Databricks account team to request a higher FMAPI rate limit tier."} (58.035s)Running from acceptance #5655 |
Contributor
|
✅ 1/1 passed, 30m19s total Running from mcp #404 |
Contributor
|
✅ 135/135 passed, 1 skipped, 8h29m45s total Running from anomaly #1769 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Adds a new row-level check
is_valid_url(column)that validates URL format against the RFC 3986 4.3 absolute-URI grammar. A scheme is required, so relative references such as/pathorexample.comare rejected. Reserved characters must be percent-encoded.Linked issues
Tests
Documentation and Demos
This pull request and its description were co-written by Isaac.