Skip to content

Add a one-pass --recurse for archives inside archives - #61

Open
davidnewhall wants to merge 1 commit into
mainfrom
feat/recurse
Open

davidnewhall wants to merge 1 commit into
mainfrom
feat/recurse

Conversation

@davidnewhall

Copy link
Copy Markdown
Contributor

Summary

  • -r / --recurse (job key recurse) extracts archives that were just written. It is one extra pass and does not repeat. A folder you pass is still searched, including its sub folders, without this flag.
  • The second pass uses the file list the extract returned, not a scan of the output folder. Tar member names are joined onto the output folder. Name rules match a folder search, including rar parts and the job's include and exclude lists. Nested archives are extracted into the folder they are in and are not deleted. A failure is logged and the job continues.
  • The output path is made absolute before extracting, so -S -o out works with the xtractr version xt pins. With both --recurse and --squash-root, a top-level name that was already in the output folder is skipped. --max-depth 1 searches the path and one level of sub folders.

Test plan

  • go test -race ./...
  • golangci-lint run for darwin, linux, freebsd, and windows
  • CI

Made with Cursor

Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A Windows-incompatible permission assertion will fail CI, while recursive directory walks can process skipped or pre-existing archives.

3 open issues
What changed in this PR

Adds opt-in, one-pass extraction of nested archives to xt, separate from its existing recursive folder search.

Changes:

  • Adds -r / --recurse and the recurse job setting, with nested extraction totals and failure logging.
  • Resolves output paths absolutely and handles nested archive filtering and squash behavior.
  • Adds recursion tests and clarifies folder-search depth and nested extraction in the help and manual.
File Description
pkg/​xt/​xt.go Integrates the second pass, absolute output paths, and extraction totals.
pkg/​xt/​xt_test.go Shares a ZIP suffix constant across tests.
pkg/​xt/​recurse.go Implements nested archive selection and extraction.
pkg/​xt/​recurse_test.go Tests recursion limits, filtering, failures, squash behavior, and job parsing.
pkg/​xt/​job.go Adds the serialized recursion setting and includes it in job summaries.
MANUAL.md Documents recursion and clarifies folder-search depth.
main.go Registers the recursion flag and updates command-line help.

Review effort: Balanced

Comment thread pkg/xt/recurse_test.go
}

info, err := os.Stat(filepath.Join(out, "hello.txt"))
if err != nil || info.Mode().Perm() != 0o600 {
Comment thread pkg/xt/recurse.go
}

if info.IsDir() {
return j.archivesUnder(path, exclude, siblings, seen)
Comment thread pkg/xt/recurse.go
Comment on lines +151 to +153
if info.IsDir() {
return j.archivesUnder(path, exclude, siblings, seen)
}
Comment thread pkg/xt/recurse_test.go
}

info, err := os.Stat(filepath.Join(out, "hello.txt"))
if err != nil || info.Mode().Perm() != 0o600 {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows gotest is red on this commit, and this is the spot: Windows doesn't honor POSIX permission bits, so Perm() can't be 0o600 there. The xtractr tests guard exactly this (t.Skip("POSIX directory permission bits") on windows in files_internal_test.go). Wrap the assertion in if runtime.GOOS != "windows" {} — plus the runtime import — and the suite goes green on all three OSes.

@qwen-pr-bot qwen-pr-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full diff at head 8259253 over base adfd916: the -r/--recurse flag, Job.Recurse (struct tags, job-file parsing, String()), the one-pass nested extractor in recurse.go, the MANUAL.md rewrites, and the new test suite. Everything works as documented except one test that can't pass on Windows, which CI confirms.

What I ran at head: go vet, gofmt -l, go test -race ./... (all green), the same suite at base (green), cross-compile for windows/darwin/freebsd, and golangci-lint v2.15.0 — the version CI pins — against linux and windows GOOS (0 issues both). Then built the binary and exercised it end to end: zip-in-zip with and without --recurse, three-level nesting (third level stays packed), a tar member holding a nested zip, and the case that made me read the library hardest — squash with ./-prefixed tar members into a folder containing an unrelated pre-existing zip. The guard logs "Skipping pre-existing output entry" for the unrelated zip and leaves it packed, while the freshly written nested archive still extracts. That matches what I read in xtractr v0.9.0's squash code (identical in v0.8.0): the ./ roots collapse to the output folder itself, the aside-and-move-back returns every name in the folder, and preexistingNames filters exactly the pollution. The design of re-deriving written archives from the returned file list instead of the archiveList third value is right too — that value is only populated by RAR/CUE/7z extractors, so relying on it would miss zip-in-zip entirely. Continue-after-nested-failure, no-deletion of failed nested archives, mode/password/squash propagation, and the TOML job-file path all behaved correctly.

One blocking item (commented inline): the mode assertion in TestRecurseZipInZip. Public CI on this commit shows gotest failing on windows while macos passed and ubuntu was cancelled, and this PR is the first place in the suite that asserts on Perm() — the base has no such checks. Fix it with the standard GOOS guard and this is in good shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants