.PHONY targets in
make(1).
-walk(1) delegates to an executable file called Walkfile within the same -directory as the target, to determine what dependencies the target has, and how -to execute it.
+walk(1) delegates to an executable file called Walkfile to determine +what dependencies the target has, and how to execute it. walk(1) searches for a +Walkfile starting from the target's directory and walking up the directory tree +until one is found. This allows a single Walkfile at the project root to handle +targets in any subdirectory.
The Walkfile determines how a target is executed, and what other targets it
depends on.
When walk(1) begins execution of a target, it attempts to find an executable
-file called Walkfile in the same directory as the target, and then executes
-it with the following positional arguments:
When walk(1) begins execution of a target, it searches for an executable file
+called Walkfile starting from the target's directory and walking up the
+directory tree. Once found, it executes the Walkfile with the following
+positional arguments:
$1deps or exec).$2hello.o).hello.o or
+subdir/hello.o if the Walkfile is in a parent directory).It's up to the Walkfile to determine what dependencies the target has, and
how to execute it.
If a Walkfile exits with code 200, walk(1) will try the next Walkfile up the
+directory tree. This allows a local Walkfile to handle specific targets while
+delegating unknown targets to a parent Walkfile:
case $target in
+ special) ;; # handle locally
+ *) exit 200 ;; # delegate to parent
+esac
+walk(1) has two phases:
diff --git a/man/walk.1 b/man/walk.1 index 5b27f8f..0ae9052 100644 --- a/man/walk.1 +++ b/man/walk.1 @@ -33,19 +33,29 @@ By default, the stdout/stderr output from the \fBWalkfile\fR is prefixed with th .SH "TARGETS" Targets can be used to represent a task, or a file that needs to be built\. They are synonymous with targets in make(1)\. In general, targets are relative paths to files that need to be built, like \fBsrc/hello\.o\fR\. When a target does not relate to an actual file on disk, it's synonymous with \fB\.PHONY\fR targets in make(1)\. .P -walk(1) delegates to an executable file called \fIWalkfile\fR within the same directory as the target, to determine what dependencies the target has, and how to execute it\. +walk(1) delegates to an executable file called \fIWalkfile\fR to determine what dependencies the target has, and how to execute it\. walk(1) searches for a Walkfile starting from the target's directory and walking up the directory tree until one is found\. This allows a single Walkfile at the project root to handle targets in any subdirectory\. .SH "WALKFILE" The \fBWalkfile\fR determines \fIhow\fR a target is executed, and what other targets it depends on\. .P -When walk(1) begins execution of a target, it attempts to find an executable file called \fBWalkfile\fR in the same directory as the target, and then executes it with the following positional arguments: +When walk(1) begins execution of a target, it searches for an executable file called \fBWalkfile\fR starting from the target's directory and walking up the directory tree\. Once found, it executes the Walkfile with the following positional arguments: .TP \fB$1\fR The \fIphase\fR (\fBdeps\fR or \fBexec\fR)\. .TP \fB$2\fR -The name of the target to build (e\.g\. \fBhello\.o\fR)\. +The target path relative to the Walkfile's directory (e\.g\. \fBhello\.o\fR or \fBsubdir/hello\.o\fR if the Walkfile is in a parent directory)\. .P It's up to the \fBWalkfile\fR to determine what dependencies the target has, and how to execute it\. +.P +If a Walkfile exits with code \fB200\fR, walk(1) will try the next Walkfile up the directory tree\. This allows a local Walkfile to handle specific targets while delegating unknown targets to a parent Walkfile: +.IP "" 4 +.nf +case $target in + special) ;; # handle locally + *) exit 200 ;; # delegate to parent +esac +.fi +.IP "" 0 .SH "PHASES" walk(1) has two phases: .TP diff --git a/man/walk.1.html b/man/walk.1.html index 05f0aac..d190725 100644 --- a/man/walk.1.html +++ b/man/walk.1.html @@ -143,29 +143,43 @@.PHONY targets in
make(1).
-walk(1) delegates to an executable file called Walkfile within the same -directory as the target, to determine what dependencies the target has, and how -to execute it.
+walk(1) delegates to an executable file called Walkfile to determine +what dependencies the target has, and how to execute it. walk(1) searches for a +Walkfile starting from the target's directory and walking up the directory tree +until one is found. This allows a single Walkfile at the project root to handle +targets in any subdirectory.
The Walkfile determines how a target is executed, and what other targets it
depends on.
When walk(1) begins execution of a target, it attempts to find an executable
-file called Walkfile in the same directory as the target, and then executes
-it with the following positional arguments:
When walk(1) begins execution of a target, it searches for an executable file
+called Walkfile starting from the target's directory and walking up the
+directory tree. Once found, it executes the Walkfile with the following
+positional arguments:
$1deps or exec).$2hello.o).hello.o or
+subdir/hello.o if the Walkfile is in a parent directory).It's up to the Walkfile to determine what dependencies the target has, and
how to execute it.
If a Walkfile exits with code 200, walk(1) will try the next Walkfile up the
+directory tree. This allows a local Walkfile to handle specific targets while
+delegating unknown targets to a parent Walkfile:
case $target in
+ special) ;; # handle locally
+ *) exit 200 ;; # delegate to parent
+esac
+walk(1) has two phases:
diff --git a/man/walk.1.md b/man/walk.1.md index b6d9f98..1e887bf 100644 --- a/man/walk.1.md +++ b/man/walk.1.md @@ -60,28 +60,41 @@ to files that need to be built, like `src/hello.o`. When a target does not relate to an actual file on disk, it's synonymous with `.PHONY` targets in make(1). -walk(1) delegates to an executable file called [Walkfile][WALKFILE] within the same -directory as the target, to determine what dependencies the target has, and how -to execute it. +walk(1) delegates to an executable file called [Walkfile][WALKFILE] to determine +what dependencies the target has, and how to execute it. walk(1) searches for a +Walkfile starting from the target's directory and walking up the directory tree +until one is found. This allows a single Walkfile at the project root to handle +targets in any subdirectory. ## WALKFILE The `Walkfile` determines _how_ a target is executed, and what other targets it depends on. -When walk(1) begins execution of a target, it attempts to find an executable -file called `Walkfile` in the same directory as the target, and then executes -it with the following positional arguments: +When walk(1) begins execution of a target, it searches for an executable file +called `Walkfile` starting from the target's directory and walking up the +directory tree. Once found, it executes the Walkfile with the following +positional arguments: * `$1`: The [phase][PHASES] (`deps` or `exec`). - + * `$2`: - The name of the target to build (e.g. `hello.o`). + The target path relative to the Walkfile's directory (e.g. `hello.o` or + `subdir/hello.o` if the Walkfile is in a parent directory). It's up to the `Walkfile` to determine what dependencies the target has, and how to execute it. +If a Walkfile exits with code `200`, walk(1) will try the next Walkfile up the +directory tree. This allows a local Walkfile to handle specific targets while +delegating unknown targets to a parent Walkfile: + + case $target in + special) ;; # handle locally + *) exit 200 ;; # delegate to parent + esac + ## PHASES walk(1) has two phases: diff --git a/plan.go b/plan.go index 1214fba..99add0b 100644 --- a/plan.go +++ b/plan.go @@ -221,6 +221,10 @@ func (e *targetError) Error() string { return fmt.Sprintf("%s: %v", e.target.Name(), e.err) } +// ExitCodeDelegate is the exit code that signals walk to try the next +// Walkfile in the inheritance chain. +const ExitCodeDelegate = 200 + // target is a Target implementation, that represents a file on disk, which may // be built by a rule. type target struct { @@ -230,13 +234,14 @@ type target struct { // The absolute path to the file. path string - // path to the rulefile to use. This is determined by the RuleFile - // function. - rulefile string + // rulefiles is the list of candidate Walkfiles, ordered from most + // specific (closest to target) to least specific (furthest up the tree). + rulefiles []string - // the directory to use as the working directory when executing the - // build file. - dir string + // rulefileIdx is the index into rulefiles of the active Walkfile. + // This is set during Dependencies() when a Walkfile successfully handles + // the target (doesn't return ExitCodeDelegate). + rulefileIdx int // The working directory. wd string @@ -247,21 +252,39 @@ type target struct { // newTarget initializes and returns a new target instance. func newTarget(wd, name string) *target { path := abs(wd, name) + rulefiles := RuleFiles(path) - rulefile := RuleFile(path) + return &target{ + name: name, + path: path, + rulefiles: rulefiles, + wd: wd, + } +} - var dir string - if rulefile != "" { - dir = filepath.Dir(path) +// rulefile returns the currently active Walkfile path. +func (t *target) rulefile() string { + if len(t.rulefiles) == 0 { + return "" } + return t.rulefiles[t.rulefileIdx] +} - return &target{ - name: name, - path: path, - rulefile: rulefile, - dir: dir, - wd: wd, +// dir returns the working directory for the active Walkfile. +func (t *target) dir() string { + if rf := t.rulefile(); rf != "" { + return filepath.Dir(rf) + } + return "" +} + +// targetName returns the target name relative to the active Walkfile's directory. +func (t *target) targetName() string { + if dir := t.dir(); dir != "" { + rel, _ := filepath.Rel(dir, t.path) + return rel } + return "" } // Name implements the Target interface. @@ -272,7 +295,7 @@ func (t *target) Name() string { // Exec executes the rule with "exec" as the first argument. func (t *target) Exec(ctx context.Context) error { // No .walk file, meaning it's a static dependency. - if t.rulefile == "" { + if t.rulefile() == "" { return nil } @@ -287,51 +310,66 @@ func (t *target) Exec(ctx context.Context) error { // out the newline delimited list of dependencies. func (t *target) Dependencies(ctx context.Context) ([]string, error) { // No .walk file, meaning it's a static dependency. - if t.rulefile == "" { + if len(t.rulefiles) == 0 { return nil, nil } - b := new(bytes.Buffer) - cmd, err := t.ruleCommand(ctx, PhaseDeps) - if err != nil { - return nil, err - } - cmd.Stdout = b + // Try each Walkfile in order until one handles the target + // (doesn't return ExitCodeDelegate). + var lastErr error + for i := range t.rulefiles { + t.rulefileIdx = i - if err := cmd.Run(); err != nil { - return nil, err - } - - var deps []string - scanner := bufio.NewScanner(b) - for scanner.Scan() { - path := scanner.Text() - if path == "" { - continue + b := new(bytes.Buffer) + cmd, err := t.ruleCommand(ctx, PhaseDeps) + if err != nil { + return nil, err } + cmd.Stdout = b - // If the path is not already and absolute path, make it one. - if !filepath.IsAbs(path) { - path = filepath.Join(t.dir, path) + if err := cmd.Run(); err != nil { + // Check if this is a fallback signal + if exitErr, ok := err.(*exec.ExitError); ok && exitErr.ExitCode() == ExitCodeDelegate { + lastErr = err + continue // Try next Walkfile + } + return nil, err } - // Make all paths relative to the working directory. - path, err := filepath.Rel(t.wd, path) - if err != nil { - return deps, err + // This Walkfile handled it - parse dependencies + var deps []string + scanner := bufio.NewScanner(b) + for scanner.Scan() { + path := scanner.Text() + if path == "" { + continue + } + + // If the path is not already and absolute path, make it one. + if !filepath.IsAbs(path) { + path = filepath.Join(t.dir(), path) + } + + // Make all paths relative to the working directory. + path, err := filepath.Rel(t.wd, path) + if err != nil { + return deps, err + } + deps = append(deps, path) } - deps = append(deps, path) + + return deps, scanner.Err() } - return deps, scanner.Err() + // All Walkfiles returned fallback - return the last error + return nil, lastErr } func (t *target) ruleCommand(ctx context.Context, phase string) (*exec.Cmd, error) { - name := filepath.Base(t.path) - cmd := exec.CommandContext(ctx, t.rulefile, phase, name) + cmd := exec.CommandContext(ctx, t.rulefile(), phase, t.targetName()) cmd.Stdout = t.stdout cmd.Stderr = t.stderr - cmd.Dir = t.dir + cmd.Dir = t.dir() return cmd, nil } @@ -353,7 +391,7 @@ func (t *verboseTarget) Exec(ctx context.Context) error { if err != nil { line = fmt.Sprintf("%s\t%s", line, err) } - if t.rulefile != "" { + if t.rulefile() != "" { fmt.Fprintf(t.stdout, "%s\n", line) } if err != nil { @@ -363,23 +401,36 @@ func (t *verboseTarget) Exec(ctx context.Context) error { } // RuleFile is used to determine the path to an executable which will be used as -// the Rule to execute the given target. At the moment, this simply looks for an -// executable file called `Walkfile` in the same directory as the target. +// the Rule to execute the given target. It walks up the directory tree from the +// target's directory until it finds a Walkfile. func RuleFile(path string) string { - dir := filepath.Dir(path) - try := []string{ - Walkfile, + rulefiles := RuleFiles(path) + if len(rulefiles) == 0 { + return "" } + return rulefiles[0] +} + +// RuleFiles returns all candidate Walkfiles for the given target path, ordered +// from most specific (closest to target) to least specific (furthest ancestor). +func RuleFiles(path string) []string { + var rulefiles []string + dir := filepath.Dir(path) - for _, n := range try { - path := filepath.Join(dir, n) - _, err := os.Stat(path) - if err == nil { - return path + for { + walkfile := filepath.Join(dir, Walkfile) + if _, err := os.Stat(walkfile); err == nil { + rulefiles = append(rulefiles, walkfile) + } + + parent := filepath.Dir(dir) + if parent == dir { + break } + dir = parent } - return "" + return rulefiles } // prefixWriter wraps an io.Writer to append a prefix to each line written. diff --git a/plan_test.go b/plan_test.go index fc292d6..76bc6dc 100644 --- a/plan_test.go +++ b/plan_test.go @@ -94,22 +94,27 @@ func TestPlan_Error(t *testing.T) { } func TestPlan_NoWalkfile(t *testing.T) { - clean(t) + // Use a temp directory that truly has no Walkfile anywhere in its ancestry + tmpdir := t.TempDir() + targetPath := filepath.Join(tmpdir, "all") b := new(bytes.Buffer) plan := newPlan() plan.NewTarget = NewTarget(TargetOptions{ - Stdout: b, + WorkingDir: tmpdir, + Stdout: b, }) - err := plan.Plan(ctx, "test/000-no-walkfile/all") + err := plan.Plan(ctx, "all") assert.NoError(t, err) err = plan.Exec(ctx, NewSemaphore(0)) assert.NoError(t, err) - // If there's no Walkfile in the directory, it might just be a static + // If there's no Walkfile in the directory (or any parent), it's a static // file. We don't really need to show these in output. assert.Equal(t, "", b.String()) + + _ = targetPath // silence unused warning } func TestPrefixWriter(t *testing.T) { @@ -161,6 +166,178 @@ prefix: to do the job in dark, airless conditions? `, b.String()) } +func TestRuleFile_Inheritance(t *testing.T) { + // Create a temp directory structure: + // tmpdir/ + // Walkfile <- should be found + // subdir/ + // target.txt <- target here, no local Walkfile + tmpdir := t.TempDir() + + walkfile := filepath.Join(tmpdir, "Walkfile") + err := os.WriteFile(walkfile, []byte("#!/bin/bash\necho test"), 0755) + assert.NoError(t, err) + + subdir := filepath.Join(tmpdir, "subdir") + err = os.Mkdir(subdir, 0755) + assert.NoError(t, err) + + targetPath := filepath.Join(subdir, "target.txt") + + // RuleFile should find the Walkfile in the parent directory + found := RuleFile(targetPath) + assert.Equal(t, walkfile, found) +} + +func TestRuleFile_Inheritance_EndToEnd(t *testing.T) { + // Create a temp directory structure with a Walkfile that handles subdirectory targets + tmpdir := t.TempDir() + + // Walkfile that echoes the target name for deps and creates a marker file on exec + walkfile := filepath.Join(tmpdir, "Walkfile") + err := os.WriteFile(walkfile, []byte(`#!/bin/bash +phase=$1 +target=$2 + +case $phase in + deps) ;; # no deps + exec) touch "$target.built" ;; +esac +`), 0755) + assert.NoError(t, err) + + // Create a subdirectory (no Walkfile here - should inherit) + subdir := filepath.Join(tmpdir, "subdir") + err = os.Mkdir(subdir, 0755) + assert.NoError(t, err) + + b := new(bytes.Buffer) + plan := newPlan() + plan.NewTarget = NewTarget(TargetOptions{ + WorkingDir: tmpdir, + Stdout: b, + }) + + // Build a target in the subdirectory + err = plan.Plan(ctx, "subdir/foo") + assert.NoError(t, err) + + err = plan.Exec(ctx, NewSemaphore(0)) + assert.NoError(t, err) + + // Verify the Walkfile was invoked and created the marker file + _, err = os.Stat(filepath.Join(tmpdir, "subdir/foo.built")) + assert.NoError(t, err, "expected subdir/foo.built to exist") +} + +func TestRuleFile_Fallback(t *testing.T) { + // Test that a local Walkfile can delegate to parent by exiting with code 200 + // Structure: + // tmpdir/ + // Walkfile <- handles *.o generically + // subdir/ + // Walkfile <- handles "special" only, exits 200 for others + // foo.o <- should fall back to parent's *.o rule + // special <- handled by local Walkfile + tmpdir := t.TempDir() + + // Parent Walkfile: handles *.o + parentWalkfile := filepath.Join(tmpdir, "Walkfile") + err := os.WriteFile(parentWalkfile, []byte(`#!/bin/bash +phase=$1 +target=$2 + +case $target in + *.o) + case $phase in + exec) touch "$target.from-parent" ;; + esac ;; + *) exit 200 ;; +esac +`), 0755) + assert.NoError(t, err) + + // Create subdirectory + subdir := filepath.Join(tmpdir, "subdir") + err = os.Mkdir(subdir, 0755) + assert.NoError(t, err) + + // Local Walkfile: handles "special" only, exits 200 for unknown targets + localWalkfile := filepath.Join(subdir, "Walkfile") + err = os.WriteFile(localWalkfile, []byte(`#!/bin/bash +phase=$1 +target=$2 + +case $target in + special) + case $phase in + exec) touch "$target.from-local" ;; + esac ;; + *) exit 200 ;; # Signal: try parent Walkfile +esac +`), 0755) + assert.NoError(t, err) + + // Test 1: "special" should be handled by local Walkfile + b := new(bytes.Buffer) + plan := newPlan() + plan.NewTarget = NewTarget(TargetOptions{ + WorkingDir: tmpdir, + Stdout: b, + }) + + err = plan.Plan(ctx, "subdir/special") + assert.NoError(t, err) + err = plan.Exec(ctx, NewSemaphore(0)) + assert.NoError(t, err) + + _, err = os.Stat(filepath.Join(subdir, "special.from-local")) + assert.NoError(t, err, "expected special.from-local to exist") + + // Test 2: "foo.o" should fall back to parent Walkfile + plan = newPlan() + plan.NewTarget = NewTarget(TargetOptions{ + WorkingDir: tmpdir, + Stdout: b, + }) + + err = plan.Plan(ctx, "subdir/foo.o") + assert.NoError(t, err) + err = plan.Exec(ctx, NewSemaphore(0)) + assert.NoError(t, err) + + _, err = os.Stat(filepath.Join(tmpdir, "subdir/foo.o.from-parent")) + assert.NoError(t, err, "expected subdir/foo.o.from-parent to exist") +} + +func TestRuleFile_LocalOverridesParent(t *testing.T) { + // Create a temp directory structure: + // tmpdir/ + // Walkfile <- parent Walkfile + // subdir/ + // Walkfile <- local Walkfile (should be used) + // target.txt + tmpdir := t.TempDir() + + parentWalkfile := filepath.Join(tmpdir, "Walkfile") + err := os.WriteFile(parentWalkfile, []byte("#!/bin/bash\necho parent"), 0755) + assert.NoError(t, err) + + subdir := filepath.Join(tmpdir, "subdir") + err = os.Mkdir(subdir, 0755) + assert.NoError(t, err) + + localWalkfile := filepath.Join(subdir, "Walkfile") + err = os.WriteFile(localWalkfile, []byte("#!/bin/bash\necho local"), 0755) + assert.NoError(t, err) + + targetPath := filepath.Join(subdir, "target.txt") + + // RuleFile should find the local Walkfile first + found := RuleFile(targetPath) + assert.Equal(t, localWalkfile, found) +} + func clean(t testing.TB) { err := Exec(ctx, NewSemaphore(0), "test/clean") assert.NoError(t, err) diff --git a/test/110-compile/Walkfile b/test/110-compile/Walkfile index a5f3c5d..31434ec 100755 --- a/test/110-compile/Walkfile +++ b/test/110-compile/Walkfile @@ -22,15 +22,14 @@ case $target in exec) clean ;; esac ;; - hello.o) - case $phase in - deps) echo hello.c ;; - exec) exec gcc -Wall -o hello.o -c $($0 deps $target) ;; - esac ;; + test) ;; # phony target hello) case $phase in deps) echo hello.o ;; exec) exec gcc -Wall -o hello $($0 deps $target) ;; - esac + esac ;; + + # Delegate unknown targets (like *.o) to parent Walkfile + *) exit 200 ;; esac diff --git a/test/Walkfile b/test/Walkfile index d0b351e..60a710b 100755 --- a/test/Walkfile +++ b/test/Walkfile @@ -40,5 +40,17 @@ case $target in exec) exec go test -race ../... esac ;; + # Generic rule: compile any .c file to .o + # This is inherited by subdirectories via exit 200 + */*.o) + src=${target%.o}.c + case $phase in + deps) echo $src ;; + exec) exec gcc -Wall -o $target -c $src ;; + esac ;; + + # Static files - no build needed + */*.c|*/*.h) ;; + *) >&2 echo "No rule for target \"$target\"" && exit 1 ;; esac