Add configurable Python interpreter discovery command

[ting-hong-shieh]

Summary

• add python-interpreter-find-cmd as a configuration-only interpreter source
• execute the configured program directly from the config directory
• require successful UTF-8 output containing exactly one non-empty path
• resolve relative output paths from the config directory
• validate mutual exclusion with other interpreter-selection options and document explicit shell usage

Root cause

Pyrefly only supported fixed interpreter paths, known environment types, and executable-name lookup. Environment managers that require a command to discover the active interpreter could not participate without first modifying the shell environment that launched Pyrefly.

The option is an argv array and does not invoke a shell implicitly. Workflows requiring shell features can opt in explicitly with sh -c, cmd /C, or PowerShell.

User impact

Projects using tools such as Poetry, direnv, or Nix can provide a reproducible interpreter-discovery command in project configuration.

Testing

• cargo test interpreter (config, command execution, failure handling, and LSP interpreter tests)
• formatter and Clippy through test.py

Fixes #1662

[meta-codesync]

This pull request has been imported. If you are a Meta employee, you can view this in D115850800. (Because this pull request was imported automatically, there will not be any future comments.)

[grievejia]

Thanks for working on this! I found several issues that we need to fix before landing. See inline comments. Feel free to ask if any part is unclear, or if you have other thoughts on them!

[grievejia]

An empty array is not a valid command, but the current type accepts it and configuration parsing currently treats it as valid -- error only surfaces when Pyrefly tries to run the command.

Please reject an empty array when the configuration is read, and store a value that always contains a program plus optional arguments. This lets the execution code assume that a program is present.

[grievejia]

The caller reaches this code only after it confirms that the command is present. Please pass the command slice into this method instead of reading the optional field again. This makes the required input clear and avoid the need to re-check if the optional is none -- an error case that cannot occur.

[grievejia]

The discovery command runs before find_interpreter has selected the highest-priority source. This method then marks the result as Auto. Later, the configured-source branch accepts only ConfigFile values. If a configuration contains both this command and conda-environment, Pyrefly runs the command, skips its result, and selects Conda. The validation reports a warning but does not stop this execution.

This is bad because a configured external program can run even when Pyrefly will not use its output. It also makes the real priority differ from the documented priority.

Please make this method accept the command slice and return only a PathBuf. Then call it in find_interpreter at the point where the configured command has won, before the configured Conda branch:

if let Some(command) = self.python_interpreter_find_cmd.as_deref() {
    let interpreter =
        Self::find_interpreter_from_command(command, path)?;
    return Ok(ConfigOrigin::auto(interpreter));
}

At that point, the early return has already applied the command's priority. The resolved path can remain Auto, so Pyrefly does not serialize it as an explicit python-interpreter-path.

[grievejia]

current_dir sets the working directory for the child process, but it does not give a relative program path a stable base on every platform. Rust documents this case as platform-specific and unstable. For example, ["./tools/find-python"] can resolve from the configuration directory on one platform and from the directory that started Pyrefly on another.

This can make the same project configuration fail on another platform or run a different file. It also conflicts with the documentation, which says that the command runs from the configuration directory.

The existing which dependency provides which_in, which resolves the executable before the process starts. working_directory already contains the configuration directory because the caller passes self.source.root(). One possible implementation is:

let program = match working_directory {
    Some(root) => which_in(program, std::env::var_os("PATH"), root),
    None => which(program),
}
.with_context(|| "Could not resolve the interpreter discovery command")?;

let mut command = Command::new(program);

which_in keeps an absolute path, resolves a relative path that contains a separator from root, and searches PATH for a bare name. Please add a Windows test that covers this case.

[grievejia]

The documentation says that all five interpreter-selection options are mutually exclusive, but the checks are split across three places. Some pairs are missed. For example, skip-interpreter-query = true together with python-interpreter-path = "./python" produces no warning, and the skip option silently wins.

This makes the documented rule unreliable. It also means that each new option needs more pair-specific conditions, such as the new python_interpreter_find_cmd.is_none() exception later in this method.

Please replace the separate checks with one check near the start of configure. Collect the names of all explicit selections, then report one warning when more than one is present:

let mut selections = Vec::new();

if matches!(
    self.interpreters.python_interpreter_path.as_ref(),
    Some(ConfigOrigin::CommandLine(_) | ConfigOrigin::ConfigFile(_))
) {
    selections.push("python-interpreter-path");
}
if self.interpreters.python_interpreter_find_cmd.is_some() {
    selections.push("python-interpreter-find-cmd");
}
if self.interpreters.fallback_python_interpreter_name.is_some() {
    selections.push("fallback-python-interpreter-name");
}
if self.interpreters.conda_environment.is_some() {
    selections.push("conda-environment");
}
if self.interpreters.skip_interpreter_query {
    selections.push("skip-interpreter-query");
}

if selections.len() > 1 {
    configure_errors.push(anyhow::anyhow!(
        "Only one interpreter selection option can be set, but found: {}.",
        selections.join(", "),
    ));
}

After this change, remove the path-versus-fallback check inside the skip_interpreter_query branch and the path-versus-Conda check near the end of configure. Extend the test with the missing skip-interpreter-query plus path case and with the command combined with each of the other four options.

[grievejia]

This is a new public configuration option, but it is absent from the JSON schema. Editors therefore cannot complete the option or validate its value in pyrefly.toml and pyproject.toml. The root schema permits unknown properties, so the current schema test does not report the omission.

Please update these three files:

1. Add this property beside the other interpreter options in schemas/pyrefly.json:

"python-interpreter-find-cmd": {
  "description": "A program and its arguments that print the path of the Python interpreter to query.",
  "type": "array",
  "items": {
    "type": "string"
  },
  "minItems": 1
}

2. Add this value to schemas/test-pyrefly.toml:

python-interpreter-find-cmd = ["poetry", "env", "info", "-e"]

3. Add the same value under [tool.pyrefly] in schemas/test-pyproject.toml.

[grievejia]

This test prints a constant relative path. If command.current_dir(working_directory) is removed, the command still prints the same text, and the later path-joining code still produces the expected value. The test therefore does not protect the documented rule that the command itself runs in the configuration directory.

Please keep this test as the check for relative output paths, and rename it to describe that purpose. Add a separate cross-platform test in which the command reports its real working directory:

#[cfg(any(unix, windows))]
#[test]
fn test_interpreter_find_command_uses_working_directory() {
    let tempdir = tempdir().unwrap();

    #[cfg(unix)]
    let command = ["sh", "-c", "pwd"];
    #[cfg(windows)]
    let command = ["cmd", "/C", "cd"];

    let interpreters = Interpreters {
        python_interpreter_find_cmd: Some(
            command.into_iter().map(str::to_owned).collect(),
        ),
        ..Default::default()
    };

    let interpreter = interpreters
        .find_interpreter(Some(tempdir.path()))
        .unwrap();

    assert_eq!(
        interpreter.as_path().canonicalize().unwrap(),
        tempdir.path().canonicalize().unwrap(),
    );
}

This test fails if the child working directory is no longer set. It also covers absolute command output and the Windows behavior described in the documentation.

[grievejia]

pyrefly dump-config formats this value after interpreter discovery. With python-interpreter-find-cmd = ["poetry", "env", "info", "-e"], a successful discovery currently prints only Using interpreter: /resolved/python. This new branch does not run after success because the resolved path is then set. If the intent is to show where the path came from, please handle the state where both the command and the resolved path are set. For example: Using interpreter: interpreter at path /resolved/python (from command poetry env info -e). This would match the existing output for the fallback command. If the source is not meant to be shown, this new branch is not needed.

[grievejia]

This list now contains more than two options, so "Both" is no longer correct. Please use "Only one of these options can be set in a configuration."