Skip to content

Stop the release-promote eval requiring an svn commit the skill never emits #939

Description

@justinmclean

Summary

The release-promote step-2 grading rule requires the output to contain both
svn mv and a separate svn commit -m. The skill emits only svn mv with an
inline -m, which is correct for svn, so the rule grades against behaviour the
skill does not and should not have.

Background

A URL-to-URL svn mv is a server-side operation that commits immediately and
takes its own -m message. There is no working copy and therefore nothing for a
follow-up svn commit to do. The skill's command block gets this right: one
svn mv carrying -m "Promoting Apache <product-name> <version>...", then
svn list to verify, then the git tag.

The eval's grading rule asks for something else. That is the wrong way round:
the fixture should encode what the skill does, and here it encodes an extra
command that would be a no-op at best. Left as is, it pressures a model toward
emitting a spurious svn commit to satisfy the grader.

Two lines of skill prose use the same loose phrasing and can be tidied in the
same pass: they refer to "the svn commit message" when they mean the -m on
svn mv.

Where to look

  • tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/output-spec.md
    line 27 - the rule requiring svn mv and svn commit -m.
  • skills/release-promote/SKILL.md lines 272-292 - the actual command block.
    Note it has no svn commit.
  • skills/release-promote/SKILL.md lines 178 and 243 - the "svn commit
    message" wording that means the -m on svn mv.

Acceptance criteria

  • The svnpubsub grading rule requires what the skill emits: an svn mv whose
    -m references the [RESULT] [VOTE] URL.
  • The rule no longer requires a separate svn commit.
  • The two prose lines refer to the svn mv message rather than an
    svn commit message.
  • The release-promote eval suite passes.

Estimated effort

~1 hour for someone new to the codebase. Understanding why svn needs no second
command is most of it.

Getting started

New to Magpie? Start with
[Your first contribution](https://github.com/apache/magpie/blob/main/CONTRIBUTING.md#your-first-contribution).

Note: this issue was drafted by an AI-assisted mentoring tool and may contain
mistakes. A Magpie maintainer, a real person, will be the next to engage on it.
If something here looks wrong or unclear, please say so on the issue and a
maintainer will follow up. See
[CONTRIBUTING.md](https://github.com/apache/magpie/blob/main/CONTRIBUTING.md)
for how Magpie handles contributions.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions