Skip to content

Commit 9b3dd02

Browse files
committed
chore: add script to audit changelog labels on merged PRs
1 parent 192c22f commit 9b3dd02

4 files changed

Lines changed: 327 additions & 20 deletions

File tree

admin/README.md

Lines changed: 24 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,33 @@ This folder contains tools or docs useful for project maintainers.
44

55
## Repositories inside https://github.com/codeigniter4
66

7-
- **CodeIgniter4** is the main development repository.
8-
It supports issues and pull requests, and has a rule to enforce GPG-signed commits.
9-
In addition to the framework source, it includes unit testing and documentation source.
10-
The three repositories following are built from this one as part of the release workflow.
7+
- **CodeIgniter4** is the main development repository.
8+
It supports issues and pull requests, and has a rule to enforce GPG-signed commits.
9+
In addition to the framework source, it includes unit testing and documentation source.
10+
The three repositories following are built from this one as part of the release workflow.
1111
This repo is meant to be forked by contributors.
12-
- **framework** is the released developer repository.
12+
- **framework** is the released developer repository.
1313
It contains all the main pieces of the framework that developers would use to
14-
build their apps, but not the framework unit testing or the user guide source.
15-
It is meant to be downloaded by developers, or composer-installed.
14+
build their apps, but not the framework unit testing or the user guide source.
15+
It is meant to be downloaded by developers, or composer-installed.
1616
This is a read-only repository.
17-
- **appstarter** is the released application starter repository.
17+
- **appstarter** is the released application starter repository.
1818
It is derived from the framework's `app` and `public` folders, with
19-
a composer requirement dependency to pull in the framework itself.
20-
It is meant to be downloaded or composer-installed.
19+
a composer requirement dependency to pull in the framework itself.
20+
It is meant to be downloaded or composer-installed.
2121
This is a read-only repository.
22-
- **userguide** is released documentation publishing repository.
22+
- **userguide** is released documentation publishing repository.
2323
It contains built versions of the user guide, corresponding to the
24-
framework releases.
25-
It could be downloaded, forked or potentially composer-installed.
24+
framework releases.
25+
It could be downloaded, forked or potentially composer-installed.
2626
This is a read-only repository.
27-
- **coding-standard** <https://github.com/CodeIgniter/coding-standard> is the coding style standards repository.
27+
- **coding-standard** <https://github.com/CodeIgniter/coding-standard> is the coding style standards repository.
2828
It contains PHP-CS-Fixer rules to ensure consistent code style
29-
within the framework itself.
29+
within the framework itself.
3030
It is meant to be composer-installed.
3131
- **translations** is the repository holding official translations of
32-
the locale-dependent system messages.
33-
It is community-maintained, and accepts issues and pull requests.
32+
the locale-dependent system messages.
33+
It is community-maintained, and accepts issues and pull requests.
3434
It could be downloaded, forked or composer-installed.
3535

3636
## Contributor Scripts
@@ -48,18 +48,22 @@ are used as part of that process:
4848

4949
- **prepare-release.php** creates the `release-4.x.x` branch and updates
5050
version references in the framework source, the user guide, and the
51-
distribution build script.
51+
distribution build script.
5252
Usage: `php admin/prepare-release.php 4.x.x`
5353
- **create-new-changelog.php** creates the changelog and upgrade guide
54-
stubs for the next version and adds them to their index files.
54+
stubs for the next version and adds them to their index files.
5555
Usage: `php admin/create-new-changelog.php <current_version> <new_version>`
56+
- **check-pr-labels.php** flags PRs merged after a release (default: the
57+
latest) that appear to be missing the labels used to generate the
58+
changelog.
59+
Usage: `php admin/check-pr-labels.php [<version>]`
5660

5761
## Other Stuff
5862

5963
- The **framework** and **starter** subfolders contain files that will over-ride
6064
those from the development repository, when the distribution repositories
6165
are built.
62-
- The subfolders inside `admin` contain "next release" files in the case of
66+
- The subfolders inside `admin` contain "next release" files in the case of
6367
`codeigniter4` and over-written distribution files in the other cases.
6468
- The CHANGELOG.md file is generated from GitHub's auto-generated release
6569
notes, as described in [RELEASE.md](./RELEASE.md).

admin/RELEASE.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,11 @@ the following [labels](https://github.com/codeigniter4/CodeIgniter4/labels):
7474
PRs with breaking changes must have the following additional label:
7575
- **breaking change** ... PRs that may break existing functionalities
7676

77+
At any time, missing labels can be checked for with
78+
`php admin/check-pr-labels.php`, which flags PRs merged since the last release
79+
that appear to lack a changelog label. It requires the authenticated
80+
[GitHub CLI](https://cli.github.com/).
81+
7782
### Generate Changelog
7883

7984
To auto-generate the changelog, navigate to the

admin/check-pr-labels.php

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
/**
6+
* Checks that PRs merged since the last release carry the labels used to
7+
* generate the changelog, suggesting a label from the PR title type.
8+
*
9+
* Requires the authenticated GitHub CLI (`gh`).
10+
*
11+
* Usage: php admin/check-pr-labels.php [version]
12+
*/
13+
function color(string $text, string $code, bool $ansi): string
14+
{
15+
return $ansi ? sprintf("\033[%sm%s\033[0m", $code, $text) : $text;
16+
}
17+
18+
function hyperlink(string $text, string $url, bool $ansi): string
19+
{
20+
return $ansi ? sprintf("\033]8;;%s\033\\%s\033]8;;\033\\", $url, $text) : $text;
21+
}
22+
23+
function format_pull_line(array $pull, bool $ansi): string
24+
{
25+
return sprintf(
26+
'* %s %s %s',
27+
hyperlink(color(sprintf('#%d', $pull['number']), '36', $ansi), $pull['url'], $ansi),
28+
$pull['title'],
29+
color(sprintf('(%s)', $pull['base']), '35', $ansi),
30+
);
31+
}
32+
33+
function touches_system_dir(string $repo, int $number): bool
34+
{
35+
exec(sprintf("gh api repos/%s/pulls/%d/files --paginate --jq '.[].filename' 2>&1", $repo, $number), $files, $exitCode);
36+
37+
if ($exitCode !== 0) {
38+
return true;
39+
}
40+
41+
foreach ($files as $file) {
42+
if (str_starts_with($file, 'system/')) {
43+
return true;
44+
}
45+
}
46+
47+
return false;
48+
}
49+
50+
chdir(__DIR__ . '/..');
51+
52+
$ansi = stream_isatty(STDOUT);
53+
$repo = 'codeigniter4/CodeIgniter4';
54+
55+
// Labels used by the changelog categories. See ".github/release.yml".
56+
$changelogLabels = ['breaking change', 'bug', 'enhancement', 'new feature', 'refactor'];
57+
58+
// PR title types that map to a changelog label.
59+
$typeToLabel = [
60+
'fix' => 'bug',
61+
'feat' => 'new feature',
62+
'perf' => 'enhancement',
63+
'refactor' => 'refactor',
64+
];
65+
66+
// PR title types that need no changelog label.
67+
$typesWithoutLabel = ['chore', 'ci', 'docs', 'style', 'test'];
68+
69+
// Release process PRs carry no labels.
70+
$releaseTitles = '/\A(?:Prep for \d+\.\d+\.\d+ release|\d+\.\d+\.\d+ (?:Ready|Merge) code)\z/';
71+
72+
$tag = null;
73+
74+
foreach (array_slice($argv, 1) as $arg) {
75+
if (preg_match('/\Av?(\d+\.\d+\.\d+)\z/', $arg, $matches) === 1) {
76+
$tag = "v{$matches[1]}";
77+
78+
continue;
79+
}
80+
81+
echo sprintf("Usage: php %s [version]\n", $argv[0]);
82+
echo sprintf("E.g.,: php %s 4.7.3\n", $argv[0]);
83+
echo "Checks the PRs merged after the given release. Defaults to the latest release.\n";
84+
85+
exit(1);
86+
}
87+
88+
$endpoint = $tag === null ? sprintf('repos/%s/releases/latest', $repo) : sprintf('repos/%s/releases/tags/%s', $repo, $tag);
89+
exec(sprintf('gh api %s 2>&1', $endpoint), $output, $exitCode);
90+
91+
if ($exitCode !== 0) {
92+
echo sprintf("Failed to fetch the release %s from GitHub:\n", $tag ?? '(latest)');
93+
echo implode("\n", $output) . "\n";
94+
95+
exit(1);
96+
}
97+
98+
$release = json_decode(implode("\n", $output), true);
99+
$since = $release['published_at'] ?? '';
100+
101+
echo sprintf("Checking PRs merged since %s (%s).\n", $release['tag_name'], $since);
102+
103+
$command = sprintf(
104+
'gh pr list --repo %s --search %s --json number,title,labels,baseRefName,url --limit 300 2>&1',
105+
$repo,
106+
escapeshellarg(sprintf('is:merged -base:master merged:>%s', $since)),
107+
);
108+
exec($command, $prOutput, $exitCode);
109+
110+
if ($exitCode !== 0) {
111+
echo "Failed to fetch the merged PRs from GitHub:\n";
112+
echo implode("\n", $prOutput) . "\n";
113+
114+
exit(1);
115+
}
116+
117+
$pulls = json_decode(implode("\n", $prOutput), true);
118+
119+
if (! is_array($pulls)) {
120+
echo "Unexpected response from GitHub.\n";
121+
122+
exit(1);
123+
}
124+
125+
echo sprintf("Found %d merged PRs.\n\n", count($pulls));
126+
127+
$missingLabel = [];
128+
$needManualLook = [];
129+
130+
foreach ($pulls as $pull) {
131+
$labels = array_column($pull['labels'], 'name');
132+
$hasLabel = array_intersect($changelogLabels, $labels) !== [];
133+
$type = null;
134+
135+
if (preg_match('/\A(\w+)(?:\([^)]*\))?:/', $pull['title'], $matches) === 1) {
136+
$type = $matches[1];
137+
}
138+
139+
if ($hasLabel || in_array($type, $typesWithoutLabel, true)) {
140+
continue;
141+
}
142+
143+
if (preg_match($releaseTitles, $pull['title']) === 1) {
144+
continue;
145+
}
146+
147+
if ($type !== null && isset($typeToLabel[$type])) {
148+
// The refactor label applies only to refactoring in system/. Test-only
149+
// refactors may use the testing label, which is not in the changelog.
150+
if ($type === 'refactor' && ! touches_system_dir($repo, (int) $pull['number'])) {
151+
continue;
152+
}
153+
154+
$missingLabel[] = [
155+
'number' => $pull['number'],
156+
'title' => $pull['title'],
157+
'base' => $pull['baseRefName'],
158+
'url' => $pull['url'],
159+
'label' => $typeToLabel[$type],
160+
];
161+
} else {
162+
$needManualLook[] = [
163+
'number' => $pull['number'],
164+
'title' => $pull['title'],
165+
'base' => $pull['baseRefName'],
166+
'url' => $pull['url'],
167+
];
168+
}
169+
}
170+
171+
if ($missingLabel !== []) {
172+
echo color('PRs that appear to be missing a changelog label:', '1', $ansi) . "\n";
173+
174+
foreach ($missingLabel as $pull) {
175+
echo sprintf("%s [suggested: %s]\n", format_pull_line($pull, $ansi), color($pull['label'], '33', $ansi));
176+
}
177+
178+
echo "\nTo add the suggested labels, run the following commands (drop any that are not warranted):\n";
179+
180+
foreach ($missingLabel as $pull) {
181+
echo sprintf("gh pr edit %d --repo %s --add-label \"%s\"\n", $pull['number'], $repo, $pull['label']);
182+
}
183+
184+
echo "\n";
185+
}
186+
187+
if ($needManualLook !== []) {
188+
echo color('PRs with no changelog label and no recognized title type (check manually):', '1', $ansi) . "\n";
189+
190+
foreach ($needManualLook as $pull) {
191+
echo format_pull_line($pull, $ansi) . "\n";
192+
}
193+
194+
echo "\n";
195+
}
196+
197+
if ($missingLabel === []) {
198+
echo color('No PRs are missing changelog labels.', '32', $ansi) . "\n";
199+
200+
exit(0);
201+
}
202+
203+
exit(1);
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
/**
6+
* This file is part of CodeIgniter 4 framework.
7+
*
8+
* (c) CodeIgniter Foundation <admin@codeigniter.com>
9+
*
10+
* For the full copyright and license information, please view
11+
* the LICENSE file that was distributed with this source code.
12+
*/
13+
14+
namespace CodeIgniter\AutoReview;
15+
16+
use Nexus\PHPUnit\Tachycardia\Attribute\TimeLimit;
17+
use PHPUnit\Framework\Attributes\CoversNothing;
18+
use PHPUnit\Framework\Attributes\Group;
19+
use PHPUnit\Framework\TestCase;
20+
21+
/**
22+
* @internal
23+
*/
24+
#[CoversNothing]
25+
#[Group('AutoReview')]
26+
final class CheckPrLabelsTest extends TestCase
27+
{
28+
public function testUsageErrorWithUnknownArgument(): void
29+
{
30+
exec('php ./admin/check-pr-labels.php foo 2>&1', $output, $exitCode);
31+
32+
$this->assertSame(1, $exitCode);
33+
$this->assertStringContainsString('Usage:', implode("\n", $output));
34+
}
35+
36+
public function testUsageErrorWithInvalidVersion(): void
37+
{
38+
exec('php ./admin/check-pr-labels.php 4.7 2>&1', $output, $exitCode);
39+
40+
$this->assertSame(1, $exitCode);
41+
$this->assertStringContainsString('Usage:', implode("\n", $output));
42+
}
43+
44+
#[TimeLimit(10.0)]
45+
public function testChecksAgainstLatestRelease(): void
46+
{
47+
$this->skipUnlessGhIsAuthenticated();
48+
49+
exec('php ./admin/check-pr-labels.php 2>&1', $output, $exitCode);
50+
$outputString = implode("\n", $output);
51+
52+
$this->assertContains($exitCode, [0, 1], "Script exited with code {$exitCode}. Output: {$outputString}");
53+
$this->assertMatchesRegularExpression('/Checking PRs merged since v\d+\.\d+\.\d+/', $outputString);
54+
$this->assertMatchesRegularExpression('/Found \d+ merged PRs\./', $outputString);
55+
}
56+
57+
#[TimeLimit(10.0)]
58+
public function testChecksSinceGivenRelease(): void
59+
{
60+
$this->skipUnlessGhIsAuthenticated();
61+
62+
$version = $this->latestChangelogVersion();
63+
64+
exec("php ./admin/check-pr-labels.php {$version} 2>&1", $output, $exitCode);
65+
$outputString = implode("\n", $output);
66+
67+
$this->assertContains($exitCode, [0, 1], "Script exited with code {$exitCode}. Output: {$outputString}");
68+
$this->assertStringContainsString("Checking PRs merged since v{$version} (", $outputString);
69+
$this->assertMatchesRegularExpression('/Found \d+ merged PRs\./', $outputString);
70+
}
71+
72+
private function latestChangelogVersion(): string
73+
{
74+
$changelog = (string) file_get_contents('./CHANGELOG.md');
75+
76+
if (preg_match('/^## \[v(\d+\.\d+\.\d+)\]/m', $changelog, $matches) !== 1) {
77+
$this->fail('Could not find a version entry in CHANGELOG.md.');
78+
}
79+
80+
return $matches[1];
81+
}
82+
83+
private function skipUnlessGhIsAuthenticated(): void
84+
{
85+
exec('gh auth status 2>&1', $output, $exitCode);
86+
87+
if ($exitCode !== 0) {
88+
$this->markTestSkipped(
89+
'The GitHub CLI is not available or not authenticated. This test is expected '
90+
. 'to be skipped in CI and runs only where `gh` is authenticated, such as on '
91+
. 'a maintainer\'s machine.',
92+
);
93+
}
94+
}
95+
}

0 commit comments

Comments
 (0)