Support for parsing gocheck tests when present

[sgp]

For those of us unlucky be stuck with a codebase hat uses the aging gocheck, but would like to be able to get the benefit of using gotestsum to report individual tests.

I think that this can be done by consuming the go test -json output and parsing any output from tests that look like gocheck output and promoting those to be top-level tests. For example, consider the following output created by running cat gocheck-before.json | go run gotest.tools/gotestsum@latest --format testname --raw-command -- cat

PASS somepackage.Test (0.02s)
PASS somepackage

DONE 1 tests in 0.633s

Ideally, I'd like to see something like this, created by modifying the JSON to promote the check-based tests to be the "real" tests in gocheck-after.json

PASS somepackage.MarshalTests.TestGeneric (0.00s)
PASS somepackage.MarshalTests.TestMarshal (0.00s)
PASS somepackage.MarshalTests.TestNoPacking (0.00s)
PASS somepackage.MarshalTests.TestPacked (0.00s)
PASS somepackage

DONE 4 tests in 0.633s

Of course, it's important for these tests to be not only visible in the output, but also recognized in the junit.xml emitted if that is desired.

It's arguable that this is too niche for gotestsum, as one could write a tool that would parse, filter and augment the emitted JSON and have that fed into gotestsum for the desired results. Also, getting this information requires the user to run the tests with -check.vv for it to show up at all, which is a limitation.

I may attempt a fork to do this for my own sake, knowing that it might not reach the mainline. I don't know how many people out there are still using gocheck, but it seems to be a non-zero amount of people based on the fact that there are abandoned projects (e.g. https://github.com/tebeka/go2xunit) out there that do support parsing gocheck test output into junit.xml.

gocheck-before.json
gocheck-after.json

[dnephin]

Another option, fork gocheck and have it emit the test2json output, change:

1. https://pkg.go.dev/gopkg.in/check.v1#C.Log and the other underlying log and write calls to use t.Log
2. suiteRunner.run to use t.Run
3. Instead of suiteRunner.runTest just run the test directly.

Something like that might be easier than parsing the output, and then gotestsum could read it directly by running it with go test -v.

The option I took years ago was to convert the test suite with an AST rewrite tool, https://github.com/gotestyourself/gotest.tools/tree/main/assert/cmd/gty-migrate-from-testify

I've seen others use sed and eg, moby/moby#39799 to do the same thing.

[sgp]

@dnephin That's not a bad idea at all. I might try that to see if it works for our codebase. One advantage to doing it this way is that gotestsum doesn't change at all. Furthermore, if it works without using any cache-busting test flags (as -check.vv will absolutely refuse to use the go test cache), that's even better.

In the meantime, I did make gotestsum work with gocheck when used with -check.vv. The patch I've committed "works" but is not battle-tested nor does it have unit tests (yet). It's not exactly pretty, and I would totally understand if no one wants to merge a hack like this into the main branch. That said, it solves my immediate needs.