Return of OOM

[joeytakeda]

We've dealt with OOM errors before, most prominently documented here in #157 (which was closed and re-opened and then closed again). The last time it was closed we had resolved the issue in #172 by removing @fork='true' from the java calls. But it looks like @fork was added back because of security manager warnings in #247 .

I've just hit this again in a project. Trying to increase memory (as we suggest in the docs)

export ANT_OPTS="-Xmx6g"; ant 

still causes the build to fail for this project with OOM issues on the json step after ~ 15 mins or so. However, removing @fork from the java call in json does not throw any SecurityManager issues for me and completes really quickly.

I'm wondering if the security manager issues are still relevant? If so, maybe we could add a fork property to the build so that users needing to fork the processes to access higher memory allocations can do so?

[joeytakeda]

May be useful to note that I am running ant 1.10.14, which is the version cited in this comment about these warnings:

https://bz.apache.org/bugzilla/show_bug.cgi?id=65381#c10

[joeytakeda]

Just tried the same on Jenkins and saw those security manager warnings. I think if I'm understanding things correctly, those security manager warnings derive from versions of Java < 18; ant 1.10.14 and above requires Java 18+; Jenkins uses Java 17, which is why there's an issue there.

[martindholmes]

Fix committed -- do we need to document this?

[martindholmes]

Jenkins is currently using Java 21, incidentally.

[martindholmes]

I'm bringing this issue back, because I'm now seeing OOM errors in the json target for the BBTI project (which ends up needing to create 177,000+ JSON files for 135,000+ tokenized documents). I notice that @fork="true" is still there on the json target java call, so I'm currently testing removing that.