[policy] Good-faith agentic contributions and LLM use, and avoiding death by a thousand AI slops

[webknjaz]

I want to research including an "ai policy" across projects I'm involved in and I'm tracking some references to relevant documents and discussions @ https://gist.github.com/webknjaz/10f1106d0a3fd489745c2bc656400d1f. I've fed that gist into Claude and let it traverse like 357 sources out of that.

Here's what it came up with: https://claude.ai/public/artifacts/de2ffa40-98dc-4458-b1b5-f3e645bf7840. That document has some summaries and hightlights, principles/observations distilled, a policy draft, integration suggestions and guidelines for coping with spam.

I think it did a good job illuminating what's beeing observed across FOSS and showing how different parties react, why a clear/transparent document is even necessary. Let's talk about putting one up for pip-tools.

cc @sirosen @hugovk

[sirosen]

I have an overall much more negative posture towards the LLM tools than you do. I feel no hostility towards the users, but the technology is not being developed or promoted in a responsible way and we're seeing some of the consequences.
I have two separate areas to address below. First, as background info and sharing, why I think a ban is not viable. Second, what I think our policy should be.

My post here will barely touch on the ethical or legal issues. I have genuine concerns on these fronts, but I think any attempt to discuss them here will derail us.

Banning the tools is incorrect (for us)

I'm not concerned that a ban might drive some contributors away. Bluntly put, anyone who can't contribute without the help of an LLM probably shouldn't be trying to make changes in pip-tools. Even with a decent level of domain expertise, I've made some mistakes, and I don't think the problem space is in any way welcoming for a new developer.

But I don't believe that a policy banning LLM-assisted contributions (as, e.g., Gentoo has done) is appropriate. As much as I admire the clarity of this posture, I think this turns the project into a place I have to police for "bad contributions" which I then block/ban. I don't want to engage with the project in this way, and we haven't had a severe enough flood of slop on our project¹ to drive me towards supporting such a ban.

I think we'll still see some level of LLM-generated issue reports and PRs even with a ban. It then turns any further interaction negative, since the appropriate action is to close the issue without further evaluation.

I'm also not that keen on the idea of taking options away from contributors. If someone finds a useful way to leverage local models (which reduce many of the ethical concerns) to do their work, and their contributions are of high quality, why should I care to forbid them from using their preferred toolchain?²

Proposed policy: you are responsible for your work

I'd like us to put together a policy which is generic -- spans LLMs and any other current or future assistive tools -- and emphasizes the following two main aspects:

1. Disclosure. If you use a tool to do something, tell us what you did. If you wrote a libcst script to refactor some things, I'd expect you to share the script. By a similar line of thinking, "I wrote this issue in Turkish and used an LLM to translate it." at the end of an issue would be not only okay but good! We need to know what we are reviewing.

2. Ownership. You, not the LLM, are the owner of your code contribution. Maintainers should be able to ask you about any given line you are changing and you must be prepared to discuss it. You don't need mastery of the codebase, and it's okay if a part of your changeset is a guess at what's correct. But as the contributor you are responsible for your work.

The above is not quite a draft of the exact policy, but that's the main thrust of it. What do you think about it as a starting point? Are there major, important elements of good contributions which this misses?

I saw Claude suggested some bullet points aligned with the Ownership element. We might adapt some such list of affirmations into our PR template, but I don't think that's the core issue. I also see that it also used the word Disclosure, but defined it narrowly to LLMs. I'd rather we define it broadly -- partly, I think this sends a good social signal that LLM users aren't being singled out and penalized. Contributors who might be scared off by an overtly hostile policy will ideally find such a definition more welcoming.

---

I find it funny and somewhat topical that while skimming that document, I spotted an error which @webknjaz would be very unlikely to make:

From curl's Daniel Stenberg: [quotation follows]
From SymPy's Oscar Benjamin: [quotation follows]
From Python's Seth Larson: [quotation follows]
From Flask's Miguel Grinberg: [quotation follows]

This makes it sound like Miguel Grinberg is a Flask maintainer. Miguel's tutorials and books have done a lot to advance Flask, but he's not (and as far as I know, has never been) a maintainer. (I also find "Python's Seth Larson" to be odd phrasing for his role, but 🤷 on that)

Even the highest quality tool (Claude is well regarded) in the hands of a domain expert produces errors. I'm sure a deeper reading would reveal other errors.

Footnotes

1. 🤞 we've actually had hardly any 🤞 ↩

2. Possible answer: legal/copyright issues. I do not find this persuasive at all. If you hold that a large model and its outputs are a "derived work", then every output is GPL licensed (at the very least). It makes perfect sense if you look at licensing and legal systems as a programmer, "programming" in the legal system. Anyone who favors use of GPL/LGPL takes this view to some degree. But to actually enforce such copyright on all consumers of LLM outputs requires a level of collective political will which I cannot fathom. ↩

[webknjaz]

Agreed, I'm also against a ban. It's not enforceable and not 100% detectable anyway. Plus we have better things to do than hunting down offenders..

---

I also liked the disclosure and ownership bits. Especially the ownership bit — one should be able to reason about their patch coherently regardless of the tooling in use or not.

Regarding disclosure, I agree as well -- showing the prompts and stuff is exactly like showing "this is how I googled it / found on stackoverflow / reached the conclusion step-by-step". I've been thinking for a while that the issue forms should ask about the LLM in use and the prompts. Haven't implemented it anywhere, though. I know that some people suggest using https://aiattribution.github.io but that seems a bit too formal so I wouldn't insist. A generic disclosure sounds good. I also saw a mention that might've not made it into the document that one is responsible for ensuring their work doesn't violate copyright.

And those "Don't let AI speak for you" / "Don't let AI think for you" seemed relatable.

---

I didn't challange the quote author titles because that's a lot of work and I was mostly interested in the ideas at the end. The maintainer claims sounded weird but I don't know who davidism works with these days. Seth is mentioned in connection to PSF earlier but then it just "forgot" a part of the abbreviation, it seems. I also think that Claude's parser got rate-limited at some point and was unable to fetch the CPython's policy so it ended up claiming that there's none.

Note that this is the initial document generated in response to the very first prompt. I didn't iterate on it so there's no mark of a "domain expert" in it whatsoever ­— only this prompt (plus initial prompts in Claude account + project + crawling the web) contributed to the output.

---

I'm also interested in those maintainer-focused suggestions — quick red flag identification + response templates. Plus the contrib checklist looks reasonable.

[webknjaz]

Zulip shared (https://github.com/community/maintainers/discussions/646#discussioncomment-15065464) that they've also added an ask to verify accuracy: zulip/zulip#36744

[sirosen]

I'm not able to access github.com/community/maintainers (I just requested access; I expect it will be approved), but I see the link on the zulip project.

---

Regarding any linkage from issue forms or the PR template, I think let's get the contributing doc updated first, and then we can decide on how we want to link it. Similar to how, per the thread below, we may add AGENTS.md/CLAUDE.md after we get the policy set.

---

At this stage I have read enough that I'm getting diminishing returns from reading more of our options and ways that others are handling this. I think that the next step is to try to draft a policy, and we can discuss how to modify it. I can put some time into this in the next few days (although I want to finish reviewing the CI changes first! 😁 ). Would you like to do that in a thread here, or in a draft PR?

[webknjaz]

A draft PR sounds good, thanks!

[sirosen]

I'm not sure how to incorporate this, but I was recently shown this project which has a hard no-AI policy:
https://github.com/grishka/Smithereen/blob/49fdc3fb1653e4259eb66742dfcef8bcbc500c09/CLAUDE.md

Claude will apparently respect this, and refuses to help a user circumvent this instruction.

Per above, I don't think such a policy is appropriate for us, but it shows that a pretty short prompt can strongly influence the behavior (perhaps specifically if it's in the branded CLAUDE.md?).
I'd rather we stick to one of the evolving standards like llms.txt if we do something like this, but I think we should think about how we prompt agents to look at our policy doc in addition to the policy itself.

[webknjaz]

I'm not sure how to incorporate this, but I was recently shown this project which has a hard no-AI policy: https://github.com/grishka/Smithereen/blob/49fdc3fb1653e4259eb66742dfcef8bcbc500c09/CLAUDE.md

Interesting, although, I don't like the source, obviously.

Claude will apparently respect this, and refuses to help a user circumvent this instruction.

Maybe to an extent. One could probably hack this with a user-global memory file, I imagine.

Per above, I don't think such a policy is appropriate for us, but it shows that a pretty short prompt can strongly influence the behavior (perhaps specifically if it's in the branded CLAUDE.md?).

Yes, this is the default memory file that claude uses: https://code.claude.com/docs/en/memory. There's also CLAUDE.local.md that's beeing deprecated in favor of sourcing other files thought @-syntax. This one would be gitignored, normally. And there's ~/.claude/CLAUDE.md living outside the projects.

I'd rather we stick to one of the evolving standards like llms.txt if we do something like this

That thing is for the docs. We should stick https://sphinx-llms-txt.rtfd.io into our Sphinx env. But there's not much to do otherwise. Would be useful for someone scanning our docs with LLMs or training LLMs.

I think we should think about how we prompt agents to look at our policy doc in addition to the policy itself.

Yes, we should link it. There's AGENTS.md (although, their frontpage lies about the adoption stats) that's meant to be more universal than per-tool configs. Many products support it out of the box. At Ansible, we've gone for a trick of putting main instructions in that one and soucring it from CLAUDE.md: https://github.com/ansible/ansible/pull/85841/files#diff-6ebdb617a8104a7756d0cf36578ab01103dc9f07e4dc6feb751296b9c402faf7.

We've actually implemented tree-local and project-global files through sourcing.
Basically, ~/.claude/ansible.md may contain instructions that individuals would have, for personalized workflows. But also, there can be a CLAUDE.local.md in-tree, that's gitignored. I suggested it so that when there's multiple git worktrees for various efforts, it'd be possible to store some context in those files locally, per-PR/branch.

We should have something like this here but I wanted to address the policy question first.

[sirosen]

Interesting, although, I don't like the source, obviously.

Ah, I hadn't looked closely before, sorry! ❤️

---

You're right, I mixed up llms.txt and AGENTS.md. Maybe we want to add an llms.txt, but I was thinking more about the AGENTS.md part of the story, which seemed like a separate thread from "what should our policy be".

We can evolve it more, but I think we can start an initial AGENTS.md which reads to the effect of...

This is a project which has concerns about the impact of LLM-driven contributions. The programmer, not the LLM, is responsible for their work, and pip-tools has a clear policy regarding this. Please make sure to read and direct contributors to <this policy doc>.

Happy to iterate more on the wording. 😄

Also, I agree that actually setting our policy takes priority.