Add AI policy to README#280
Conversation
Added a section on AI policy regarding contributions.
suprstarrd
left a comment
suprstarrd
left a comment
There was a problem hiding this comment.
We should discuss what "near-fully" AI-generated code would be - see #236
|
Makes sense. I changed this pull request to a draft so it can be analyzed and revised further. |
|
|
||
| ## AI (Artificial Intelligence) Policy | ||
|
|
||
| As a community largely composed of passionate developers and reverse engineers, we've seen the rise of AI impact the homebrew scene. We would prefer if fully or near-fully AI generated code and/or projects be kept away from our project. However, you are welcome to use a minor amount of AI "assistance" provided you are able to maintain your contributions. |
There was a problem hiding this comment.
Largely NH staff.
There was a problem hiding this comment.
It says developers and reverse engineers. Since this also affects Wii U, how many developers and reverse engineers there share this this opinion?
There was a problem hiding this comment.
Who is "we"? [...] It says developers and reverse engineers.
From my understanding:
- "We" refers to most of the community staff and most of main guide contributors (clarity for the uninformed: both @jan-hofmeier and @TechnoEggnog-2 are on the CRC - acronym for "community resource collaborators" - which makes them part of the latter group. I am not but I made the issue that sort of sparked this). If this gets merged, I assume that this would be their group position and would likely extend to the other Nintendo Homebrew guides.
- "Developers and reverse engineers" refers to the people in the community that the staff manages.
Ideally, all three groups (community staff, guide contributors, and community members) would have opinions in sync, but of course that doesn't happen in a community every single time. So, leading into the next section:
how many [Wii U] developers and reverse engineers there share this this opinion?
I personally think this is a very valid question. I also think it would be valid to ask this about the end-users who will be using the guide as a separate demographic to additionally weigh.
...Is it time for yet another survey app..? Maybe a series of them, since this policy would also affect the Wii and 3DS?
There was a problem hiding this comment.
Surveying would probably be a good idea to collect a larger sample size, considering my brief research for #236 was just in a small discord server.
There was a problem hiding this comment.
The first question is kind of pointless. No one says they only want code from AI.
The relevant question is: Do you mind? And if so are you willing take the consequences (having less choice of homebrew).
And with the default no and only asked to draw a line if they said yes, that would mean all the no Sayers would vote against having aroma in the guide. I am not sure if you want that.
If you wanted an unbiased answer you would just ask in every case where they draw the line.
And if you want them to make a informed instead of a feelings based decision, you should also inform about the consequences for each selection if you give selections for drawing the line.
There was a problem hiding this comment.
This implies they get to decide how it is developed. But that's not the case.
I don't believe that is the intention of the surveys. The intention of the survey this question is for is so that the guide authors to understand how SPECIFICALLY users feel about AI in homebrew. I imagine there will be a separate survey for the guide authors to understand how SPECIFICALLY developers and reverse engineers feel about AI in homebrew - or they can share here, I guess.
I don't think you understood what I meant. The question I quoted there implied to the users they decide how the homebrew gets developed with their answer.
So to the user it sound like: do you rather have that homebrew developed with AI or without it.
But thats not the case. They are deciding about the policy which decides what gets included, which is significant different.
With homebrew it's take it or leave it. You don't get to make demands to the developer.
There was a problem hiding this comment.
The first question is kind of pointless. No one says they only want code from AI.
This is not what I meant.
Copy and pasting my message from Discord:
To make my own stance personally very clear: I do not like AI. Very deeply. I hate it and would prefer to never see it if I can help it. But I also recognize that lots of applications have it or use it, and I can't help that. I willingly make an effort to avoid AI and seek alternatives where possible, but I do recognize I can't always avoid it so I have formed a tolerance to it when I require other things that depend on something else that uses AI in some part of the process. That's why I think it would be good to ask those 3 questions instead of an all-encompassing one
So in this case, I'd prefer not to but do tolerate it. Some people may be fine / like the idea of AI helping make new homebrew apps faster, and they would say "yes" to that question. Hope that makes things clearer; we can probably clarify that for the end-user.
And with the default no [...]
The default would be "no opinion / undecided / I don't know". Having a default "yes" or "no" either way is a bias.
If you wanted an unbiased answer you would just ask in every case where they draw the line.
Answering that prior question partly specifies where you draw the line. Answering "no" would mean they draw the line at having any AI in homebrew at all.
And if you want them to make a informed instead of a feelings based decision, you should also inform about the consequences for each selection if you give selections for drawing the line.
The user is expected to have some knowledge of AI if they change from "undecided" to "no" or "yes". Again, you are correct that they do not get to decide how future homebrew is developed, but this is specifically for the guide authors to understand what these demographics think about in these cases. Sharing the consequences is irrelevant because this doesn't actually affect their ability to get said homebrew, and maybe not even if they're informed about what homebrew apps do have AI assistance.
I don't think you understood what I meant. The question I quoted there implied to the users they decide how the homebrew gets developed with their answer.
So to the user it sound like: do you rather have that homebrew developed with AI or without it.But thats not the case. They are deciding about the policy which decides what gets included, which is significant different.
This is why I specified: "And explain that this information is being used to draft a policy around AI tools in the guides, and to help developers understand expectations of what users want. Would also be very useful for the app stores too." I can concede that not all (or even any) homebrew developers will take that feedback in mind when making apps in the future, and that can be clarified.
There was a problem hiding this comment.
So in this case, I'd prefer not to but do tolerate it. Some people may be fine / like the idea of AI helping make new homebrew apps faster, and they would say "yes" to that question. Hope that makes things clearer; we can probably clarify that for the end-user.
That's not how I would understand the question and I don't think that's how users would read it and I am not sure if it is even relevant to this.
And with the default no [...]
The default would be "no opinion / undecided / I don't know". Having a default "yes" or "no" either way is a bias.
I mean implied default. If you say there is something to tolerate it kind of implies that the neutral position would be no. (since if people are unsure they usually don't take any action, and in that case not doing means not tolerating.
If you wanted an unbiased answer you would just ask in every case where they draw the line.
Answering that prior question partly specifies where you draw the line. Answering "no" would mean they draw the line at having any AI in homebrew at all.
No it forces the user to make up their mind on that binary question, before they know there are more nuanced options. And once you get someone to make up their mind they will usually keep their position.
This is why I specified: "And explain that this information is being used to draft a policy around AI tools in the guides, and to help developers understand expectations of what users want. Would also be very useful for the app stores too." I can concede that not all (or even any) homebrew developers will take that feedback in mind when making apps in the future, and that can be clarified.
No you can't give any statement beyond the guide here. Not even "might" as that creates some expectation.
I think the most honest answer would be to ask:
to what extend are you ok with the guide including homebrew with AI generated code.
I don't care | I don't care as long as it is tested / proven | only if it was reviewed | function level | remote ai based auto complete | local ai auto complete | non ai auto complete | nothing at all, just notepad
Also notice I included pretty extreme options at both ends. People without a very strong opinion tend to stay away from the fringes. Having buffer answers there causes less skewing on the relevant answers.
If you want the users to be informed, you would include examples of homebrew using (openly) AI assist to that degree. Most users didn't knew that Aroma already included AI code, so I question this expectation:
The user is expected to have some knowledge of AI if they change from "undecided" to "no" or "yes".
With that expectation you exclude much of the "I don't care" group. Also the AI haters here are pretty loud, without having much knowledge.
And this also brings me to another relevant point: depending on how you Title / invite people to the survey you will also cause a bias by this first filter. I would expect a large group of don't care to varying degrees, these will be unlikely to click on it if it is marketed as a survey about AI. Then you have a group which is very passionate about hating AI, they will definitely use it. I don't expect there to be many if any at all people who passionate about that others have to use it. And if it is about introducing a ban you have to count the don't care against the ban. Either someone cares enough about banning something or they don't. The default on ban / allow should always be allowed or at least the current state (which is allowed atm).
A more generic title like "Vote on what homebrew should be allowed on the guides" would help with that bias.
Sharing the consequences is irrelevant because this doesn't actually affect their ability to get said homebrew, and maybe not even if they're informed about what homebrew apps do have AI assistance.
I would argue it is a consequence what the guide includes. If it was of no consequence, then we wouldn't need the guide at all.
There was a problem hiding this comment.
Further discussion on the survey is happening on the Nintendo Homebrew Discord server in #ai-contribution-discussion
|
|
||
| ## AI (Artificial Intelligence) Policy | ||
|
|
||
| As a community largely composed of passionate developers and reverse engineers, we've seen the rise of AI impact the homebrew scene. We would prefer if fully or near-fully AI generated code and/or projects be kept away from our project. However, you are welcome to use a minor amount of AI "assistance" provided you are able to maintain your contributions. |
There was a problem hiding this comment.
Also what is the impact you have seen on the homebrew scene? Did anything bad happen to the scene because of the usage of such tools?
There was a problem hiding this comment.
Have you… not seen how god awful AI assistance has been?
There was a problem hiding this comment.
This is about "code". An no I don't have seen negative impact of AI code on the homebrew scene. Can you provide me with an example?
|
Further discussion on how this policy may form is happening on the Nintendo Homebrew Discord server in #ai-contribution-discussion |
3 participants
Added a section on AI policy regarding contributions.