[RFC]: Add guidance concerning AI usage

[batpigandme]

Description

This RFC proposes we add guidelines regarding the usage of AI/LLMs in the project as a whole, and in making contributions to the codebase. This could appear in brief in CONTRIBUTING and point to a longer, more explicit policy, or simply exist as its own entity.

Guiding principles

1. Maintainer capacity is a limiting resource in open source. All contributions should respect the time and effort maintainers put into reviewing contributions.
2. Indiscriminate use of AI to generate code can lead to low-quality contributions that can impose significant maintainer burden.
3. This project is licensed under the Apache-2.0 License, and is subject to its terms--including the disclosure of any copyrighted materials included in submissisions. All contributions are subject to the Developer's Ceritifcate of Origin already in our contributing guidelines.
4. Regardless of whether or not AI is used, contributors should should understand and be able to explain the code they are submitting.
5. There are beneficial, permissible uses of AI in contributing to this project. For example, use AI to do R&D, learn to navigate the codebase, or suggest potential improvements to your code (including things like spelling and adherence to project guidelines).
6. If contributors do use AI to assist in their contribution, they must disclose this in their pull request or issue. This should include a description of how AI was used and what specific contributions it made.
7. The legal landscape around AI-generated content is still evolving, and (6) will allow the project to adapt to future changes.
8. Using AI to resolve issues marked as good first issue is not permitted (i.e. we want you to build a mental model of the codebase yourself, and get familiar with the mechanics of how the process works).

Short-version polcy

Adapted from Matplotlib's Restriction on Generative AI Usage, and not inclusive of all the items listed in guiding principles:

We expect authentic engagement in our community.

• Do not post output from Large Language Models or similar generative AI as comments on GitHub or Zulip, as such comments tend to be formulaic and low-value.

• If you use generative AI tools as an aid in developing code or documentation changes, ensure that you fully understand the proposed changes and can explain why they are the correct approach, and disclose how AI was used in your pull request.

Make sure you have added value based on your personal competency to your contributions. Just taking some input, feeding it to an AI and posting the result is not of value to the project, or your growth as a developer.

Inspired by/useful resources

• ASF Generative Tooling Guidance
• Fedora Council's AI-Assisted Contribution Policy
• scikit-learn Automated Contributions Policy
• OpenInfra Foundation Policy for AI Generated Content
• Linux Foundation Guidance Regarding Use of Generative AI Tools for Open Source Software Development

Related Issues

No response

Questions

No. I'd like feedback (of course), and will draft a PR with a more substantive policy, but thought the issue would be a more useful place for preserving the broader discussion.

Other

No.

Checklist

• I have read and understood the Code of Conduct.
• Searched for existing issues and pull requests.
• The issue name begins with RFC:.

[stdlib-bot]

👋 Hi there! 👋

And thank you for opening your first issue! We will get back to you shortly. 🏃 💨

If you have any further questions while waiting for a response, please join our Zulip community to chat with project maintainers and other community members.

[Ayush2k02]

Would it be helpful to include specific examples in CONTRIBUTING.md ?

[kgryte]

Regardless of whether or not AI is used, contributors should should understand and be able to explain the code they are submitting.

This is a key point. Especially because of AI's tendency to over-engineer and over-complicate, the expectation should be, IMO, that one should be prepared to defend and explain any chosen algorithm/architecture decisions, independent of AI. If one cannot explain why a particular design choice was made, then that it is a good candidate of a contribution which should not be accepted.

Another point that we should make, in addition to affirmative examples of how to use AI, may be to include examples of how not to use AI. For example, I just reviewed a PR where AI had been used for tests. In an attempt to ensure the tests passed, the tests were written such that they always passed regardless of the actual test result (the equivalent of t.ok( true )). In general, test cases, while sometimes tedious, are a good example of something which should be written by hand, and, in those scenarios where AI is used to help with, say, fixture generation, all generated output should be meticulously reviewed.

Tests are a primary way that contributions are reviewed. And if the tests are poor quality, reviewers are forced to think twice as hard about the validity of an implementation.