Project Documentation: Useful Instructions, Not Generated Noise
Use AI to draft evidence-backed readmes, release notes, ownership rules, and process diagrams that reviewers can verify.
Write the handover you would want at 2 a.m.
A hotel handover book does not repeat every conversation from the previous shift. It tells the next person what changed, what needs attention, and where the evidence is. Project documentation should do the same.
AI is useful at collecting and reshaping known material. It is less trustworthy when asked to fill missing operational facts. “Write a deployment guide” can produce a plausible command that has never worked. “Draft a guide from this reviewed pipeline and identify missing prerequisites” is a safer starting point.
| Artifact | Reader’s question | Source to verify |
|---|---|---|
| README | How do I build, test, and start this project? | Scripts, manifests, tested setup |
| Wiki | How does this system work and who operates it? | Architecture decisions and runbooks |
| Changelog | What changed between versions? | Merged work and version history |
| Release notes | What does this release mean for users? | Tagged release and known limitations |
| License | What use and redistribution are permitted? | Approved license decision |
| CODEOWNERS | Who should review particular paths? | Maintainer responsibilities and access |
| Process diagram | Where does work or data move? | Actual components and boundaries |
A release is based on a Git tag, but a tag alone is not a deployed service. Link release notes to deployment evidence when describing what is available to customers.
Give the assistant a bounded writing contract
For a fictional booking API, a useful request is:
Draft a README update from package.json, the reviewed build workflow,
and the existing operations runbook.
Include prerequisites, build/test commands, configuration names
without secret values, and a troubleshooting section.
Do not invent support contacts, deployment regions, or successful tests.
Mark anything not supported by these files as an open question.
Return a proposed diff for human review; do not publish it.Read the diff against the real files. Check each command in a disposable environment before calling the procedure tested. Avoid copying production environment files into the assistant’s context. A configuration key name can explain a prerequisite without disclosing its value.
Treat issue descriptions, repository text, and fetched web pages as untrusted input. A line inside a document telling the assistant to ignore policy is content to evaluate, not an instruction that should override the writing contract.
Make ownership enforceable where appropriate
A minimal illustrative .github/CODEOWNERS file could be:
* @example-org/maintainers
/infra/ @example-org/platform
/docs/ @example-org/docsReplace these example teams with real teams that have the required repository access. GitHub uses the CODEOWNERS file from the PR’s base branch. It searches .github, repository root, and docs in that order, using the first file it finds. Later matching patterns take precedence; do not assume every matching line accumulates owners.
CODEOWNERS requests reviews; it does not by itself require a code owner’s approval before merge. Configure the relevant branch protection or ruleset requirement if that is the policy. Verify the rule applies to the destination branch and inspect who can bypass it.
An AI reviewer can help detect stale paths and typos, but it is not automatically an approving human code owner. Keep accountability with named maintainers.
Document a process without pretending it exists
Use a simple flow description before adding decoration:
Request -> Work item -> Draft PR -> CI checks -> Review
-> Approved artifact -> Test deployment -> Production rollout
-> Health evidence -> Feedback to work itemNow annotate the boundaries: who approves production, which artifact crosses environments, and where a failure returns the work. If the current pipeline has no staging environment, do not draw one as though it does. Label a proposed future state separately.
For release notes, summarize user-visible changes, migration needs, and known issues. Compare the selected tag with the previous release, not with a moving default branch that already contains later work. Generated release notes are a draft, not proof that every listed change reached production.
Licensing needs a deliberate maintainer decision. An assistant should not choose a license merely because another repository uses it. Do not add a license or remove existing notices without the appropriate authorization.
A repository has CODEOWNERS, but a PR can merge without owner approval. What is the missing control?
Try it: review an AI-written handover
Prerequisites: a disposable repository, a small project you can build locally, and an approved AI assistant if available. No paid Azure resources are needed.
- Give the assistant only the test/build scripts and a short description of the project’s current behavior.
- Request a README diff using the bounded contract above.
- Execute the proposed non-destructive local commands. Mark untested commands explicitly.
- Add a path ownership example using a real sandbox team, then inspect a draft PR to see whether a review is requested.
- Compare the README’s claimed workflow with the real workflow file.
Expected observation: every stated command has a source or a recorded test; unresolved facts remain questions. A requested owner review is distinguishable from an enforced merge requirement.
Cleanup: close the draft PR and discard only your sandbox documentation changes. Do not publish generated license decisions or fabricated test results.
Decision check: is the README still accurate after a pipeline change?
Review documentation in the same PR as the behavior change. A separate reminder to update it later is easy to lose. Test the smallest setup path a new contributor would actually follow.