# Commenter

> Adds comments to your code

This sample automates the process of adding comments to source code using an LLM and validates that the changes haven’t introduced any code modifications.

To achieve this, we could use a combination of tools to validate the transformation: source formatters, compilers, linters, or an LLM-as-judge.

The algorithm can be summarized as follows:

```txt
for each file of files
    // generate
    add comments using GenAI


    // validate validate validate!
    format generated code (optional) -- keep things consistent
    build generated -- let's make sure it's still valid code
    check that only comments were changed -- LLM as a judge


// and more validate
final human code review
```

Let’s get started with analyzing the script.

### Getting Files to Process

[Section titled “Getting Files to Process”](#getting-files-to-process)

The user can select which files to comment on or, if none are selected, we’ll use Git to find all modified files.

```ts
let files = env.files
if (files.length === 0)
    // no files selected, use git to find modified files
    files = await ..."git status --porcelain"... // details in sources
```

### Processing Each File

[Section titled “Processing Each File”](#processing-each-file)

We process each file separately to avoid overwhelming the token context and to keep the AI focused. We can use [inline prompts](/genaiscript/reference/scripts/inline-prompts) to make inner queries.

```ts
for (const file of files) {
    ... add comments
    ... format generated code (optional) -- keep things consistent
    ... build generated -- let's make sure it's still valid code
    ... check that only comments were changed -- LLM as judge
    ... save changes
}
```

### The Prompt for Adding Comments

[Section titled “The Prompt for Adding Comments”](#the-prompt-for-adding-comments)

Within the `addComments` function, we prompt GenAI to add comments. We do this twice to increase the likelihood of generating useful comments, as the LLM might have been less effective on the first pass.

```ts
const res = await runPrompt(
    (ctx) => {
        ctx.$`You can add comments to this code...` // prompt details in sources
    },
    { system: ["system", "system.files"] }
)
```

We provide a detailed set of instructions to the AI on how to analyze and comment on the code.

### Format, build, lint

[Section titled “Format, build, lint”](#format-build-lint)

At this point, we have source code modified by an LLM. We should try to use all available tools to validate the changes. It is best to start with formatters and compilers, as they are deterministic and typically fast.

### Judge results with LLM

[Section titled “Judge results with LLM”](#judge-results-with-llm)

We issue one more prompt to judge the modified code (`git diff`) and make sure the code is not modified.

```ts
async function checkModifications(filename: string): Promise<boolean> {
    const diff = await host.exec(`git diff ${filename}`)
    if (!diff.stdout) return false
    const res = await runPrompt(
        (ctx) => {
            ctx.def("DIFF", diff.stdout)
            ctx.$`You are an expert developer at all programming languages.


        Your task is to analyze the changes in DIFF and make sure that only comments are modified.
        Report all changes that are not comments and print "<MODIFIED>".
        `
        },
        {
            cache: "cmt-check",
        }
    )
    return res.text?.includes("<MODIFIED>")
}
```

## How to Run the Script

[Section titled “How to Run the Script”](#how-to-run-the-script)

To run this script, you’ll first need to install the GenAIScript CLI. [Follow the installation guide here](https://microsoft.github.io/genaiscript/getting-started/installation).

```sh
genaiscript run cmt
```

## Format and build

[Section titled “Format and build”](#format-and-build)

One important aspect is to normalize and validate the AI-generated code. The user can provide a `format` command to run a formatter and a `build` command to check if the code is still valid.

```ts
script({...,
    parameters: {
        format: {
            type: "string",
            description: "Format source code command",
        },
        build: {
            type: "string",
            description: "Build command",
        },
    },
})


const { format, build } = env.vars.build
```

```sh
genaiscript run cmt --vars "build=npm run build" "format=npm run format"
```

## Full source ([GitHub](https://github.com/microsoft/genaiscript/blob/main/samples/sample/genaisrc/samples/cmt.genai.mts))

[Section titled “Full source (GitHub)”](#full-source-github)

cmt.genai.mts

```ts
script({
  title: "Source Code Comment Generator",
  description: `Add comments to source code to make it more understandable for AI systems or human developers.`,
  parameters: {
    format: {
      type: "string",
      description: "Format source code command",
    },
    build: {
      type: "string",
      description: "Build command",
    },
  },
});


const { format, build } = env.vars;


// Get files from environment or modified files from Git if none provided
let files = env.files;
if (!files.length) files = await git.listFiles("staged", { askStageOnEmpty: true });
if (!files.length) files = await git.listFiles("modified-base");


// custom filter to only process code files
files = files.filter(
  ({ filename }) =>
    /\.(py|m?ts|m?js|cs|java|c|cpp|h|hpp)$/.test(filename) && // known languages only
    !/\.test/.test(filename), // ignore test files
);


// Shuffle files
files = files.sort(() => Math.random() - 0.5);


console.log(YAML.stringify(files.map((f) => f.filename)));


// Process each file separately to avoid context explosion
const jobs = host.promiseQueue(5);
await jobs.mapAll(files, processFile);


async function processFile(file: WorkspaceFile) {
  console.log(`processing ${file.filename}`);
  if (!file.content) console.log(`empty file, continue`);
  try {
    const newContent = await addComments(file);
    // Save modified content if different
    if (newContent && file.content !== newContent) {
      console.log(`updating ${file.filename}`);
      await workspace.writeText(file.filename, newContent);
      let revert = false;
      // try formatting
      if (format) {
        const formatRes = await host.exec(`${format} ${file.filename}`);
        if (formatRes.exitCode !== 0) {
          revert = true;
        }
      }
      // try building
      if (!revert && build) {
        const buildRes = await host.exec(`${build} ${file.filename}`);
        if (buildRes.exitCode !== 0) {
          revert = true;
        }
      }
      // last LLM as judge check
      if (!revert) revert = await checkModifications(file.filename);


      // revert
      if (revert) {
        console.error(`reverting ${file.filename}...`);
        await workspace.writeText(file.filename, file.content);
      }
    }
  } catch (e) {
    console.error(`error: ${e}`);
  }
}


// Function to add comments to code
async function addComments(file: WorkspaceFile): Promise<string | undefined> {
  let { filename, content } = file;
  if ((await tokenizers.count(file.content)) > 20000) return undefined; // too big


  const res = await runPrompt(
    (ctx) => {
      // Define code snippet for AI context with line numbers
      const code = ctx.def("CODE", { filename, content }, { lineNumbers: false });


      // AI prompt to add comments for better understanding
      ctx.def("FILE", code, { detectPromptInjection: "available" });
      ctx.$`You are an expert developer at all programming languages.


You are tasked with adding comments to code in FILE to make it more understandable for AI systems or human developers.
You should analyze it, and add/update appropriate comments as needed.


To add or update comments to this code, follow these steps:


1. Analyze the code to understand its structure and functionality.
- If you are not familiar with the programming language, emit an empty file.
- If there is no code, emit an empty file.
2. Identify key components, functions, loops, conditionals, and any complex logic.
3. Add comments that explain:
- The purpose of functions or code blocks using the best comment format for that programming language.
- How complex algorithms or logic work
- Any assumptions or limitations in the code
- The meaning of important variables or data structures
- Any potential edge cases or error handling
- All function arguments and return value
- A Top level file comment that describes the code in the file


When adding or updating comments, follow these guidelines:


- Use clear and concise language
- Avoid stating the obvious (e.g., don't just restate what the code does)
- Focus on the "why" and "how" rather than just the "what"
- Use single-line comments for brief explanations
- Use multi-line comments for longer explanations or function/class descriptions
- Always place comments above the code they refer to.
- If comments already exist, review and update them as needed.
- Minimize changes to existing comments.
- For TypeScript functions, classes and fields, use JSDoc comments. do NOT add type annotations in comments.
- For Python functions and classes, use docstrings.
- do NOT modify comments with TODOs.
- do NOT modify comments with URLs or links as they are reference to external resources.
- do NOT add comments to imports


Your output should be the original code with your added comments. Make sure to preserve ALL the original code's formatting and structure. DO NOT BE LAZY.


Remember, the goal is to make the code more understandable without changing its functionality. DO NOT MODIFY THE CODE ITSELF.
Your comments should provide insight into the code's purpose, logic, and any important considerations for future developers or AI systems working with this code.
`;
    },
    {
      system: [
        "system.assistant",
        "system.safety_jailbreak",
        "system.safety_harmful_content",
        "system.safety_validate_harmful_content",
      ],
      label: `comment ${filename}`,
    },
  );
  const { text, fences } = res;
  const newContent = fences?.[0]?.content ?? text;
  return newContent;
}


async function checkModifications(filename: string): Promise<boolean> {
  const diff = await git.diff({ paths: filename });
  if (!diff) return false;
  const res = await runPrompt(
    (ctx) => {
      ctx.def("DIFF", diff, { language: "diff" });
      ctx.$`You are an expert developer at all programming languages.


        Your task is to analyze the changes in DIFF and make sure that only comments are modified.
        Report all changes that are not comments or spacing and print <MOD>;
        otherwise, print <NO_MOD>.
        `;
    },
    {
      system: ["system.assistant", "system.safety_jailbreak"],
      cache: "cmt-check",
      label: `check comments in ${filename}`,
    },
  );


  const modified = res.text?.includes("<MOD>") || !res.text?.includes("<NO_MOD>");
  return modified;
}
```

## Content Safety

[Section titled “Content Safety”](#content-safety)

The following measures are taken to ensure the safety of the generated content:

* This script includes system prompts to prevent prompt injection and harmful content generation.

  * [system.safety\_jailbreak](/genaiscript/reference/scripts/system#systemsafety_jailbreak)
  * [system.safety\_harmful\_content](/genaiscript/reference/scripts/system#systemsafety_harmful_content)

* The generated description is saved to a file at a specific path, which allows for a manual review before committing the changes.

Additional measures to further enhance safety include running [a model with a safety filter](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new) or validating the message with a [content safety service](/genaiscript/reference/scripts/content-safety).

Refer to the [Transparency Note](/genaiscript/reference/transparency-note/) for more information on content safety.