# Image Alt Textify

> Generate alt text for images in markdown files

Alt text is essential for making images accessible to everyone, including those with visual impairments. It provides a textual description of the image, allowing screen readers to convey the content to users who cannot see the image. However, writing alt text for images can be time-consuming, especially when dealing with a large number of images. This is where AI can help. By using a language model like OpenAI’s GPT-4, you can generate alt text for images automatically, thus saving time and effort.

In this sample, we will build a tool that generates alt text for images in Markdown files.

## Configuring the script

[Section titled “Configuring the script”](#configuring-the-script)

This script is composed of TypeScript code, designed to run with the GenAIScript CLI. Let’s break it down:

```ts
script({
    title: "Image Alt Textify",
    description: "Generate alt text for images in markdown files",
    accept: "none",
    parameters: {
        docs: {
            type: "string",
            description: "path to search for markdown files",
            default: "**.{md,mdx}",
        },
        force: {
            type: "boolean",
            description: "regenerate all descriptions",
            default: false,
        },
        assets: {
            type: "string",
            description: "image assets path",
            // change the default path to your assets folder
            default: "./assets/images",
        },
    },
})
```

Here we declare the script with a title and description, specifying it uses OpenAI’s GPT-4 model. We also set parameters for the file paths, the option to regenerate all descriptions, and the assets path.

Next, we extract environmental variables:

```ts
const { docs, force, assets } = env.vars
```

## Searching for images

[Section titled “Searching for images”](#searching-for-images)

Following that, we define a regular expression to find images in Markdown:

```ts
const rx = force
    ? // match ![alt?](path) with alt text or not
      /!\[[^\]]*\]\(([^\)]+\.(png|jpg))\)/g
    : // match ![](path) without alt text
      /!\[\s*\]\(([^\)]+\.(png|jpg))\)/g


const { files } = await workspace.grep(rx, {
    path: docs,
    glob: "*.{md,mdx}",
    readText: true,
})
```

The script uses [workspace.grep](/genaiscript/reference/scripts/files#grep) to find all occurrences of the regex pattern in the specified documents.

## Generating alt text

[Section titled “Generating alt text”](#generating-alt-text)

For each image URL found, we generate alt text using an [inline prompt](/genaiscript/reference/scripts/inline-prompts) and [defImages](/genaiscript/reference/scripts/images).

```ts
for (const file of files) {
    const { filename, content } = file


    // map documentation relative path to assets path
    const url = resolveUrl(filename)


    // execute a LLM query to generate alt text
    const { text } = await runPrompt(
        (_) => {
            _.defImages(resolvedUrl)
            _.$`
                You are an expert in assistive technology.


                You will analyze the image
                and generate a description alt text for the image.


                - Do not include alt text in the description.
                - Keep it short but descriptive.
                - Do not generate the [ character.`
        },
        {
            // safety system message to prevent generating harmful text
            system: ["system.safety_harmful_content"],
            // use multi-model model
            model: "openai:gpt-4o",
            ...
        }
    )


    imgs[url] = text
}
```

## Updating files

[Section titled “Updating files”](#updating-files)

Finally, we update the Markdown content with the generated alt text:

```ts
const newContent = content.replace(
    rx,
    (m, url) => `![${imgs[url] ?? ""}](${url})`
)
if (newContent !== content) await workspace.writeText(filename, newContent)
```

We replace the placeholder in the original content with the alt text and save the updated file.

## 💻 How to Run the Script

[Section titled “💻 How to Run the Script”](#-how-to-run-the-script)

To run this script, you’ll need the GenAIScript CLI. If you haven’t installed it yet, check out the [installation guide](https://microsoft.github.io/genaiscript/getting-started/installation).

Once you have the CLI, you can run the script with the following command:

```bash
npx genaiscript run iat
```

## Automate with GitHub Actions and GitHub Models

[Section titled “Automate with GitHub Actions and GitHub Models”](#automate-with-github-actions-and-github-models)

The following GitHub Action automates the process of generating alt text for images in Markdown files. It runs on every push to the `dev` branch and uses the `genaiscript` CLI to execute the script. It uses the OpenAI `gpt-4.1` model through GitHub Models for inference.

.github/workflows/genai-iat.yml

```yml
name: genai iat
on:
    workflow_dispatch:
    push:
        branches:
            - dev
        paths:
            - "**.md*"
concurrency:
    group: iat-${{ github.workflow }}-${{ github.ref }}
    cancel-in-progress: true
permissions:
    pull-requests: write
    models: read
    contents: write
jobs:
    generate-alt-text:
        runs-on: ubuntu-latest
        steps:
            - uses: actions/checkout@v4
            - uses: actions/setup-node@v4
            - name: genaiscript
              run: npx --yes genaiscript run iat -m github:openai/gpt-4.1 --out-trace $GITHUB_STEP_SUMMARY
              env:
                  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
            - uses: stefanzweifel/git-auto-commit-action@v5
              with:
                  commit_message: "[genai] image alt text"
                  commit_user_name: "genaiscript"
```

## Full source ([GitHub](https://github.com/microsoft/genaiscript/blob/main/samples/sample/genaisrc/samples/iat.genai.mts))

[Section titled “Full source (GitHub)”](#full-source-github)

iat.genai.mts

```ts
/*
 * Markdown image alt text updater
 */
script({
  title: "Image Alt Textify",
  description: "Generate alt text for images in markdown files",
  accept: "none",
  unlisted: true,
  parameters: {
    docs: {
      type: "string",
      description: "path to search for markdown files",
      default: "docs",
    },
    force: {
      type: "boolean",
      description: "regenerate all descriptions",
      default: false,
    },
    assets: {
      type: "string",
      description: "image assets path",
      default: "./slides/public",
    },
    dryRun: {
      type: "boolean",
      description: "show matches, do not compute alt text",
      default: false,
    },
  },
});


/** ------------------------------------------------
 * Configuration
 */
const { docs, force, assets, dryRun } = env.vars;


/** ------------------------------------------------
 * Helper functions (update as needed)
 */
/**
 *  Return the resolved url for the image
 */
const resolveUrl = (filename: string, url: string) => {
  // github assets ok
  if (/^https:\/\/github.com\/user-attachments\/assets\//i.test(url)) return url;
  if (url?.startsWith("https://raw.githubusercontent.com/")) return url;
  // ignore external urls
  if (/^http?s:\/\//i.test(url)) return undefined;
  // map / to assets
  else if (/^\//.test(url)) return path.join(assets, url.slice(1));
  // resolve local paths
  else return path.join(path.dirname(filename), url);
};


/** ------------------------------------------------
 * Collect files
 */
// search for ![](...) in markdown files and generate alt text for images
const rx = force // upgrade all urls
  ? // match ![alt](url) with any alt
    /!\[[^\]]*\]\(([^\)]+)\)/g
  : // match ![alt](url) where alt is empty
    /!\[\s*\]\(([^\)]+)\)/g;
console.log(`Searching for ${rx} in ${docs}`);
const { files } = await workspace.grep(rx, {
  path: docs,
  glob: "**/*.md*",
  readText: true,
});


/** ------------------------------------------------
 * Generate alt text for images
 * and update markdown files
 */


// a cache of generated alt text for images
const imgs: Record<string, string> = {};


// process each file
for (const file of files) {
  const { filename, content } = file;
  console.log(filename);
  const matches = Array.from(content.matchAll(rx));
  console.log(`.. found ${matches.length} matches`);
  // pre-compute matches
  for (const match of matches) {
    const url = match[1];
    if (imgs[url]) continue; // already processed
    console.log(`.. processing ${url}`);


    const resolvedUrl = resolveUrl(filename, url);
    if (!resolvedUrl) {
      console.log(`... unknown image url`);
      continue; // can't process url
    }
    console.log(`└─ ${resolvedUrl}`);


    if (dryRun) continue;


    // execute a LLM query to generate alt text
    const { text, error } = await runPrompt(
      (_) => {
        _.defImages(resolvedUrl);
        _.$`
                You are an expert in assistive technology.


                You will analyze the image
                and generate a description alt text for the image.


                - Do not include alt text in the description.
                - Keep it short but descriptive.
                - Do not generate the [ character.`;
      },
      {
        // safety system message to prevent generating harmful text
        system: [
          "system.assistant",
          "system.safety_jailbreak",
          "system.safety_harmful_content",
          "system.safety_validate_harmful_content",
        ],
        maxTokens: 4000,
        temperature: 0.5,
        cache: "alt-text",
        label: `alt-textify ${resolvedUrl}`,
      },
    );
    if (error) throw error;
    else if (!text) console.log(`.. no description generated`);
    else imgs[url] = text.replace(/\[/g, ""); // remove [ from alt text
  }
  // apply replacements
  const newContent = content.replace(rx, (m, url) => `![${imgs[url] ?? ""}](${url})`);
  // save updated content
  if (newContent !== content) {
    console.log(`.. updating ${filename}`);
    await workspace.writeText(filename, newContent);
  }
}
```

## Content Safety

[Section titled “Content Safety”](#content-safety)

The following measures are taken to ensure the safety of the generated content.

* This script includes system prompts to prevent prompt injection and harmful content generation.

  * [system.safety\_jailbreak](/genaiscript/reference/scripts/system#systemsafety_jailbreak)
  * [system.safety\_harmful\_content](/genaiscript/reference/scripts/system#systemsafety_harmful_content)

* The generated description is saved to a file at a specific path, which allows for manual review before committing the changes.

Additional measures to further enhance safety would be to run [a model with a safety filter](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new) or validate the message with a [content safety service](/genaiscript/reference/scripts/content-safety).

Refer to the [Transparency Note](/genaiscript/reference/transparency-note/) for more information on content safety.