# Custom AI Provider

Build a custom AI-powered image generation provider for CE.SDK using the `@imgly/plugin-ai-generation-web` package. You’ll implement an AI provider from scratch using the schema-based approach and integrate it seamlessly with the editor.

![Custom AI Provider](/docs/cesdk/_astro/browser.hero.C9lGwbqx_h8JQQ.webp)

15 mins

estimated time

Live Demo[

Download](https://github.com/imgly/cesdk-web-examples/archive/refs/heads/main.zip)[

StackBlitz](https://stackblitz.com/~/github.com/imgly/cesdk-web-examples)[

GitHub](https://github.com/imgly/cesdk-web-examples)

This guide walks you through creating an image generation provider that connects to your own AI service. You’ll learn about the provider interface, OpenAPI schema-based input configuration, quick actions, middleware patterns, and CE.SDK integration.

```
import type { EditorPlugin, EditorPluginContext } from '@cesdk/cesdk-js';import ImageGeneration from '@imgly/plugin-ai-image-generation-web';import {  Provider,  ImageOutput,  loggingMiddleware,  uploadMiddleware,  CommonProviderConfiguration} from '@imgly/plugin-ai-generation-web';import type CreativeEditorSDK from '@cesdk/cesdk-js';import packageJson from './package.json';import apiSchema from './myApiSchema.json';
// Define your input type based on your schemainterface MyProviderInput {  prompt: string;  width: number;  height: number;  style: string;  image_url?: string; // For image-to-image operations}
// Define provider configuration interface extending CommonProviderConfigurationinterface MyProviderConfiguration extends CommonProviderConfiguration<  MyProviderInput,  ImageOutput> {  // Add any provider-specific configuration here  customApiKey?: string;}
// Mock API function that simulates image generation// In production, this would be replaced with actual API callsasync function mockGenerateImage(  input: MyProviderInput,  _abortSignal?: AbortSignal): Promise<{ imageUrl: string }> {  // Simulate network delay  await new Promise((resolve) => setTimeout(resolve, 1500));
  // Return real demo image URLs based on the style  const sampleImages: Record<string, string> = {    photorealistic:      'https://cdn.img.ly/assets/demo/v1/ly.img.image/images/sample_1.jpg',    cartoon:      'https://cdn.img.ly/assets/demo/v1/ly.img.image/images/sample_2.jpg',    sketch:      'https://cdn.img.ly/assets/demo/v1/ly.img.image/images/sample_3.jpg',    painting:      'https://cdn.img.ly/assets/demo/v1/ly.img.image/images/sample_4.jpg'  };
  return {    imageUrl: sampleImages[input.style] || sampleImages.photorealistic  };}
// Create a function that returns your providerexport function MyImageProvider(  _config: MyProviderConfiguration): (context: {  cesdk: CreativeEditorSDK;}) => Promise<Provider<'image', MyProviderInput, ImageOutput>> {  // Return a function that returns the provider  return async ({ cesdk: _cesdk }) => {    // Create and return the provider    const provider: Provider<'image', MyProviderInput, ImageOutput> = {      // Unique identifier for your provider      id: 'my-image-provider',
      // Define output type as 'image'      kind: 'image',
      // Initialize your provider      initialize: async () => {        console.log('Initializing my image provider');        // Any setup needed (e.g., API client initialization)      },
      // Define input panel and UI using schema      input: {        panel: {          type: 'schema',          document: apiSchema, // Your OpenAPI schema          inputReference: '#/components/schemas/GenerationInput', // Reference to your input schema          userFlow: 'placeholder', // Creates a block first, then updates it with the generated content          orderExtensionKeyword: 'x-order-properties', // Used to control property display order
          // Convert API input to block parameters          getBlockInput: async (input) => ({            image: {              width: input.width || 512,              height: input.height || 512,              label: `AI: ${input.prompt?.substring(0, 20)}...`            }          })        },
        // Add quick actions for canvas menu        quickActions: {          supported: {            // Map quick action IDs to provider input transformations            'ly.img.editImage': {              mapInput: (quickActionInput) => ({                prompt: quickActionInput.prompt,                image_url: quickActionInput.uri,                width: 512,                height: 512,                style: 'photorealistic'              })            },            'ly.img.swapBackground': {              mapInput: (quickActionInput) => ({                prompt: quickActionInput.prompt,                image_url: quickActionInput.uri,                width: 512,                height: 512,                style: 'photorealistic'              })            },            'ly.img.createVariant': {              mapInput: (quickActionInput) => ({                prompt: quickActionInput.prompt,                image_url: quickActionInput.uri,                width: 512,                height: 512,                style: 'photorealistic'              })            },            'ly.img.styleTransfer': {              mapInput: (quickActionInput) => ({                prompt: quickActionInput.style,                image_url: quickActionInput.uri,                width: 512,                height: 512,                style: 'photorealistic'              })            }          }        }      },
      // Define output generation behavior      output: {        // Allow cancellation of generation        abortable: true,
        // Store generated assets in browser's IndexedDB        history: '@imgly/indexedDB',
        // Add middleware for logging and uploading        middleware: [          loggingMiddleware({ enable: true }),          // Example of upload middleware that stores generated images on your server          uploadMiddleware(async (output: ImageOutput) => {            // In production, upload the image to your server            // For this example, we just return the output as-is            console.log('Upload middleware: Processing output', output.url);            return output;          }),          // Custom error handling middleware          async (input, options, next) => {            try {              return await next(input, options);            } catch (error: any) {              // Prevent default error notification              options.preventDefault();
              // Show custom error notification              options.cesdk?.ui.showNotification({                type: 'error',                message: `Image generation failed: ${error.message}`              });
              throw error;            }          }        ],
        // Configure success/error notifications        notification: {          success: {            show: true,            message: 'Image generated successfully!'          },          error: {            show: true,            message: (context) => `Generation failed: ${context.error}`          }        },
        // The core generation function        generate: async (input, { abortSignal }) => {          try {            // Use mock API for demonstration            // In production, replace with actual API call:            // const response = await fetch(config.proxyUrl, { ... });            const data = await mockGenerateImage(input, abortSignal);
            // Return the image URL            return {              kind: 'image',              url: data.imageUrl            };          } catch (error) {            console.error('Image generation failed:', error);            throw error;          }        }      }    };
    return provider;  };}
class Example implements EditorPlugin {  name = packageJson.name;  version = packageJson.version;
  async initialize({ cesdk }: EditorPluginContext): Promise<void> {    if (!cesdk) {      throw new Error('CE.SDK instance is required for this plugin');    }
    // Load asset sources    await cesdk.addDefaultAssetSources();    await cesdk.addDemoAssetSources({ sceneMode: 'Design' });
    // Create a design scene    await cesdk.createDesignScene();
    // Add translations for the custom provider    cesdk.i18n.setTranslations({      en: {        'panel.my-image-provider.generate': 'Generate Image'      }    });
    // Add your image generation provider    await cesdk.addPlugin(      ImageGeneration({        providers: {          text2image: MyImageProvider({            proxyUrl: 'https://your-proxy-server.com/api/proxy',            headers: {              'x-client-version': '1.0.0',              'x-request-source': 'cesdk-tutorial'            }          })        },        debug: true      })    );
    // Add the dock component to open the AI image generation panel    cesdk.ui.setDockOrder([      'ly.img.ai.image-generation.dock',      ...cesdk.ui.getDockOrder()    ]);
    // Open the AI Image Generation panel    cesdk.ui.openPanel('ly.img.ai.image-generation');  }}
// Example: Control which features are visible in the UIfunction configureFeatures(cesdk: CreativeEditorSDK) {  // Hide the provider dropdown if you only have one provider  cesdk.feature.enable(    'ly.img.plugin-ai-image-generation-web.providerSelect',    false  );  // Enable text-to-image generation  cesdk.feature.enable('ly.img.plugin-ai-image-generation-web.fromText', true);  // Disable image-to-image generation  cesdk.feature.enable('ly.img.plugin-ai-image-generation-web.fromImage', false);}
export default Example;
```

This guide covers:

*   Understanding the Provider interface for image generation
*   Creating an OpenAPI schema for input configuration
*   Implementing a custom image provider with quick actions
*   Adding middleware for logging, uploads, and error handling
*   Integrating the provider with CE.SDK

## Prerequisites[#](#prerequisites)

*   Basic knowledge of TypeScript and React
*   Familiarity with CreativeEditor SDK
*   An image generation API to integrate with

## Project Setup[#](#project-setup)

First, set up your project and install the necessary packages:

Terminal window

```