Unlock Creative Potential: Integrate Image Generation with fofr/sdxl-dots Cognitive Actions

23 Apr 2025
Unlock Creative Potential: Integrate Image Generation with fofr/sdxl-dots Cognitive Actions

In the world of digital creativity, the ability to generate unique images can enhance applications across various domains, from art to marketing. The fofr/sdxl-dots spec offers a powerful Cognitive Action that enables developers to create images using advanced inpainting techniques. This feature not only allows for the preservation of certain areas of an image but also provides extensive customization options to tailor the output to specific needs. In this article, we'll dive into how to effectively use the Generate Image with Inpainting action, detailing its inputs and outputs, along with a conceptual implementation guide.

Prerequisites

Before you get started, ensure you have:

  • An API key for the Cognitive Actions platform.
  • Basic familiarity with making API calls and handling JSON data.

In general, authentication will involve passing the API key in the request headers when calling the Cognitive Actions endpoint.

Cognitive Actions Overview

Generate Image with Inpainting

The Generate Image with Inpainting action is designed to create images by inpainting selected areas based on specified prompts and configurations. This method is particularly useful for applications requiring creative freedom, allowing the generation of images that blend existing visuals with new elements.

Input:

The action accepts a comprehensive set of parameters defined in the input schema:

  • mask (string): URI of the input mask for inpainting. Black areas remain unchanged, while white areas are inpainted.
  • seed (integer): Optional random seed for image generation; leave blank for randomization.
  • image (string): URI of the input image used in img2img or inpainting modes.
  • width (integer): Width of the output image in pixels (default: 1024).
  • height (integer): Height of the output image in pixels (default: 1024).
  • prompt (string): Textual description to guide image generation (default: "An astronaut riding a rainbow unicorn").
  • loraScale (number): Scale factor for LoRA (Low-Rank Adaptation) between 0 and 1 (default: 0.6).
  • scheduler (string): Scheduling algorithm used for generation (default: "K_EULER").
  • numOutputs (integer): Number of output images to generate (default: 1, max: 4).
  • loraWeights (string): Specifies the LoRA weights; leave blank for defaults.
  • refineSteps (integer): Number of refinement steps for the image.
  • refineStyle (string): Style for refinement with options like "no_refiner" (default).
  • guidanceScale (number): Scale for classifier-free guidance (default: 7.5).
  • highNoiseFrac (number): Fraction of noise for refinement (default: 0.8).
  • applyWatermark (boolean): Enables watermarking on images (default: true).
  • negativePrompt (string): Defines elements to exclude from generation.
  • promptStrength (number): Strength of the prompt (default: 0.8).
  • numInferenceSteps (integer): Total denoising iterations (default: 50).
  • disableSafetyChecker (boolean): Option to disable safety checks for generated images (default: false).

Here’s an example input JSON payload for invoking this action:

{
  "width": 1024,
  "height": 1024,
  "prompt": "Black and yellow dots in the style of TOK",
  "loraScale": 0.4,
  "scheduler": "K_EULER",
  "numOutputs": 1,
  "refineStyle": "expert_ensemble_refiner",
  "guidanceScale": 7.5,
  "highNoiseFrac": 0.9,
  "applyWatermark": false,
  "negativePrompt": "",
  "promptStrength": 0.8,
  "numInferenceSteps": 30
}

Output:

The action typically returns a URL pointing to the generated image. For example:

[
  "https://assets.cognitiveactions.com/invocations/87b8a9e0-911f-472c-887c-f081070c14ab/0a6ddf3e-8e09-449c-b464-3ca94533d5a8.png"
]

This output provides direct access to the created visual content, making it easy for developers to display images in their applications.

Conceptual Usage Example (Python):

Here’s how you might call the Generate Image with Inpainting action through a hypothetical Cognitive Actions API endpoint:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"  # Hypothetical endpoint

action_id = "0d2e4f30-ab66-43b6-8659-3f2fd71bb44d"  # Action ID for Generate Image with Inpainting

# Construct the input payload based on the action's requirements
payload = {
    "width": 1024,
    "height": 1024,
    "prompt": "Black and yellow dots in the style of TOK",
    "loraScale": 0.4,
    "scheduler": "K_EULER",
    "numOutputs": 1,
    "refineStyle": "expert_ensemble_refiner",
    "guidanceScale": 7.5,
    "highNoiseFrac": 0.9,
    "applyWatermark": False,
    "negativePrompt": "",
    "promptStrength": 0.8,
    "numInferenceSteps": 30
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this snippet, replace the API key and endpoint with your own values. The payload is structured according to the input requirements of the action, ensuring that you supply all necessary parameters for successful execution.

Conclusion

The Generate Image with Inpainting action from the fofr/sdxl-dots spec opens a world of creative possibilities for developers. By leveraging this powerful tool, you can integrate advanced image generation capabilities directly into your applications. Explore various configurations to suit your project needs, and consider utilizing this action for use cases in digital art, design, and content creation. Happy coding!

← Back to All Articles