Unlock Image Generation: Using Cognitive Actions for Inpainting with ControlNet and LoRA

24 Apr 2025
Unlock Image Generation: Using Cognitive Actions for Inpainting with ControlNet and LoRA

In the rapidly evolving field of artificial intelligence, the ability to manipulate and generate images has become increasingly accessible. The lightweight-ai/model3_4 API offers a powerful Cognitive Action designed specifically for image generation: Generate Inpainted Image with ControlNet and LoRA. This action allows developers to create inpainted images by partially modifying a base image based on user-defined prompts and masks. With the option to utilize ControlNet and LoRA models, developers can achieve a high degree of customization in their outputs.

Prerequisites

Before you can start using the Cognitive Actions, ensure you have the following:

  • An API key for the Cognitive Actions platform.
  • Basic knowledge of how to make HTTP requests and handle JSON data.

Authentication typically involves including your API key in the request headers when calling the action endpoints.

Cognitive Actions Overview

Generate Inpainted Image with ControlNet and LoRA

This action generates inpainted images by modifying specific areas of a base image. Users can influence the output through prompts, masks, and optional models, allowing for versatile image creation tailored to specific needs.

Category: Image Generation

Input

The input schema for this action requires a JSON object containing various properties. Below is a breakdown of the required and optional fields:

  • mask (string, required): A URI to a mask image indicating the areas to modify (white areas to change, black areas to preserve).
  • seed (integer, optional): A random seed for reproducibility.
  • image (string, required): A URI to the base image for modifications.
  • width (integer, optional): Output image width (default: 1024).
  • height (integer, optional): Output image height (default: 1024).
  • prompt (string, optional): Text prompt to guide image generation (default: "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves").
  • loraList (array, optional): List of Lora modifiers for the image generation.
  • enableInpainting (boolean, optional): Toggle for enabling inpainting (default: false).
  • imageOutputFormat (string, optional): Format of the output image (default: png).
  • numberOfOutputs (integer, optional): Total number of images to generate (default: 1, max: 4).

Example Input:

{
  "width": 1024,
  "height": 1024,
  "prompt": "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves",
  "loraList": [],
  "scheduleType": "K_EULER",
  "guidanceScale": 3.5,
  "outputQuality": 100,
  "promptStrength": 0.8,
  "numberOfOutputs": 1,
  "undesiredPrompt": "",
  "enableInpainting": false,
  "imageOutputFormat": "png",
  "inferenceStepCount": 28,
  "loraIntensityLevels": []
}

Output

The output of this action is a JSON array containing URLs to the generated images. Here is an example of what you might receive:

Example Output:

[
  "https://assets.cognitiveactions.com/invocations/960726c3-44d4-41c0-9b93-6a65c982ff9b/679283c2-b284-4fbd-826e-30f2c6f6289e.png"
]

Conceptual Usage Example (Python)

Below is a Python code snippet illustrating how to call the Cognitive Actions endpoint to generate an inpainted image:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "6f392c65-eb58-4f7e-ad65-b5f6ba2acaac" # Action ID for Generate Inpainted Image with ControlNet and LoRA

# Construct the input payload based on the action's requirements
payload = {
    "width": 1024,
    "height": 1024,
    "prompt": "A bohemian-style female travel blogger with sun-kissed skin and messy beach waves",
    "loraList": [],
    "scheduleType": "K_EULER",
    "guidanceScale": 3.5,
    "outputQuality": 100,
    "promptStrength": 0.8,
    "numberOfOutputs": 1,
    "enableInpainting": false,
    "imageOutputFormat": "png",
    "inferenceStepCount": 28,
    "loraIntensityLevels": []
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this snippet:

  • Replace YOUR_COGNITIVE_ACTIONS_API_KEY with your actual API key.
  • The payload dictionary contains the required input parameters based on the action's schema.

Conclusion

The Generate Inpainted Image with ControlNet and LoRA action within the lightweight-ai/model3_4 API provides developers with a robust tool for creative image manipulation. By allowing customization through various parameters, this action opens the door to innovative applications in design, art, and content creation. Start experimenting with these capabilities in your projects today!

← Back to All Articles