Enhance Image Manipulation with ControlNet and IP Adapter Actions

23 Apr 2025
Enhance Image Manipulation with ControlNet and IP Adapter Actions

In the world of image processing, the ability to manipulate and enhance images is paramount. The ControlNet X Majic Mix Realistic X IP Adapter offers a powerful set of Cognitive Actions that allow developers to perform sophisticated image inpainting and manipulation tasks. By utilizing these pre-built actions, developers can save time and effort, focusing instead on creating rich user experiences. This article will guide you through how to leverage these actions to achieve impressive image processing results.

Prerequisites

Before you can start using the ControlNet and IP Adapter actions, ensure that you have the following:

  • An API key for accessing the Cognitive Actions platform.
  • Basic knowledge of JSON format and how to structure API requests.
  • A suitable development environment, such as Python, for making HTTP requests.

Authentication typically involves sending your API key in the request headers, which will allow you to access the Cognitive Actions execution endpoint.

Cognitive Actions Overview

Perform Inpainting with ControlNet and IP Adapter

The Perform Inpainting with ControlNet and IP Adapter action is designed for image inpainting and manipulation. It provides flexibility through multiple ControlNet configurations, enabling improved detail and noise control in generated images.

Input

The input for this action is defined by a schema that requires a prompt as a mandatory field and includes several optional fields to customize the output:

  • prompt (required): A string that describes what you want to generate (e.g., "cat").
  • guessMode (optional): Boolean to enable guess mode for content interpretation (default: false).
  • lineArtImage (optional): URI for a line art control image.
  • guidanceScale (optional): Numeric scale for classifier-free guidance (range: 0.1 to 30).
  • ipAdapterImage (optional): URI for the IP Adapter control image.
  • negativePrompt (optional): Describes undesirable elements to avoid (default: "Longbody, lowres, bad anatomy...").
  • ipAdapterWeight (optional): Weight for IP Adapter effects (default: 1).
  • maximumImageWidth (optional): Maximum width of the output image (default: 512).
  • maximumImageHeight (optional): Maximum height of the output image (default: 512).
  • Additional fields for customizing the process further (e.g., conditioning scales, number of inference steps).

Example Input JSON:

{
  "prompt": "cat",
  "guessMode": false,
  "lineArtImage": "https://replicate.delivery/pbxt/JvfJGnmbU8uxfzsCeqLBCdEoeOQT09xZOAQLGbGLqyNYrrKc/output.png",
  "guidanceScale": 8.72,
  "schedulerType": "K_EULER_ANCESTRAL",
  "ipAdapterImage": "https://replicate.delivery/pbxt/JvfJGnjAmUqqNLFm8OyfbOXynAQLvVOtgSrq2cWOWeAb2wZW/Cool-PC-Wallpapers-HD-2.jpg",
  "negativePrompt": "(worst quality:2),(low quality:2),(normal quality:2),lowres,watermark,",
  "ipAdapterWeight": 0.3,
  "maximumImageWidth": 712,
  "disableSafetyCheck": true,
  "maximumImageHeight": 712,
  "numberOfInferenceSteps": 30,
  "numberOfGeneratedImages": 1,
  "lineartConditioningScale": 1.2,
  "brightnessConditioningScale": 1,
  "inpaintingConditioningScale": 1
}

Output

The output of this action typically returns an array of URLs pointing to the generated images. Here’s an example of a possible output:

Example Output:

[
  "https://assets.cognitiveactions.com/invocations/4e68eedb-3219-4834-adcf-9e84ca8830f0/3a13cd75-cb98-4560-8d2a-692fc04c8a32.png"
]

Conceptual Usage Example (Python)

Here’s how you might structure a Python request to use this action:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "eb08ebe2-854a-484f-865d-832d9c77e4f8" # Action ID for Perform Inpainting with ControlNet and IP Adapter

# Construct the input payload based on the action's requirements
payload = {
    "prompt": "cat",
    "guessMode": false,
    "lineArtImage": "https://replicate.delivery/pbxt/JvfJGnmbU8uxfzsCeqLBCdEoeOQT09xZOAQLGbGLqyNYrrKc/output.png",
    "guidanceScale": 8.72,
    "schedulerType": "K_EULER_ANCESTRAL",
    "ipAdapterImage": "https://replicate.delivery/pbxt/JvfJGnjAmUqqNLFm8OyfbOXynAQLvVOtgSrq2cWOWeAb2wZW/Cool-PC-Wallpapers-HD-2.jpg",
    "negativePrompt": "(worst quality:2),(low quality:2),(normal quality:2),lowres,watermark,",
    "ipAdapterWeight": 0.3,
    "maximumImageWidth": 712,
    "disableSafetyCheck": true,
    "maximumImageHeight": 712,
    "numberOfInferenceSteps": 30,
    "numberOfGeneratedImages": 1,
    "lineartConditioningScale": 1.2,
    "brightnessConditioningScale": 1,
    "inpaintingConditioningScale": 1
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this code snippet, replace "YOUR_COGNITIVE_ACTIONS_API_KEY" with your actual API key. The payload is structured according to the action's requirements, and the request is sent to a hypothetical endpoint.

Conclusion

The ControlNet and IP Adapter Cognitive Actions empower developers to create impressive image manipulations with ease. By leveraging the capabilities of inpainting and various control images, you can enhance your applications significantly. As you explore these actions, consider potential use cases in image editing, content creation, and beyond. Start integrating these actions into your applications today and unlock new creative possibilities!

← Back to All Articles