Generate Stunning Images with pnyompen/sd-controlnet-lora Cognitive Actions

25 Apr 2025
Generate Stunning Images with pnyompen/sd-controlnet-lora Cognitive Actions

In today's digital landscape, generating high-quality images programmatically can significantly enhance applications in fields such as gaming, design, and content creation. The pnyompen/sd-controlnet-lora specification provides powerful Cognitive Actions that enable developers to generate images using advanced techniques like Canny control and LoRA (Low-Rank Adaptation). These pre-built actions simplify the process of creating visually compelling images, whether through direct generation or transformation of existing images.

Prerequisites

Before diving into the integration of these Cognitive Actions, ensure you have the following:

  • An API key for the Cognitive Actions platform to authenticate your requests.
  • Familiarity with JSON payloads, as you will need to structure your requests accordingly.
  • Basic knowledge of making HTTP requests in your programming language of choice.

To authenticate your requests, you will typically include your API key in the headers of your HTTP calls.

Cognitive Actions Overview

Generate Image with SD1.5 Canny ControlNet and LoRA

This action allows you to generate images using the SD1.5 Canny controlnet, enabling refined image synthesis with a variety of options, including image-to-image transformations, background removal, and automatic captioning.

Category: Image Generation

Input: The input schema for this action consists of several fields. Below is a description of the most relevant fields along with an example JSON payload.

  • seed (integer): Random seed for reproducibility. Leave blank for randomization.
  • image (string): URI of the input image for 'img2img' or 'inpaint' mode.
  • prompt (string): The main textual input prompt used to generate the image.
  • img2img (boolean): Enables the use of the img2img pipeline.
  • strength (number): Denoising strength when 'img2img' is enabled.
  • loraScale (number): Controls the impact of LoRA on the output.
  • scheduler (string): Specifies the scheduling algorithm for the denoising process.
  • numOutputs (integer): Number of images to output (1 to 4).
  • autoGenerateCaption (boolean): If enabled, uses BLIP to generate captions for input images.

Example Input:

{
  "image": "https://replicate.delivery/pbxt/JiOTMCHj4oGrTTf8Pg2r7vyI8YdXc5jL2IDyC2SfhuggjYe6/out-0%20%281%29.png",
  "prompt": "An astronaut riding a rainbow unicorn, In the style of oil painting",
  "img2img": true,
  "strength": 1,
  "loraScale": 0.95,
  "scheduler": "KarrasDPM",
  "numOutputs": 1,
  "guidanceScale": 7.5,
  "conditionScale": 0.5,
  "ipAdapterScale": 0.1,
  "negativePrompt": "worst quality, photorealistic ",
  "numInferenceSteps": 15,
  "autoGenerateCaption": true,
  "generatedCaptionWeight": 0.1
}

Output: The action typically returns an array of image URLs generated based on the input parameters.

Example Output:

[
  "https://assets.cognitiveactions.com/invocations/8cbeeb14-2586-4d52-b809-72cfa77658de/94fc2882-e164-4803-83e2-6ef0e4bb044e.png"
]

Conceptual Usage Example (Python): Here's how you can invoke this action using a conceptual Python code snippet:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "1e1e71c8-02e5-4f1a-8bdf-4bc725cf83a1" # Action ID for Generate Image with SD1.5 Canny ControlNet and LoRA

# Construct the input payload based on the action's requirements
payload = {
    "image": "https://replicate.delivery/pbxt/JiOTMCHj4oGrTTf8Pg2r7vyI8YdXc5jL2IDyC2SfhuggjYe6/out-0%20%281%29.png",
    "prompt": "An astronaut riding a rainbow unicorn, In the style of oil painting",
    "img2img": True,
    "strength": 1,
    "loraScale": 0.95,
    "scheduler": "KarrasDPM",
    "numOutputs": 1,
    "guidanceScale": 7.5,
    "conditionScale": 0.5,
    "ipAdapterScale": 0.1,
    "negativePrompt": "worst quality, photorealistic ",
    "numInferenceSteps": 15,
    "autoGenerateCaption": True,
    "generatedCaptionWeight": 0.1
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this code snippet, replace the COGNITIVE_ACTIONS_API_KEY with your actual API key and modify the URL as required. The action_id corresponds to the action you are invoking, and the payload is structured according to the input schema outlined above.

Conclusion

The pnyompen/sd-controlnet-lora Cognitive Actions empower developers to integrate sophisticated image generation capabilities into their applications effortlessly. With features such as img2img transformations, background removal, and automatic captioning, you can create unique images tailored to your needs. Explore these actions further to see how they can enhance your projects, and don't hesitate to experiment with different prompts and parameters to see the creative possibilities unfold!

← Back to All Articles