Seamlessly Generate Object Masks with the Segment Anything Automatic Actions

21 Apr 2025
Seamlessly Generate Object Masks with the Segment Anything Automatic Actions

In the realm of image processing, the ability to accurately segment objects within an image is paramount. The Segment Anything Automatic actions harness the power of the Segment Anything Model (SAM) to automate the generation of high-quality object masks based on user-defined prompts. This not only enhances the speed and efficiency of image segmentation tasks but also enables developers to integrate advanced capabilities into their applications with ease.

Prerequisites

Before diving into the integration of Cognitive Actions, ensure you have the following prerequisites:

  • An API key for accessing the Cognitive Actions platform.
  • A basic understanding of JSON and how to make HTTP requests.
  • Familiarity with Python programming for executing the provided conceptual code examples.

For authentication, you will typically pass your API key in the request headers, ensuring secure access to the Cognitive Actions.

Cognitive Actions Overview

Generate Automatic Masks

The Generate Automatic Masks action is designed to utilize the Segment Anything Model (SAM) for producing high-quality object masks from input prompts, such as points or boxes. This action supports zero-shot performance across various segmentation tasks, generating masks for all objects in an image.

Input

The input for this action requires the following fields:

  • image (string, required): The URI of the input image. Must be a valid URL.
  • cropLayers (integer, optional): Number of layers for crop-based mask prediction. Default is 0.
  • resizeWidth (integer, optional): The width to resize the image before running inference. Default is 1024.
  • pointsPerSide (integer, optional): Specifies the number of sampling points along each side of the image. Default is 32.
  • cropOverlapRatio (number, optional): Defines the overlap fraction for image crops. Default is approximately 0.341.
  • stabilityScoreOffset (number, optional): Adjusts the stability score threshold. Default is 1.
  • minimumMaskRegionArea (integer, optional): Area threshold for postprocessing. Default is 0.
  • predictionIoUThreshold (number, optional): Threshold for filtering based on predicted mask quality. Default is 0.88.
  • stabilityScoreThreshold (number, optional): Stability threshold for mask binarization. Default is 0.95.
  • cropPointsDownscaleFactor (integer, optional): Downscale factor for sampling points. Default is 1.
  • boxNonMaxSuppressionThreshold (number, optional): IoU threshold for non-max suppression. Default is 0.7.
  • cropNonMaxSuppressionThreshold (number, optional): IoU threshold for non-max suppression across crops. Default is 0.7.

Example Input:

{
  "image": "https://replicate.delivery/pbxt/IbLtTz5PFfyk5W9GZCCKXyiyldxQyRGhmLlGo4zdCf2snIbW/chameleon.jpg",
  "resizeWidth": 1080,
  "pointsPerSide": 32,
  "cropOverlapRatio": 0.3413333333333333,
  "stabilityScoreOffset": 1,
  "minimumMaskRegionArea": 30,
  "predictionIoUThreshold": 0.88,
  "stabilityScoreThreshold": 0.95,
  "cropPointsDownscaleFactor": 1,
  "boxNonMaxSuppressionThreshold": 0.7,
  "cropNonMaxSuppressionThreshold": 0.7
}

Output

Upon successful execution, this action returns a URL pointing to the generated mask image. The output typically looks like this:

Example Output:

https://assets.cognitiveactions.com/invocations/b3fbcdb8-ca66-4287-ad28-d4057885cb0c/ae87e09e-00a4-454b-a4aa-5b7239ceb2b8.png

Conceptual Usage Example (Python)

Here’s a conceptual Python snippet demonstrating how to invoke the Generate Automatic Masks action:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "a51be888-1145-42b3-961c-6aef01128805" # Action ID for Generate Automatic Masks

# Construct the input payload based on the action's requirements
payload = {
    "image": "https://replicate.delivery/pbxt/IbLtTz5PFfyk5W9GZCCKXyiyldxQyRGhmLlGo4zdCf2snIbW/chameleon.jpg",
    "resizeWidth": 1080,
    "pointsPerSide": 32,
    "cropOverlapRatio": 0.3413333333333333,
    "stabilityScoreOffset": 1,
    "minimumMaskRegionArea": 30,
    "predictionIoUThreshold": 0.88,
    "stabilityScoreThreshold": 0.95,
    "cropPointsDownscaleFactor": 1,
    "boxNonMaxSuppressionThreshold": 0.7,
    "cropNonMaxSuppressionThreshold": 0.7
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

This Python code snippet demonstrates how to properly structure the API call, focusing on setting the action ID and the input payload. Note that the endpoint URL and request structure are illustrative for conceptual understanding.

Conclusion

The Segment Anything Automatic actions provide a powerful toolset for developers looking to enhance their applications with advanced image segmentation capabilities. By leveraging the Generate Automatic Masks action, you can automate the extraction of object masks efficiently. Moving forward, consider exploring additional use cases, such as real-time image processing or integrating these capabilities into larger machine learning workflows to capitalize on the potential of automated segmentation.

← Back to All Articles