Effortless Image Segmentation with lucataco/segment-anything-2 Cognitive Actions

22 Apr 2025
Effortless Image Segmentation with lucataco/segment-anything-2 Cognitive Actions

The lucataco/segment-anything-2 API provides developers with the ability to perform advanced image segmentation using the state-of-the-art SAM 2 algorithm developed by Meta. By leveraging these pre-built Cognitive Actions, developers can effortlessly integrate automatic mask generation into their applications, enhancing image processing capabilities and enabling rich functionalities like object detection and scene analysis.

Prerequisites

To get started with the lucataco/segment-anything-2 Cognitive Actions, you will need:

  • An API key for the Cognitive Actions platform, which allows you to authenticate your requests.
  • Basic knowledge of how to make HTTP requests, preferably using a programming language like Python.

Authentication typically involves passing your API key in the request headers, allowing you to securely access the Cognitive Actions.

Cognitive Actions Overview

Automatically Generate Image Masks

The Automatically Generate Image Masks action is designed to utilize SAM 2 by Meta for automatic mask generation. This action segments objects in images, providing real-time processing capabilities with detailed and accurate predictions.

  • Category: Image Segmentation

Input

The input schema for this action requires at least one field: the image URI. Below is the structure of the input:

{
  "image": "https://replicate.delivery/pbxt/LMUHQoCnNzN15MpwWLPJUVF96g3zxer6p3dNTHtVrNxqMMe0/socer.png",
  "maskLimit": 2,
  "maskToMask": true,
  "pointsPerSide": 64,
  "pointsPerBatch": 128,
  "boxNmsThreshold": 0.7,
  "multiMaskOutput": false,
  "minMaskRegionArea": 25,
  "cropNumberOfLayers": 1,
  "stabilityScoreOffset": 0.7,
  "predictionIouThreshold": 0.7,
  "stabilityScoreThreshold": 0.92,
  "cropNumberOfPointsDownscaleFactor": 2
}
  • Required Fields:
    • image: URI of the input image to be processed.
  • Optional Fields:
    • maskLimit: Maximum number of masks to return (default is -1 for all masks).
    • maskToMask: Enables refinement using previous mask predictions (default is true).
    • pointsPerSide: Number of points sampled along one side of the image (default is 64).
    • pointsPerBatch: Number of points processed simultaneously (default is 128).
    • boxNmsThreshold: IoU threshold for non-maximal suppression (default is 0.7).
    • multiMaskOutput: Determines if multiple masks are output at each point (default is false).
    • minMaskRegionArea: Minimum area for mask filtering (default is 25).
    • cropNumberOfLayers: Additional mask predictions on cropped regions (default is 1).
    • stabilityScoreOffset: Adjustment factor for stability score cutoff (default is 0.7).
    • predictionIouThreshold: Threshold for filtering masks based on IoU (default is 0.7).
    • stabilityScoreThreshold: Threshold based on mask stability (default is 0.92).
    • cropNumberOfPointsDownscaleFactor: Scaling factor for points in successive layers (default is 2).

Output

The output of this action typically returns an array of image URIs representing the generated masks. Here’s an example of the output you can expect:

[
  "https://assets.cognitiveactions.com/invocations/5fff6eaf-62be-4e59-a110-99c426a341a2/63bf44f6-1c36-4899-8418-48da7e60c5a4.png",
  "https://assets.cognitiveactions.com/invocations/5fff6eaf-62be-4e59-a110-99c426a341a2/d0d160fe-c937-4c60-9519-00fec7e1e710.png"
]

Conceptual Usage Example (Python)

Below is a conceptual Python code snippet demonstrating how you might call the Automatically Generate Image Masks action:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "5fdae670-fc6f-4111-bca7-4cafe34baf62" # Action ID for Automatically Generate Image Masks

# Construct the input payload based on the action's requirements
payload = {
    "image": "https://replicate.delivery/pbxt/LMUHQoCnNzN15MpwWLPJUVF96g3zxer6p3dNTHtVrNxqMMe0/socer.png",
    "maskLimit": 2,
    "maskToMask": True,
    "pointsPerSide": 64,
    "pointsPerBatch": 128,
    "boxNmsThreshold": 0.7,
    "multiMaskOutput": False,
    "minMaskRegionArea": 25,
    "cropNumberOfLayers": 1,
    "stabilityScoreOffset": 0.7,
    "predictionIouThreshold": 0.7,
    "stabilityScoreThreshold": 0.92,
    "cropNumberOfPointsDownscaleFactor": 2
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this example, replace YOUR_COGNITIVE_ACTIONS_API_KEY with your actual API key. The input payload is structured according to the action's requirements, and you can observe how the action ID is integrated into the request.

Conclusion

The lucataco/segment-anything-2 Cognitive Actions provide a powerful and efficient way to integrate image segmentation capabilities into your applications. By utilizing the Automatically Generate Image Masks action, developers can enhance their projects with automatic and accurate segmentation, paving the way for advanced image analysis and object detection features. As you explore these capabilities, consider experimenting with various configurations to optimize mask generation for your specific use cases. Happy coding!

← Back to All Articles