image-analysis

Effortless Image Masking with Mask Maker Actions

25 Apr 2025
Effortless Image Masking with Mask Maker Actions

In the world of image processing, the ability to accurately and efficiently create masks for various objects is crucial for a multitude of applications. The "Mask Maker" service provides a powerful set of Cognitive Actions designed to streamline this process, enabling developers to integrate sophisticated image analysis capabilities into their applications. By leveraging advanced models like DINO and SAM, Mask Maker simplifies the task of detecting and refining regions within images, delivering precise masking data in an easy-to-use format.

Imagine a scenario where you need to segment images for machine learning, create visually appealing graphics, or automate content moderation. Mask Maker can help you achieve these goals with speed and efficiency, reducing the complexity of image processing tasks and allowing you to focus on building innovative solutions.

Prerequisites

To get started with Mask Maker, you'll need an API key for the Cognitive Actions service and a basic understanding of making API calls.

Detect and Refine Regions with DINO and SAM

This action utilizes the DINO model to detect various regions in an image and refines these detections with the SAM model. The result is a detailed masking data output that is encoded in RLE (Run-Length Encoding) format as JSON. This action is particularly useful for applications requiring precise object segmentation from images.

Input Requirements

The input for this action consists of a JSON object with the following properties:

  • image: A URI or file path pointing to the input image that needs processing.
  • threshold: A confidence threshold for object detection ranging from 0 to 1, with a default value of 0.2.
  • maskPrompt: A comma-separated list of objects you want to detect in the image (e.g., "dog, horse, man").

Example Input:

{
  "image": "https://replicate.delivery/pbxt/M0LMz3UdbYxGNrMD4zLnnvmONJz54mI8yrI3nLBBKUs1PCxK/sample.jpg",
  "threshold": 0.2,
  "maskPrompt": "dog, horse, man"
}

Expected Output

The output of this action includes metadata about the detection process and the refined masks for each specified object, returned in RLE encoded format. The output JSON provides details such as the processing time, the number of detections, and the specific mask encodings for each object.

Example Output:

{
  "meta": {
    "threshold": 0.2,
    "dino_model": "GroundingDINO",
    "request_id": "a4c076e7-975a-4355-b7ec-d9929ca9b1f6",
    "term_count": 3,
    "image_width": 533,
    "mask_prompt": "dog, horse, man",
    "image_height": 799,
    "request_date": "2025-04-02 13:45:15",
    "mask_encoding": "Custom RLE",
    "detection_count": 6,
    "processing_time": "3.2080 seconds",
    "mask_maker_version": "1.2.0"
  },
  "terms": [
    {
      "term": "dog",
      "mask_rle": { /* RLE data */ },
      "single_flg": false
    },
    {
      "term": "horse",
      "mask_rle": { /* RLE data */ },
      "single_flg": false
    },
    {
      "term": "man",
      "mask_rle": { /* RLE data */ },
      "single_flg": false
    }
  ]
}

Use Cases for this Specific Action

  • Image Segmentation for Machine Learning: Automatically segment images into specific categories (like animals or objects) to train machine learning models more effectively.
  • Content Creation: Enhance workflows in graphic design or video production by easily isolating and manipulating elements within images.
  • Automated Content Moderation: Quickly identify and mask inappropriate content in images for moderation purposes in social media or content platforms.

```python
import requests
import json

# Replace with your actual Cognitive Actions API key and endpoint
# Ensure your environment securely handles the API key
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
# This endpoint URL is hypothetical and should be documented for users
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"

action_id = "e7dc508d-5c38-4da0-8e32-367712713fa4" # Action ID for: Detect and Refine Regions with DINO and SAM

# Construct the exact input payload based on the action's requirements
# This example uses the predefined example_input for this action:
payload = {
  "image": "https://replicate.delivery/pbxt/M0LMz3UdbYxGNrMD4zLnnvmONJz54mI8yrI3nLBBKUs1PCxK/sample.jpg",
  "threshold": 0.2,
  "maskPrompt": "dog, horse, man"
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json",
    # Add any other required headers for the Cognitive Actions API
}

# Prepare the request body for the hypothetical execution endpoint
request_body = {
    "action_id": action_id,
    "inputs": payload
}

print(f"--- Calling Cognitive Action: {action.name or action_id} ---")
print(f"Endpoint: {COGNITIVE_ACTIONS_EXECUTE_URL}")
print(f"Action ID: {action_id}")
print("Payload being sent:")
print(json.dumps(request_body, indent=2))
print("------------------------------------------------")

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json=request_body
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully. Result:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body (non-JSON): {e.response.text}")
    print("------------------------------------------------")



## Conclusion
Mask Maker's Cognitive Actions provide developers with the tools to automate the complex task of image masking. By integrating these actions into your applications, you can improve efficiency, enhance user experience, and open up new possibilities for image analysis. Whether you are working on machine learning, content creation, or moderation, Mask Maker offers the precision and speed needed to elevate your projects. Start exploring the capabilities of Mask Maker today and see how it can transform your image processing workflows!
← Back to All Articles