image-classification

Enhance Image Analysis with Sam's Image Prediction Action

25 Apr 2025
Enhance Image Analysis with Sam's Image Prediction Action

In the world of artificial intelligence, image classification stands out as a powerful tool for developers looking to enhance their applications with visual recognition capabilities. Sam's "Run Image Prediction" action provides a seamless way to perform predictions on images, enabling developers to extract meaningful insights from visual data. By leveraging a composite request format that includes image URIs and detailed prompt information, this action simplifies the process of identifying and labeling objects within images.

The benefits of using Sam's image prediction capabilities are numerous. Developers can automate the process of image analysis, improve the accuracy of object detection, and save valuable time by integrating this functionality into their applications. Common use cases for this action include e-commerce platforms identifying products in images, social media apps detecting faces or objects, and even healthcare applications analyzing medical images.

Prerequisites

To utilize Sam's Cognitive Actions, you'll need a valid API key and a basic understanding of how to make API calls.

Run Image Prediction

The "Run Image Prediction" action is designed to perform predictions on images by analyzing the content and returning labels based on provided prompts. This action is essential for developers who want to implement image classification features in their projects.

Input Requirements

The action requires a composite request containing:

  • Image: A URI string pointing to the image resource that you want to analyze. This should be a publicly accessible image URL.
  • Prompts JSON: A JSON string that defines the prompts for label detection. It should conform to the specific format outlined in the documentation, detailing the expected labels and their coordinates.

Example Input:

{
  "image": "https://replicate.delivery/pbxt/LYUNJEdh5dDgPO1XaJ3gbRYcKVSIohszubWGvicK9Q19lQd7/987E3D87-3E4B-4A17-A306-9C06E2ADB56D.png",
  "promptsJson": "{\"prompts\": [{\"label\": \"yogurt\", \"xmin\": 0.33819444444444446, \"xmax\": 0.6993055555555555, \"ymin\": 0.40260416666666665, \"ymax\": 0.6151041666666667, \"is_positive\": true}]}"
}

Expected Output

The output will include an array of masks, each containing the label detected in the image along with a URI to the corresponding mask image.

Example Output:

{
  "masks": [
    {
      "mask": "https://assets.cognitiveactions.com/invocations/719c54f9-40e4-4293-9ca4-d825ea1e180d/d47676e4-6ef0-4232-a070-55ab0223d5c3.jpg",
      "label": "yogurt"
    }
  ]
}

Use Cases for this Specific Action

  • E-commerce Applications: Automatically tag products in user-uploaded photos, enhancing searchability and user experience.
  • Social Media Platforms: Detect and label faces or objects in images to enable features like tagging or content moderation.
  • Healthcare Solutions: Analyze medical images to identify specific conditions or anomalies, aiding in diagnostics and patient care.

```python
import requests
import json

# Replace with your actual Cognitive Actions API key and endpoint
# Ensure your environment securely handles the API key
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
# This endpoint URL is hypothetical and should be documented for users
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"

action_id = "784ce4a4-ff8a-40b2-99c6-940ffbc07ba6" # Action ID for: Run Image Prediction

# Construct the exact input payload based on the action's requirements
# This example uses the predefined example_input for this action:
payload = {
  "image": "https://replicate.delivery/pbxt/LYUNJEdh5dDgPO1XaJ3gbRYcKVSIohszubWGvicK9Q19lQd7/987E3D87-3E4B-4A17-A306-9C06E2ADB56D.png",
  "promptsJson": "{\"prompts\": [{\"label\": \"yogurt\", \"xmin\": 0.33819444444444446, \"xmax\": 0.6993055555555555, \"ymin\": 0.40260416666666665, \"ymax\": 0.6151041666666667, \"is_positive\": true}]}"
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json",
    # Add any other required headers for the Cognitive Actions API
}

# Prepare the request body for the hypothetical execution endpoint
request_body = {
    "action_id": action_id,
    "inputs": payload
}

print(f"--- Calling Cognitive Action: {action.name or action_id} ---")
print(f"Endpoint: {COGNITIVE_ACTIONS_EXECUTE_URL}")
print(f"Action ID: {action_id}")
print("Payload being sent:")
print(json.dumps(request_body, indent=2))
print("------------------------------------------------")

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json=request_body
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully. Result:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body (non-JSON): {e.response.text}")
    print("------------------------------------------------")



## Conclusion
Incorporating Sam's "Run Image Prediction" action into your applications can significantly enhance image analysis capabilities, enabling faster and more accurate object detection. Whether you're improving an e-commerce site, enhancing a social media platform, or building innovative healthcare solutions, this action offers a robust solution for developers. Start integrating image classification into your projects today and unlock the potential of visual data analysis!
← Back to All Articles