Streamline Language Model Outputs with OpenBuddy's Mistral-7B Cognitive Actions

23 Apr 2025

In today's fast-paced development environment, leveraging powerful machine learning models can significantly enhance the capabilities of your applications. The OpenBuddy Mistral-7B Cognitive Actions provide a robust API for integrating advanced language model functionalities into your projects. This set of actions allows developers to harness the power of the finetuned Mistral-7B model, optimized for efficiency in 4-bit quantization. Whether you need to generate responses, analyze text, or assist users in various tasks, these Cognitive Actions can streamline your development process.

Prerequisites

Before you begin integrating the OpenBuddy Mistral-7B Cognitive Actions into your application, ensure you have the following:

API Key: You will need an API key to authenticate your requests to the Cognitive Actions service.
Development Environment: Set up your development environment with access to libraries such as requests for making HTTP calls.

Authentication typically involves passing the API key in the request headers, allowing you to securely access the Cognitive Actions API.

Cognitive Actions Overview

Stream Language Model Output

The Stream Language Model Output action allows you to stream responses from the Mistral-7B model, enabling efficient text generation with customizable parameters. This action is particularly useful for applications that require real-time interaction, such as chatbots or educational tools.

Category: Language Detection

Input

The input for this action requires a prompt, but also allows for several optional parameters to fine-tune the model's output. Here’s a breakdown of the input schema:

prompt (required): The initial text input for the model.
topK (optional): Integer specifying the number of highest probability tokens to consider (default: 40).
topP (optional): Float that sets the cumulative probability threshold for output generation (default: 0.95).
temperature (optional): Float to control randomness in outputs (default: 0.7).
debug (optional): Boolean to include debugging info in logs (default: false).
numberOfBeams (optional): Integer for beam search (default: 1).
maximumNewTokens (optional): Integer that defines the maximum number of tokens to generate (default: 1024).
stopSequences (optional): Defines sequences that will stop generation when encountered.
promptTemplate (optional): Template for formatting prompts.

Example Input:

{
  "topK": 40,
  "topP": 0.95,
  "debug": true,
  "prompt": "Problem statement: \"\"\"一个盒子里有4个红球和6个蓝球。从盒子里不放回地抽出两个球。求一个球是红色，另一个球是蓝色的概率是多少？\"\"\"\n\nCorrect solution: \"\"\"我们可以先抽出红球，然后抽出蓝球，也可以先抽出蓝球，然后抽出红球。所以，我们需要把这两种情况的概率加起来。\n\n先抽出红球，然后抽出蓝球的概率是 (4/10)*(6/9) = 24/90 = 4/15。\n\n先抽出蓝球，然后抽出红球的概率是 (6/10)*(4/9) = 24/90 = 4/15。\n\n所以，一个球是红色，另一个球是蓝色的概率是 4/15 + 4/15 = 8/15。\"\"\n\nStudent’s solution: \"\"\"(4/10)*(6/9) = 24/90 = 4/15\"\"\"\n---\n请根据上面题目的标准答案和学生的回答，针对这道题给学生辅导.",
  "temperature": 0.7,
  "useSampling": true,
  "numberOfBeams": 1,
  "promptTemplate": "You are a helpful high school Math tutor. If you don't know the answer to a question, please don't share false information. You can speak fluently in many languages.\nUser: Hi\nAssistant: Hello, how can I help you?</s>\nUser: {prompt}\nAssistant:",
  "usePaddingMode": true,
  "maximumNewTokens": 1024,
  "repetitionPenalty": 1
}

Output

The action returns a stream of generated text based on the provided prompt and parameters. The output typically consists of an array of strings that represent the model's response.

Example Output:

[
  "",
  "好",
  "的",
  "",
  "，根",
  "据",
  "学",
  "生",
  "的",
  "回",
  "答",
  "",
  "，他",
  "的",
  "答",
  "案",
  "是",
  "正",
  "确",
  "的",
  "",
  "。\n",
  "\n",
  "我",
  "们",
  "可",
  "以",
  "用",
  "乘",
  "法",
  "原",
  "理",
  "来",
  "解",
  "决",
  "这",
  "个",
  "问",
  "题",
  "",
  "。首",
  "先",
  "",
  "，我",
  "们",
  "可",
  "以",
  "计",
  "算",
  "第",
  "一个",
  "球",
  "是",
  "红",
  "色",
  "的",
  "概",
  "率",
  "",
  "，那",
  "么",
  "第",
  "二",
  "个",
  "球",
  "是",
  "蓝",
  "色",
  "的",
  "概",
  "率",
  "就",
  "是",
  "",
  "：\n",
  "\n",
  "(4/10) ",
  "* ",
  "(6/9) ",
  "= ",
  "24/90 ",
  "= ",
  "4/15\n",
  "\n",
  "同",
  "理",
  "",
  "，第",
  "一个",
  "球",
  "是",
  "蓝",
  "色",
  "的",
  "概",
  "率",
  "",
  "，第",
  "二",
  "个",
  "球",
  "是",
  "红",
  "色",
  "的",
  "概",
  "率",
  "也",
  "是",
  " ",
  "4/15。\n",
  "\n",
  "因",
  "此",
  "",
  "，我",
  "们",
  "将",
  "这",
  "两",
  "种",
  "情",
  "况",
  "的",
  "概",
  "率",
  "加",
  "起",
  "来",
  "",
  "，得",
  "到",
  "一",
  "个",
  "球",
  "是",
  "红",
  "色",
  "",
  "，另",
  "一个",
  "球",
  "是",
  "蓝",
  "色",
  "的",
  "概",
  "率",
  "为",
  "",
  "8/15。\n",
  "",
  ""
]

Conceptual Usage Example (Python)

Here’s a conceptual example of how to use the Stream Language Model Output action in Python:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "8710cc91-50d1-4970-82f2-5ccc520164aa" # Action ID for Stream Language Model Output

# Construct the input payload based on the action's requirements
payload = {
    "topK": 40,
    "topP": 0.95,
    "debug": True,
    "prompt": "Problem statement: \"\"\"一个盒子里有4个红球和6个蓝球。从盒子里不放回地抽出两个球。求一个球是红色，另一个球是蓝色的概率是多少？\"\"\"\n\nCorrect solution: \"\"\"我们可以先抽出红球，然后抽出蓝球，也可以先抽出蓝球，然后抽出红球。所以，我们需要把这两种情况的概率加起来。\n\n先抽出红球，然后抽出蓝球的概率是 (4/10)*(6/9) = 24/90 = 4/15。\n\n先抽出蓝球，然后抽出红球的概率是 (6/10)*(4/9) = 24/90 = 4/15。\n\n所以，一个球是红色，另一个球是蓝色的概率是 4/15 + 4/15 = 8/15。\"\"\n\nStudent’s solution: \"\"\"(4/10)*(6/9) = 24/90 = 4/15\"\"\"\n---\n请根据上面题目的标准答案和学生的回答，针对这道题给学生辅导.",
    "temperature": 0.7,
    "useSampling": True,
    "numberOfBeams": 1,
    "promptTemplate": "You are a helpful high school Math tutor. If you don't know the answer to a question, please don't share false information. You can speak fluently in many languages.\nUser: Hi\nAssistant: Hello, how can I help you?</s>\nUser: {prompt}\nAssistant:",
    "usePaddingMode": True,
    "maximumNewTokens": 1024,
    "repetitionPenalty": 1
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this example, we set up the necessary headers for authentication and constructed the input payload according to the action's schema. The action ID and input payload are passed to the API, and the response is processed accordingly. Note that the endpoint and request structure shown are illustrative.

Conclusion

The OpenBuddy Mistral-7B Cognitive Actions offer a powerful way to integrate advanced language modeling capabilities into your applications. By utilizing the Stream Language Model Output action, you can enhance user interaction, improve educational tools, and create more engaging applications. Explore these Cognitive Actions and consider how they can streamline your development efforts today!