아마존 Titan 텍스트 모델

Amazon Titan Text 모델은 다음과 같은 추론 파라미터를 지원합니다.

텍스트 프롬프트 엔지니어링 지침에 대한 자세한 내용은 Titan 텍스트 프롬프트 엔지니어링 지침을 참조하십시오Titan.

Titan모델에 대한 자세한 내용은 을 참조하십시오아마존 Titan 모델.

요청 및 응답

요청 본문은 InvokeModelor InvokeModelWithResponseStream요청 body 필드에 전달됩니다.

Request

{
    "inputText": string,
    "textGenerationConfig": {
        "temperature": float,  
        "topP": float,
        "maxTokenCount": int,
        "stopSequences": [string]
    }
}

다음 파라미터는 필수 파라미터입니다.

• InputText — 응답 생성을 위한 모델을 제공하라는 프롬프트입니다. 대화형 스타일로 응답을 생성하려면 다음 형식을 사용하여 프롬프트를 줄 바꿈하십시오.

  "inputText": "User: <prompt>\nBot:

textGenerationConfig은 선택 사항입니다. 이를 사용하여 다음과 같은 추론 파라미터를 구성할 수 있습니다.

• 온도 — 더 낮은 값을 사용하면 반응의 임의성을 줄일 수 있습니다.

  기본값	최소	Maximum
  0.7	0.0	1.0

• TopP — 가능성이 낮은 옵션은 무시하고 반응의 다양성을 줄이려면 낮은 값을 사용합니다.

  기본값	최소	Maximum
  0.9	0.0	1.0

• maxTokenCount— 응답에서 생성할 최대 토큰 수를 지정합니다. 최대 토큰 한도는 엄격하게 적용됩니다.

  모델	기본값	최소	Maximum
  Titan Text Lite	512	0	4,096
  Titan Text Express	512	0	8,192
  타이탄 텍스트 프리미어	512	0	3,072

• StopSequences — 모델이 멈춰야 하는 위치를 나타내는 문자 시퀀스를 지정합니다.

InvokeModel Response

응답 본문에는 다음과 같은 가능한 필드가 있습니다.

{
    'inputTextTokenCount': int,
    'results': [{
        'tokenCount': int,
        'outputText': '\n<response>\n',
        'completionReason': string
    }]
}

각 필드에 대한 자세한 내용은 아래에 나와 있습니다.

• inputTextTokenCount - 프롬프트에 있는 토큰 수.

• tokenCount - 응답에 있는 토큰 수.

• outputText - 응답의 텍스트.

• completionReason - 응답 생성이 완료된 이유. 발생 가능한 이유는 다음과 같습니다.

  • FINISHED - 응답이 완전히 생성되었습니다.

  • LENGTH - 설정한 응답 길이 때문에 응답이 잘렸습니다.

InvokeModelWithResponseStream Response

응답 스트림 본문의 각 텍스트 청크는 다음과 같은 형식으로 되어 있습니다. bytes 필드를 디코딩해야 합니다(예시로 API를 사용하여 단일 프롬프트로 모델 간접 호출 섹션 참조).

{
    'chunk': {
        'bytes': b'{
            "index": int,
            "inputTextTokenCount": int,
            "totalOutputTextTokenCount": int,
            "outputText": "<response-chunk>",
            "completionReason": string
        }'
    }
}

• index - 스트리밍 응답에 있는 청크의 인덱스.

• inputTextTokenCount - 프롬프트에 있는 토큰 수.

• totalOutputTextTokenCount - 응답에 있는 토큰 수.

• outputText - 응답의 텍스트.

• completionReason - 응답 생성이 완료된 이유. 발생 가능한 이유는 다음과 같습니다.

  • FINISHED - 응답이 완전히 생성되었습니다.

  • LENGTH - 설정한 응답 길이 때문에 응답이 잘렸습니다.

코드 예시

다음 예제는 Python SDK를 사용하여 Amazon Titan Text Premier 모델을 사용하여 추론을 실행하는 방법을 보여줍니다.

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to create a list of action items from a meeting transcript
with the Amazon Titan Text model (on demand).
"""
import json
import logging
import boto3

from botocore.exceptions import ClientError


class ImageError(Exception):
    "Custom exception for errors returned by Amazon Titan Text models"

    def __init__(self, message):
        self.message = message


logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


def generate_text(model_id, body):
    """
    Generate text using Amazon Titan Text models on demand.
    Args:
        model_id (str): The model ID to use.
        body (str) : The request body to use.
    Returns:
        response (json): The response from the model.
    """

    logger.info(
        "Generating text with Amazon Titan Text model %s", model_id)

    bedrock = boto3.client(service_name='bedrock-runtime')

    accept = "application/json"
    content_type = "application/json"

    response = bedrock.invoke_model(
        body=body, modelId=model_id, accept=accept, contentType=content_type
    )
    response_body = json.loads(response.get("body").read())

    finish_reason = response_body.get("error")

    if finish_reason is not None:
        raise ImageError(f"Text generation error. Error is {finish_reason}")

    logger.info(
        "Successfully generated text with Amazon Titan Text model %s", model_id)

    return response_body


def main():
    """
    Entrypoint for Amazon Titan Text model example.
    """
    try:
        logging.basicConfig(level=logging.INFO,
                            format="%(levelname)s: %(message)s")

        # You can replace the model_id with any other Titan Text Models
        # Titan Text Model family model_id is as mentioned below:
        # amazon.titan-text-premier-v1:0, amazon.titan-text-express-v1, amazon.titan-text-lite-v1
        model_id = 'amazon.titan-text-premier-v1:0'

        prompt = """Meeting transcript: Miguel: Hi Brant, I want to discuss the workstream  
            for our new product launch Brant: Sure Miguel, is there anything in particular you want
            to discuss? Miguel: Yes, I want to talk about how users enter into the product.
            Brant: Ok, in that case let me add in Namita. Namita: Hey everyone 
            Brant: Hi Namita, Miguel wants to discuss how users enter into the product.
            Miguel: its too complicated and we should remove friction.  
            for example, why do I need to fill out additional forms?  
            I also find it difficult to find where to access the product
            when I first land on the landing page. Brant: I would also add that
            I think there are too many steps. Namita: Ok, I can work on the
            landing page to make the product more discoverable but brant
            can you work on the additonal forms? Brant: Yes but I would need 
            to work with James from another team as he needs to unblock the sign up workflow.
            Miguel can you document any other concerns so that I can discuss with James only once?
            Miguel: Sure.
            From the meeting transcript above, Create a list of action items for each person. """

        body = json.dumps({
            "inputText": prompt,
            "textGenerationConfig": {
                "maxTokenCount": 3072,
                "stopSequences": [],
                "temperature": 0.7,
                "topP": 0.9
            }
        })

        response_body = generate_text(model_id, body)
        print(f"Input token count: {response_body['inputTextTokenCount']}")

        for result in response_body['results']:
            print(f"Token count: {result['tokenCount']}")
            print(f"Output text: {result['outputText']}")
            print(f"Completion reason: {result['completionReason']}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    except ImageError as err:
        logger.error(err.message)
        print(err.message)

    else:
        print(
            f"Finished generating text with the Amazon Titan Text Premier model {model_id}.")


if __name__ == "__main__":
    main()

다음 예제는 Python SDK를 사용하여 Amazon Titan Text G1 - Express 모델로 추론을 실행하는 방법을 보여줍니다.

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to create a list of action items from a meeting transcript
with the Amazon &titan-text-express; model (on demand).
"""
import json
import logging
import boto3

from botocore.exceptions import ClientError


class ImageError(Exception):
    "Custom exception for errors returned by Amazon &titan-text-express; model"

    def __init__(self, message):
        self.message = message


logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


def generate_text(model_id, body):
    """
    Generate text using Amazon &titan-text-express; model on demand.
    Args:
        model_id (str): The model ID to use.
        body (str) : The request body to use.
    Returns:
        response (json): The response from the model.
    """

    logger.info(
        "Generating text with Amazon &titan-text-express; model %s", model_id)

    bedrock = boto3.client(service_name='bedrock-runtime')

    accept = "application/json"
    content_type = "application/json"

    response = bedrock.invoke_model(
        body=body, modelId=model_id, accept=accept, contentType=content_type
    )
    response_body = json.loads(response.get("body").read())

    finish_reason = response_body.get("error")

    if finish_reason is not None:
        raise ImageError(f"Text generation error. Error is {finish_reason}")

    logger.info(
        "Successfully generated text with Amazon &titan-text-express; model %s", model_id)

    return response_body


def main():
    """
    Entrypoint for Amazon &titan-text-express; example.
    """
    try:
        logging.basicConfig(level=logging.INFO,
                            format="%(levelname)s: %(message)s")

        model_id = 'amazon.titan-text-express-v1'

        prompt = """Meeting transcript: Miguel: Hi Brant, I want to discuss the workstream  
            for our new product launch Brant: Sure Miguel, is there anything in particular you want
            to discuss? Miguel: Yes, I want to talk about how users enter into the product.
            Brant: Ok, in that case let me add in Namita. Namita: Hey everyone 
            Brant: Hi Namita, Miguel wants to discuss how users enter into the product.
            Miguel: its too complicated and we should remove friction.  
            for example, why do I need to fill out additional forms?  
            I also find it difficult to find where to access the product
            when I first land on the landing page. Brant: I would also add that
            I think there are too many steps. Namita: Ok, I can work on the
            landing page to make the product more discoverable but brant
            can you work on the additonal forms? Brant: Yes but I would need 
            to work with James from another team as he needs to unblock the sign up workflow.
            Miguel can you document any other concerns so that I can discuss with James only once?
            Miguel: Sure.
            From the meeting transcript above, Create a list of action items for each person. """

        body = json.dumps({
            "inputText": prompt,
            "textGenerationConfig": {
                "maxTokenCount": 4096,
                "stopSequences": [],
                "temperature": 0,
                "topP": 1
            }
        })

        response_body = generate_text(model_id, body)
        print(f"Input token count: {response_body['inputTextTokenCount']}")

        for result in response_body['results']:
            print(f"Token count: {result['tokenCount']}")
            print(f"Output text: {result['outputText']}")
            print(f"Completion reason: {result['completionReason']}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    except ImageError as err:
        logger.error(err.message)
        print(err.message)

    else:
        print(
            f"Finished generating text with the Amazon &titan-text-express; model {model_id}.")


if __name__ == "__main__":
    main()