@Generated(value="com.amazonaws:aws-java-sdk-code-generator") public class InferenceConfiguration extends Object implements Serializable, Cloneable, StructuredPojo
Contains inference parameters to use when the agent invokes a foundation model in the part of the agent sequence
defined by the promptType. For more information, see Inference parameters for foundation
models.
| Constructor and Description |
|---|
InferenceConfiguration() |
| Modifier and Type | Method and Description |
|---|---|
InferenceConfiguration |
clone() |
boolean |
equals(Object obj) |
Integer |
getMaximumLength()
The maximum number of tokens to allow in the generated response.
|
List<String> |
getStopSequences()
A list of stop sequences.
|
Float |
getTemperature()
The likelihood of the model selecting higher-probability options while generating a response.
|
Integer |
getTopK()
While generating a response, the model determines the probability of the following token at each point of
generation.
|
Float |
getTopP()
While generating a response, the model determines the probability of the following token at each point of
generation.
|
int |
hashCode() |
void |
marshall(ProtocolMarshaller protocolMarshaller)
Marshalls this structured data using the given
ProtocolMarshaller. |
void |
setMaximumLength(Integer maximumLength)
The maximum number of tokens to allow in the generated response.
|
void |
setStopSequences(Collection<String> stopSequences)
A list of stop sequences.
|
void |
setTemperature(Float temperature)
The likelihood of the model selecting higher-probability options while generating a response.
|
void |
setTopK(Integer topK)
While generating a response, the model determines the probability of the following token at each point of
generation.
|
void |
setTopP(Float topP)
While generating a response, the model determines the probability of the following token at each point of
generation.
|
String |
toString()
Returns a string representation of this object.
|
InferenceConfiguration |
withMaximumLength(Integer maximumLength)
The maximum number of tokens to allow in the generated response.
|
InferenceConfiguration |
withStopSequences(Collection<String> stopSequences)
A list of stop sequences.
|
InferenceConfiguration |
withStopSequences(String... stopSequences)
A list of stop sequences.
|
InferenceConfiguration |
withTemperature(Float temperature)
The likelihood of the model selecting higher-probability options while generating a response.
|
InferenceConfiguration |
withTopK(Integer topK)
While generating a response, the model determines the probability of the following token at each point of
generation.
|
InferenceConfiguration |
withTopP(Float topP)
While generating a response, the model determines the probability of the following token at each point of
generation.
|
public void setMaximumLength(Integer maximumLength)
The maximum number of tokens to allow in the generated response.
maximumLength - The maximum number of tokens to allow in the generated response.public Integer getMaximumLength()
The maximum number of tokens to allow in the generated response.
public InferenceConfiguration withMaximumLength(Integer maximumLength)
The maximum number of tokens to allow in the generated response.
maximumLength - The maximum number of tokens to allow in the generated response.public List<String> getStopSequences()
A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.
public void setStopSequences(Collection<String> stopSequences)
A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.
stopSequences - A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop
generating the response.public InferenceConfiguration withStopSequences(String... stopSequences)
A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.
NOTE: This method appends the values to the existing list (if any). Use
setStopSequences(java.util.Collection) or withStopSequences(java.util.Collection) if you want
to override the existing values.
stopSequences - A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop
generating the response.public InferenceConfiguration withStopSequences(Collection<String> stopSequences)
A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.
stopSequences - A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop
generating the response.public void setTemperature(Float temperature)
The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.
temperature - The likelihood of the model selecting higher-probability options while generating a response. A lower
value makes the model more likely to choose higher-probability options, while a higher value makes the
model more likely to choose lower-probability options.public Float getTemperature()
The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.
public InferenceConfiguration withTemperature(Float temperature)
The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.
temperature - The likelihood of the model selecting higher-probability options while generating a response. A lower
value makes the model more likely to choose higher-probability options, while a higher value makes the
model more likely to choose lower-probability options.public void setTopK(Integer topK)
While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for topK is the number of most-likely candidates from which the
model chooses the next token in the sequence. For example, if you set topK to 50, the model selects
the next token from among the top 50 most likely choices.
topK - While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for topK is the number of most-likely candidates from
which the model chooses the next token in the sequence. For example, if you set topK to 50,
the model selects the next token from among the top 50 most likely choices.public Integer getTopK()
While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for topK is the number of most-likely candidates from which the
model chooses the next token in the sequence. For example, if you set topK to 50, the model selects
the next token from among the top 50 most likely choices.
topK is the number of most-likely candidates from
which the model chooses the next token in the sequence. For example, if you set topK to 50,
the model selects the next token from among the top 50 most likely choices.public InferenceConfiguration withTopK(Integer topK)
While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for topK is the number of most-likely candidates from which the
model chooses the next token in the sequence. For example, if you set topK to 50, the model selects
the next token from among the top 50 most likely choices.
topK - While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for topK is the number of most-likely candidates from
which the model chooses the next token in the sequence. For example, if you set topK to 50,
the model selects the next token from among the top 50 most likely choices.public void setTopP(Float topP)
While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for Top P determines the number of most-likely candidates from
which the model chooses the next token in the sequence. For example, if you set topP to 80, the
model only selects the next token from the top 80% of the probability distribution of next tokens.
topP - While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for Top P determines the number of most-likely candidates
from which the model chooses the next token in the sequence. For example, if you set topP to
80, the model only selects the next token from the top 80% of the probability distribution of next tokens.public Float getTopP()
While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for Top P determines the number of most-likely candidates from
which the model chooses the next token in the sequence. For example, if you set topP to 80, the
model only selects the next token from the top 80% of the probability distribution of next tokens.
Top P determines the number of most-likely candidates
from which the model chooses the next token in the sequence. For example, if you set topP to
80, the model only selects the next token from the top 80% of the probability distribution of next
tokens.public InferenceConfiguration withTopP(Float topP)
While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for Top P determines the number of most-likely candidates from
which the model chooses the next token in the sequence. For example, if you set topP to 80, the
model only selects the next token from the top 80% of the probability distribution of next tokens.
topP - While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for Top P determines the number of most-likely candidates
from which the model chooses the next token in the sequence. For example, if you set topP to
80, the model only selects the next token from the top 80% of the probability distribution of next tokens.public String toString()
toString in class ObjectObject.toString()public InferenceConfiguration clone()
public void marshall(ProtocolMarshaller protocolMarshaller)
StructuredPojoProtocolMarshaller.marshall in interface StructuredPojoprotocolMarshaller - Implementation of ProtocolMarshaller used to marshall this object's data.