Embabel

Core Types

LlmOptions

The LlmOptions class specifies which LLM to use and its hyperparameters. It’s defined in the embabel-common project and provides a fluent API for LLM configuration:

// Create LlmOptions with model and temperature
var options = LlmOptions
    .withModel(OpenAiModels.GPT_4O_MINI)
    .withTemperature(0.8);

// Use different hyperparameters for different tasks
var analyticalOptions = LlmOptions
    .withModel(OpenAiModels.GPT_4O_MINI)
    .withTemperature(0.2)
    .withTopP(0.9);

Important Methods:

  • withModel(String): Specify the model name
  • withRole(String): Specify the model role. The role must be one defined in configuration via embabel.models.llms.<role>=<model-name>
  • withTemperature(Double): Set creativity/randomness (0.0-1.0)
  • withTopP(Double): Set nucleus sampling parameter
  • withTopK(Integer): Set top-K sampling parameter
  • withPersona(String): Add a system message persona

LlmOptions is deserializable, so you can set properties of type LlmOptions in application.yml and other application configuration files. This is a powerful way of externalizing not only models, but hyperparameters.

PromptRunner

All LLM calls in user applications should be made via the PromptRunner interface. Once created, a PromptRunner can run multiple prompts with the same LLM, hyperparameters, tool groups and PromptContributors.

Getting a PromptRunner

You obtain a PromptRunner from an OperationContext using the fluent API:

@Action
public Story createStory(UserInput input, OperationContext context) {
    // Get PromptRunner with default LLM
    var runner = context.ai().withDefaultLlm();

    // Get PromptRunner with specific LLM options
    var customRunner = context.ai().withLlm(
        LlmOptions.withModel(OpenAiModels.GPT_4O_MINI)
            .withTemperature(0.8)
    );

    return customRunner.createObject("Write a story about: " + input.getContent(), Story.class);
}

PromptRunner Methods

Core Object Creation:

  • createObject(String, Class<T>): Create a typed object from a prompt, otherwise throw an exception. An exception triggers retry. If retry fails repeatedly, re-planning occurs.
  • createObjectIfPossible(String, Class<T>): Try to create an object, return null on failure. This can cause replanning.
  • generateText(String): Generate simple text response

Normally you want to use one of the createObject methods to ensure the response is typed correctly.

Tool and Context Management:

  • withToolGroup(String): Add tool groups for LLM access
  • withToolObject(Object): Add domain objects with @Tool methods
  • withPromptContributor(PromptContributor): Add context contributors
  • withImage(AgentImage): Add an image to the prompt for vision-capable LLMs
  • withImages(AgentImage...): Add multiple images to the prompt

LLM Configuration:

  • withLlm(LlmOptions): Use specific LLM configuration
  • withGenerateExamples(Boolean): Control example generation

Returning a Specific Type

  • creating(Class<T>): Go into the Creating fluent API for returning a particular type.

For example:

var story = context.ai()
    .withDefaultLlm()
    .withToolGroup(CoreToolGroups.WEB)
    .creating(Story.class)
    .fromPrompt("Create a story about: " + input.getContent());

The main reason to do this is to add strongly typed examples for few-shot prompting. For example:

var story = context.ai()
    .withDefaultLlm()
    .withToolGroup(CoreToolGroups.WEB)
    .creating(Story.class)
    .withExample("A children's story", new Story("Once upon a time...")) // ①
    .fromPrompt("Create a story about: " + input.getContent());
  1. Example: The example will be included in the prompt in JSON format to guide the LLM.

Working with Images:

var image = AgentImage.fromFile(imageFile);

var answer = context.ai()
    .withLlm(AnthropicModels.CLAUDE_35_HAIKU)  // ①
    .withImage(image)  // ②
    .generateText("What is in this image?");
  1. Vision-capable model required: Use Claude 3.x, GPT-4 Vision, or other multimodal LLMs
  2. Add image: Images are sent with the text prompt to the LLM. Can be used multiple times for multiple images.

Advanced Features:

  • rendering(String): Use Jinja templates for prompts (returns Rendering interface)
  • withTool(Subagent.ofClass(MyAgent.class).consuming(MyInput.class)): Enable handoffs to other agents (see )
  • evaluateCondition(String, String): Evaluate boolean condition

Validation

Embabel supports JSR-380 bean validation annotations on domain objects. When creating objects via PromptRunner.createObject or createObjectIfPossible, validation is automatically performed after deserialization. If validation fails, Embabel transparently retries the LLM call to obtain a valid object, describing the validation errors to the LLM to help it correct its response.

If validation fails a second time, InvalidLlmReturnTypeException is thrown. This will trigger replanning if not caught. You can also choose to catch it within the action method making the LLM call, and take appropriate action in your own code.

Simple example of annotation use:

public class User {

    @NotNull(message = "Name cannot be null")
    private String name;

    @AssertTrue(message = "Working must be true")
    private boolean working;

    @Size(min = 10, max = 200, message
      = "About Me must be between 10 and 200 characters")
    private String aboutMe;

    @Min(value = 18, message = "Age should not be less than 18")
    @Max(value = 150, message = "Age should not be greater than 150")
    private int age;

    @Email(message = "Email should be valid")
    private String email;

    // standard setters and getters
}

You can also use custom annotations with validators that will be injected by Spring. For example:

@Target(\{ElementType.FIELD, ElementType.PARAMETER}) // ①
@Retention(RetentionPolicy.RUNTIME)
@Constraint(validatedBy = PalindromeValidator.class)
public @interface MustBePalindrome {
    String message() default "Must be a palindrome";
    Class<?>[] groups() default {};
    Class<? extends Payload>[] payload() default {};
}

public class Palindromic {
    @MustBePalindrome // ②
    private String eats;

    public Palindromic(String eats) {
        this.eats = eats;
    }

    public String getEats() {
        return eats;
    }
}

@Component // ③
public class PalindromeValidator implements ConstraintValidator<MustBePalindrome, String> {

    private final Ai ai; // ④

    public PalindromeValidator(Ai ai) {
        this.ai = ai;
    }

    @Override
    public boolean isValid(String field, ConstraintValidatorContext context) {
        if (field == null) {
            return false;
        }
        return field.equals(new StringBuilder(field).reverse().toString());
    }
}
  1. Define the custom annotation
  2. Apply the annotation to a field
  3. Implement the validator as a Spring component. Note the @Component annotation.
  4. Spring will inject the validator with dependencies, such as the Ai instance in this case

Thus we have standard JSR-280 validation with full Spring dependency injection support.

AgentImage

Represents an image for use with vision-capable LLMs.

Factory Methods:

  • AgentImage.fromFile(File): Load from file (auto-detects MIME type from common extensions)
  • AgentImage.fromPath(Path): Load from path (auto-detects MIME type)
  • AgentImage.create(String, byte[]): Create with explicit MIME type and byte array
  • AgentImage.fromBytes(String, byte[]): Create from filename and bytes (auto-detects MIME type)

For uncommon image formats or if auto-detection fails, use AgentImage.create() with an explicit MIME type.

Was this page helpful?

Share