Decision specs (experimental)

A decision spec is an ordered set of named questions that a model answers in one decision. A decision response holds the model’s answer to each question. The types are in embabel-agent-ai, in the package com.embabel.common.ai.decision. They carry their own Jackson annotations.

This page covers building, serializing and reading specs and responses. covers selecting a decision service and running a spec on it.

Every decision type on this page carries @ApiStatus.Experimental. The types become stable once they have a named owner, a consumer application that uses them, and a compatibility assessment. Promotion is decided at the first release after all three are in place.

Types

A response holds no question objects, no instructions and no input. Code can read a response without the spec that produced it.

Builder

var triage = DecisionSpec.builder()
    .proposition("is_urgent", question -> question
        .asking("Does this convey urgency?"))
    .choice("department", question -> question
        .asking("Which team should handle this?")
        .option("billing", "Payments, invoicing, refunds")
        .option("technical", "Bugs, outages, integrations")
        .option("sales", "Pricing, upgrades, new accounts"))
    .rating("frustration", question -> question
        .asking("How frustrated is the customer?")
        .level("Calm")
        .level("Frustrated")
        .level("Very angry"))
    .build();

DecisionSpec.builder() starts an empty builder. The proposition, choice and rating methods each take a question name and a customizer. The customizer receives a builder of that kind that already carries the name. It runs once, before the method returns.

A blank or repeated name throws IllegalArgumentException, and the customizer does not run. An invalid definition also throws IllegalArgumentException. A declaration that throws adds nothing to the builder. A customizer that declares a question on its own enclosing builder gets an IllegalStateException.

build() returns a new spec on each call. Later declarations do not change specs already built. build() throws IllegalArgumentException when no question has been declared. A builder is not safe for use from several threads at once.

A spec is immutable. getQuestions() returns the questions in declared order. question(name) returns the question with that name, or null when there is none. Two specs are equal when they hold equal questions in the same order.

Named questions

var urgent = Questions.named("is_urgent")
    .proposition("Does this convey urgency?")
    .build();

var department = Questions.named("department")
    .choice("Which team should handle this?")
    .option("billing", "Payments, invoicing, refunds")
    .option("technical", "Bugs, outages, integrations")
    .option("sales", "Pricing, upgrades, new accounts")
    .build();

var triage = DecisionSpec.of(urgent, department);

Questions.named(name) starts a question on its own. Its proposition, choice and rating methods take the instructions and return the builder for that kind. A question built this way can live in a field, join several specs, and read its typed answer from a response.

DecisionSpec.of takes questions as arguments or as a list. DecisionSpec.Builder.question(question) adds a question built earlier to a builder. The spec in this example equals a spec built with the builder from the same two questions.

Kotlin DSL

val triage = decisionSpec {
    proposition("is_urgent") {
        asking("Does this convey urgency?")
    }
    choice("department") {
        asking("Which team should handle this?")
        option("billing", "Payments, invoicing, refunds")
        option("technical", "Bugs, outages, integrations")
        option("sales", "Pricing, upgrades, new accounts")
    }
    rating("frustration") {
        asking("How frustrated is the customer?")
        level("Calm")
        level("Frustrated")
        level("Very angry")
    }
}

decisionSpec { } builds a spec from a receiver block on DecisionSpecScope. The block runs once. Each call inside it goes through the matching DecisionSpec.Builder method, so a DSL spec validates exactly like a builder spec. DecisionSpecScope.question(question) adds a question built earlier. The spec is built once, after the block returns.

The @DecisionSpecDsl marker sits on the scope and on the three question builders. Code inside a question block cannot call the enclosing spec scope without an explicit label. Java code does not see decisionSpec or the scope methods.

Classification specs

A ClassificationSpec is a decision spec with one choice question. It equals a DecisionSpec that holds the same question, and it writes the same JSON. A ClassificationRequest is a DecisionRequest whose spec is a ClassificationSpec. shows how to build one and classify with it.

Question kinds

Every question needs a nonblank name and nonblank instructions. Option ids and level ids must be nonblank and unique within their question. level(label) uses the label as the level’s id and description. Choice options are Category values. Rating levels are RatingLevel values, and the first level has index 0.

ChoiceQuestionSpec.validate(result) and RatingQuestionSpec.validate(result) check that a result fits the question. Each returns the result unchanged or throws IllegalArgumentException.

Rating evidence

RatingResult has three classes: Answered, Inconclusive and Failure. Inconclusive holds a ModelProvenance. Failure holds a FailureReason. Answered holds the evidence in this table:

An Answered result reports a selected level, a distribution, a score, or a combination of these. Each part comes from the provider’s response as reported. A selected level is present only when the provider selected one, and a distribution only when the provider reported one.

A RatingScore pairs a value with a RatingStatistic. The one statistic is EXPECTED_LEVEL_INDEX, the probability-weighted mean of the zero-based level positions. It is a continuous number over the level range.

The checks against the question’s levels run when a result is validated against a RatingQuestionSpec or added to a response. The other rules run when the result is constructed.

Responses

DecisionResponse.builder(spec) starts a response to a spec. Its answer(question, outcome) overloads take a question of each kind with the matching result type. Answers can be added in any order, and the response lists them in spec order. DecisionResponse.failed(spec, reason) records a request that failed as a whole. Every answer in a failed response is a failure with that reason, and getRequestFailure() returns it.

The builder’s answer method throws IllegalArgumentException in these cases:

• The spec has no question with that name.
• The spec’s question with that name is not equal to the given question.
• The question already has an answer.
• A choice selection is not one of the options.
• Rating evidence does not fit the levels.

build() throws IllegalArgumentException when a question of the spec has no answer.

A response read from JSON or storage is checked answer by answer, with no reference to a spec. requireMatches(spec) checks it against a spec. It throws IllegalArgumentException when an answer is missing, extra or out of order, or when an answer’s kind, options or levels differ from its question’s.

Typed lookup

PropositionResult urgency = response.answer(urgent);
ClassificationResult team = response.answer(department);
RatingResult anger = response.answer(frustration);

answer(question) returns the outcome with the result type of the question’s class. The question can be a different object from the one that built the response. The answer must have the question’s name and kind, and a choice answer’s options or a rating answer’s levels must equal the question’s. Otherwise the lookup throws IllegalArgumentException, and the message names the question and what differs. An answer holds no instructions, so a question whose instructions were reworded still matches.

Lookup by name

String route = switch (response.answer("department")) {
    case DecisionAnswer.Choice choice
        when choice.getOutcome() instanceof ClassificationResult.Selected selected -> selected.getCategoryId();
    case DecisionAnswer.Choice choice -> "triage-queue";
    case DecisionAnswer.Proposition proposition -> "not a choice";
    case DecisionAnswer.Rating rating -> "not a choice";
};

answer(name) returns the DecisionAnswer with that name. It throws IllegalArgumentException when the response has no answer with that name. DecisionAnswer is sealed, with the classes DecisionAnswer.Proposition, DecisionAnswer.Choice and DecisionAnswer.Rating. A Java switch or a Kotlin when over an answer covers every case. DecisionAnswer.Choice also holds the options it was answered with, and DecisionAnswer.Rating holds its levels. An answer read from JSON holds them as written, and the typed lookup and requireMatches check them against a question.

Requests and capabilities

DecisionRequest.of(input, spec) pairs an input with a spec. DecisionRequest.of(input, questions...) builds the spec from the questions in order. The input may be empty. A request’s toString() shows the spec and leaves the input out.

DecisionCapabilities describes what a service accepts: the question kinds it answers. DecisionCapabilities.of(questionKinds) creates one.

JSON format

Each decision type has fixed member names:

The answers member is an array of answers in spec order. Each answer carries its name. An answer written on its own has the same members. Failure reasons and rating statistics use the lower-case constant name, such as invalid_response and expected_level_index. Question kinds use the wire names from the question kinds table.

Each outcome has a status member that names its result class:

A provenance object has modelName and provider, and the optional version and requestId. A distribution entry has levelId and probability. A score has value and statistic. An explicit null in an optional member reads the same as an absent member.

The response from the typed lookup example writes this JSON, shown here with line breaks:

{
  "answers": [
    {
      "name": "is_urgent",
      "kind": "proposition",
      "outcome": {
        "status": "answered",
        "answer": true,
        "pTrue": 0.93,
        "provenance": { "modelName": "jev-latest", "provider": "typesafe" }
      }
    },
    {
      "name": "department",
      "kind": "choice",
      "options": [
        { "id": "billing", "description": "Payments, invoicing, refunds" },
        { "id": "technical", "description": "Bugs, outages, integrations" },
        { "id": "sales", "description": "Pricing, upgrades, new accounts" }
      ],
      "outcome": {
        "status": "selected",
        "categoryId": "billing",
        "confidence": 0.91,
        "provenance": { "modelName": "jev-latest", "provider": "typesafe" }
      }
    },
    {
      "name": "frustration",
      "kind": "rating",
      "levels": [
        { "id": "Calm", "description": "Calm" },
        { "id": "Frustrated", "description": "Frustrated" },
        { "id": "Very angry", "description": "Very angry" }
      ],
      "outcome": {
        "status": "answered",
        "selectedLevelId": "Frustrated",
        "provenance": { "modelName": "jev-latest", "provider": "typesafe" }
      }
    }
  ]
}

Spec and request JSON include each question’s instructions. Response JSON holds no instructions and no input.

Reading and writing with Jackson

Any Jackson 3 mapper reads and writes the decision types, with no module or other registration:

JsonMapper mapper = JsonMapper.builder().build();

String specJson = mapper.writeValueAsString(spec);
DecisionSpec specRead = mapper.readValue(specJson, DecisionSpec.class);

String responseJson = mapper.writeValueAsString(response);
DecisionResponse responseRead = mapper.readValue(responseJson, DecisionResponse.class);
ClassificationResult team = responseRead.answer(department);

A Spring Boot application’s auto-configured mapper and the platform’s mapper from EmbabelObjectMapperHolder.createDefault() work the same way. The member names are fixed, so the mapper’s naming strategy changes neither the written nor the accepted names. PropositionResult, ClassificationResult, ModelProvenance and Category write and read the same JSON on their own as inside decision JSON.

Reading applies these rules:

• An unknown member fails, whatever the mapper’s FAIL_ON_UNKNOWN_PROPERTIES setting is. A member that a later version adds fails in a reader that does not know it.
• A missing required member, an unknown kind, status, reason or statistic, and answers written as an object or a single value all fail.
• Every value goes through the same builders and factories as Java code, so a value they refuse fails with the builder’s message and the IllegalArgumentException as the cause.
• Each answer in a response is checked on its own, so an outcome must fit the answer’s own options or levels. Reading does not compare a response with a spec. Call requireMatches(spec) for that.
• A member that appears twice in one object follows Jackson’s default, where the last value wins.

Each failure is a Jackson DatabindException.

Projection

DecisionProjection.of(response, type) reads the response’s answered values and converts them to a caller type with Jackson. The result holds the converted value and the original response. DecisionProjection.answeredValues(response) returns the values as a map keyed by question name:

Any other outcome has no value, and neither does a rating answered with only a score or a distribution. Both calls then throw DecisionProjectionException, and its getQuestions() lists every answer without a value.

A record whose component names match the question names receives the values. The type argument is a Class<T> or a TypeReference<T>, and a TypeReference<T> suits a generic target such as a map. Each overload also takes an optional ObjectMapper. The default mapper fails when a constructor parameter of the target has no answered value. A mapper passed in keeps its own settings. A failed conversion throws DecisionProjectionException with an empty getQuestions() and the Jackson exception as the cause.

Converter

static class SupportRouteConverter implements Converter<DecisionResponse, SupportRoute> {

    private final PropositionQuestionSpec urgent;
    private final ChoiceQuestionSpec department;

    SupportRouteConverter(PropositionQuestionSpec urgent, ChoiceQuestionSpec department) {
        this.urgent = urgent;
        this.department = department;
    }

    @Override
    public SupportRoute convert(DecisionResponse response) {
        boolean isUrgent = switch (response.answer(urgent)) {
            case PropositionResult.Answered answered -> answered.getAnswer();
            // Without an answer, a person should look at the ticket soon.
            case PropositionResult.Inconclusive inconclusive -> true;
            case PropositionResult.Failure failure -> true;
        };
        String team = switch (response.answer(department)) {
            case ClassificationResult.Selected selected -> selected.getCategoryId();
            case ClassificationResult.NoMatch noMatch -> "general";
            case ClassificationResult.Inconclusive inconclusive -> "triage-queue";
            case ClassificationResult.Failure failure -> "triage-queue";
        };
        return new SupportRoute(isUrgent, team);
    }
}

A Spring Converter<DecisionResponse, T> reads each answer with typed lookup. Its switch statements give a value for every outcome class, including inconclusive and failed answers.