Running decision specs (experimental)

This page covers selecting a decision service and running a decision spec on it. covers building specs and reading responses. and cover the two provider families.

The types are in embabel-agent-ai. Selection is in the package com.embabel.common.ai.model. Execution hooks, the response assembler and the content-capture switch are in com.embabel.common.ai.decision.spi. The no-op and stub services are in com.embabel.common.ai.decision.support.

flowchart TD
    Ai[Ai.decisions / classifications] --> Sel[ServiceSelector]
    Sel --> Reg[DecisionServiceRegistry]
    Sel --> Bind[operation binding]
    Bind --> Obs[ObservedDecisionService]
    Reg --> Obs
    Obs -->|ask: embabel.ai.ask| Exec[preflight and execution]
    Exec -->|question-set service| Hook[askQuestionSet: embabel.ai.decision ask_question_set]
    Exec -->|any other service| Split[assess / classify / rate]
    Hook --> Jev[TypeSafeDecisionService]
    Hook --> Llm[LlmDecisionService]
    Hook --> Fix[NoOpDecisionService / StubDecisionService]
    Jev --> Asm[DecisionResponseAssembler]
    Llm --> Asm

Selecting a service

Ai.decisions() returns a ServiceSelector<DecisionService>. Ai.classifications() returns a ServiceSelector<ClassificationService>. Each selector has four terminals, and each terminal returns the service itself:

DecisionService byRole = ai.decisions().byRole("support-triage");
DecisionService byName = ai.decisions().named("triage-stub");
DecisionService supplied = ai.decisions().using(stub);

DecisionResponse response = byRole.ask(TICKET, TRIAGE);
PropositionResult urgent = response.answer(URGENT);
ClassificationResult department = response.answer(DEPARTMENT);

A manually implemented Ai inherits selectors backed by an empty registry. Only using(service) succeeds until the implementation overrides decisions() and classifications() to select from its populated registry. SpringContextPlatformServices also uses an empty registry when the context has no DecisionServiceRegistry bean. The platform auto-configuration supplies that bean; an application building its own platform services can override decisionServices().

Inside a workflow each terminal returns an operation-bound service. It keeps the selected service’s name, provider, model family and capabilities. Each call on it runs under the observation that was current when the service was selected, including calls made from another thread. The binding creates no observation of its own.

Roles are scoped to one family. A role bound under embabel.models.decision.roles is unknown to classifications().byRole(...), and the reverse holds too. The classification family holds every registered service, because a decision service also classifies. A decision service selected through classifications() keeps the model family DECISION.

ServiceSelectionException is an IllegalStateException. Its reason is one of the ServiceSelectionException.Reason values in the table above. Its message names the requested name or role, the services eligible for the family, the family’s role bindings, the family default or the candidates considered, and the fix. The fix is the property to set, such as embabel.models.decision.roles.<role>, and the DecisionServiceRegistry.Builder method for plain Java.

A decision service registry with no services backs the default Ai implementation. There, using(service) works and the other terminals throw ServiceSelectionException. No terminal falls back to an LLM.

Classification only

An application that only sorts text into categories needs Ai.classifications() and a ClassificationSpec. Select the service by role, as the family default, by registration name, or pass an instance:

ClassificationService byRole = ai.classifications().byRole("ticket-routing");
ClassificationService byDefault = ai.classifications().defaultService();
ClassificationService byName = ai.classifications().named("ticket-classifier");
ClassificationService supplied = ai.classifications().using(new KeywordClassifier());

Describe the categories once in a ClassificationSpec:

static final ClassificationSpec DEPARTMENTS = ClassificationSpec.builder()
    .asking("Which team should handle this?")
    .category("billing", "Payments, invoicing, refunds")
    .category("technical", "Bugs, outages, integrations")
    .build();

Classify with the spec and switch over the four outcomes:

static String queueFor(Ai ai, String ticketText) {
    ClassificationService classifier = ai.classifications().byRole("ticket-routing");
    return switch (classifier.classify(ticketText, DEPARTMENTS)) {
        case ClassificationResult.Selected selected -> selected.getCategoryId();
        case ClassificationResult.NoMatch noMatch -> "general";
        case ClassificationResult.Inconclusive inconclusive -> "triage";
        case ClassificationResult.Failure failure -> "triage";
    };
}

A CategoryMapping maps each category to an enum constant. CategoryMapping.fromEnum builds the spec from the enum, and map turns a selection into the constant:

enum Department { BILLING, TECHNICAL }

static final CategoryMapping<Department> DEPARTMENT_MAPPING = CategoryMapping.fromEnum(
    Department.class,
    "Which team should handle this?",
    department -> switch (department) {
        case BILLING -> "Payments, invoicing, refunds";
        case TECHNICAL -> "Bugs, outages, integrations";
    });

static Optional<Department> departmentFor(Ai ai, String ticketText) {
    ClassificationService classifier = ai.classifications().byRole("ticket-routing");
    ClassificationResult result = classifier.classify(ticketText, DEPARTMENT_MAPPING.spec());
    return switch (DEPARTMENT_MAPPING.map(result)) {
        case MappedClassificationResult.Selected<Department> selected -> Optional.of(selected.getValue());
        case ClassificationResult.NoMatch noMatch -> Optional.empty();
        case ClassificationResult.Inconclusive inconclusive -> Optional.empty();
        case ClassificationResult.Failure failure -> Optional.empty();
    };
}

In Kotlin, classificationSpec { ... } builds the spec and when covers the outcomes:

val departments = classificationSpec {
    asking("Which team should handle this?")
    category("billing", "Payments, invoicing, refunds")
    category("technical", "Bugs, outages, integrations")
}

fun queueFor(ai: Ai, ticketText: String): String {
    val classifier = ai.classifications().byRole("ticket-routing")
    return when (val result = classifier.classify(ticketText, departments)) {
        is ClassificationResult.Selected -> result.categoryId
        is ClassificationResult.NoMatch -> "general"
        is ClassificationResult.Inconclusive, is ClassificationResult.Failure -> "triage"
    }
}

In a Spring application the spec can be a bean:

@Configuration(proxyBeanMethods = false)
static class TicketRoutingConfiguration {

    @Bean
    ClassificationSpec departments() {
        return ClassificationSpec.builder()
                .asking("Which team should handle this?")
                .category("billing", "Payments, invoicing, refunds")
                .category("technical", "Bugs, outages, integrations")
                .build();
    }
}

To use a prompted service for classification, declare it under embabel.agent.platform.decisions.llm.services.<name> and bind it to a role under embabel.models.classification.roles.<role>. The ticket-classifier entry in the configuration example below does both. A decision service also classifies, so a classification role can name it.

A ClassificationSpec is a decision spec with one choice question, so a decision service answers it too.

Registry

DecisionServiceRegistry holds the registered services, the family defaults and the role bindings. DecisionServiceRegistry.builder() starts a builder:

build() throws IllegalArgumentException for a blank or repeated name, for one service instance registered under two names, and for a default, role or candidate that names a missing or ineligible service. The message names the binding and the property that sets it. DecisionServiceRegistry.empty() returns a registry with no services.

decisions() and classifications() return the two selectors. registrationNames() lists the registration names in registration order. listServices() returns one metadata snapshot per registered service, each with its model family.

The observation registry passed to the builder must be the one the registered services observe with. Operation bindings open scopes on it. With another registry, provider calls lose their parent observation.

Family default

A family default resolves in this order:

1. The explicit default, from embabel.models.decision.default or decisionDefault(name).
2. The single default candidate eligible for the family.
3. The single service eligible for the family.

Otherwise defaultService() throws ServiceSelectionException. No eligible service gives NO_DEFAULT. Several eligible candidates, or several eligible services with no candidate, give AMBIGUOUS_DEFAULT. The classification family resolves its default the same way with embabel.models.classification.default.

The TypeSafe auto-configuration contributes DecisionServiceRegistry.DefaultCandidate("typeSafeDecisionService"). When TypeSafe is configured and no explicit default is set, Jev is the default of both families. Prompted LLM services contribute no candidate. An application that has only prompted services and several of them sets the default explicitly.

Startup snapshot

A registry is an immutable snapshot taken when build() runs. In Spring that happens when the decisionServiceRegistry bean is created. The registry holds the ClassificationService beans defined by then: bean definitions, including those the prompted and TypeSafe configuration adds, and singletons registered before the registry bean is created. A service registered later is not visible, and named reports UNKNOWN_NAME for it. Pass a service built at run time, such as a per-user service, to using(service).

Configuration

These properties configure services and selection together:

The registration name of a Spring service is its bean name. The service’s own name is its model name. Named Jev services share the credential, base URL and response limit set under embabel.agent.platform.models.typesafe. The default typeSafeDecisionService bean stays registered next to them.

embabel:
  agent:
    platform:
      models:
        typesafe:
          api-key: $\{TYPESAFE_API_KEY}
          services:
            jev:
              model: jev-latest
            jev-fast:
              model: jev-fast
      decisions:
        llm:
          services:
            llm-review:
              llm: gpt-4.1-mini
            ticket-classifier:
              llm: gpt-4.1-mini
        capture-content: false
  models:
    decision:
      roles:
        support-triage: jev
        dice-revision-review: llm-review
    classification:
      roles:
        dice-revision: llm-review
        ticket-routing: ticket-classifier

Startup fails when a default or role names a missing service, or when a decision default or role names a classification-only service. The error names the property. Startup also fails when a Jev entry has a blank model or an unknown key, naming the property. A default that is missing or ambiguous fails at lookup, when defaultService() is called.

An application bean replaces configuration in these cases:

Plain Java

Plain Java builds Jev services with TypeSafeModelFactory, no-op and stub services with their constructors and builders, and the registry with DecisionServiceRegistry.builder():

// Every Jev service built by one factory shares its credential, transport and observations.
var jevFactory = new TypeSafeModelFactory(
        TypeSafeClientOptions.defaults(),
        () -> System.getenv("TYPESAFE_API_KEY"),
        observations);

// Plain Java cannot build a prompted LLM service: it needs the platform's LLM operations.
return DecisionServiceRegistry.builder()
        .register("typeSafeDecisionService", jevFactory.build())
        .register("jev", jevFactory.build("jev-latest"))
        .register("jev-fast", jevFactory.build("jev-fast"))
        .register("triage-stub", StubDecisionService.builder("triage-stub").build())
        .register("disabled", new NoOpDecisionService("disabled"))
        .defaultCandidate("typeSafeDecisionService")
        .decisionRole("support-triage", "jev")
        .decisionRole("tests", "triage-stub")
        .classificationRole("dice-revision", "jev")
        // Pass the observation registry the services observe with.
        .observationRegistry(observations)
        .build();

A prompted service needs the platform’s LLM operations, an internal platform interface held by a Spring-built agent platform. Plain Java code cannot build one. An application with a Spring context can take services from the LlmDecisionServiceFactory bean and register them in a registry it builds.

Asking

DecisionService has two ask overloads: ask(input, spec) and ask(request). Both return a DecisionResponse with one answer per question in spec order. Provider failures come back as typed failure outcomes.

A service that implements QuestionSetExecution answers the whole spec in one askQuestionSet call. The provider receives the complete question set in one operation; a prompted LLM service sends one prompt containing all the questions. This hook does not select the LLM’s structured-output mode. Any other service answers each question in its own call, in spec order.

DecisionRequest request = DecisionRequest.of(TICKET, URGENT, DEPARTMENT);

// The triage stub answers the whole request in one question-set call.
DecisionService triage = ai.decisions().byRole("support-triage");
DecisionResponse shared = triage.ask(request);

// This stub answers one question per call, through assess and classify, in spec order.
StubDecisionService perQuestion = StubDecisionService.builder("triage-per-question")
    .proposition("urgent", URGENT_ANSWER)
    .choice("department", BILLING)
    .perQuestion()
    .build();
DecisionResponse separate = perQuestion.ask(request);

ask, assess and classify throw no checked exception. An interruption surfaces as an unchecked java.util.concurrent.CancellationException whose cause is the InterruptedException, and the thread’s interrupt flag stays set. Catch CancellationException where the caller needs to stop cleanly.

Capabilities and hooks

DecisionService.capabilities() returns the DecisionCapabilities the service reports. DecisionCapabilities is a public, JSON-serializable description of the question kinds a service accepts. The prompted adapter reports all three kinds; its configuration does not override that set. A capability declaration must be backed by the service’s execution hooks.

The default derives from the hook interfaces the service implements:

The default always includes proposition and choice questions. Every decision service is a classification service, so it can answer a choice question through classify. A service declares other kinds by overriding capabilities(). The hook interfaces declare no default methods, so a class can implement all three.

A service that implements none of the hooks answers proposition questions through ask, one assess call per proposition, with the question’s instructions as the proposition. It answers each choice question with one classify call. A rating question on such a service fails the preflight. Its classify and assess methods stay directly usable.

A choice question is a classification request: ClassificationRequest.of(input, ClassificationSpec.of(question)). The provider sees the question’s own instructions and categories. A ClassificationRequest is also a DecisionRequest, so ask(request) accepts one and response.answer(spec.getQuestion()) returns the classification result.

Routing

Execution planning here validates the route for a decision request before any provider call. It does not maintain a proposition store or construct the agent’s GOAP plan.

A service that implements QuestionSetExecution answers the whole spec in one askQuestionSet call. Any other service answers each question in spec order:

A typed failure is recorded and the next question is asked. An IllegalArgumentException from classify or rate, or from checking its answer against the question, becomes that question’s INVALID_RESPONSE failure. An IllegalArgumentException from either proposition assess overload also propagates unchanged. Proposition answers have no option or level id to validate, so providers report an unusable proposition reply as PropositionResult.Failure(INVALID_RESPONSE). Other operational failures also use typed Failure outcomes. Throwing from assess stops the request. Any other exception stops the request and propagates unchanged. The thread’s interrupt flag is checked before each question, and a set flag stops the request with a CancellationException caused by an InterruptedException.

Preflight

The whole request is checked before any provider call. The checks run in this order: every question kind is in the capabilities, every question has a hook, then a decorator implements every hook the request is routed through.

A request that fails the kind check throws UnsupportedDecisionException, an UnsupportedOperationException. Its message names the service, each question concerned with its kind, what is missing, the service’s capabilities and a remedy. A kind missing from the capabilities gets one of two explanations:

• The service lacks the hook for the kind. Only rating questions need a hook. The message says the question needs RatingAssessment, with the remedy Use a service that implements RatingAssessment, or remove these questions.
• The service can answer the kind, and its capabilities leave the kind out. For a per-question service the message names the method that backs the question: assess, PropositionAssessment, classify or RatingAssessment. The remedy is Report CHOICE in the service’s capabilities(), use a service whose capabilities include CHOICE, or remove these questions.

The message holds question names and kinds only. The input, instructions, options and levels stay out of it.

Capabilities that claim rating questions on a service that implements neither RatingAssessment nor QuestionSetExecution throw IllegalStateException before any provider call. The message names the service and the missing hook interface.

Decorators

ObservedDecisionService and the operation binding forward capabilities(), classify and every hook to their delegate. A decorator adds no kind. Both implement DelegatingDecisionService, so preflight inspects the hooks of the innermost service that does the work. Routing follows the hooks of that innermost service, and execution calls those hooks on the decorator. A decorator that reports a hook source implements every hook interface the hook source implements. When a request is routed through a hook the decorator lacks, preflight throws IllegalStateException before any provider call. A choice question needs no hook on the decorator, because execution calls the decorator’s own classify. The message names the decorator, its class, the hook source and the missing hook, and asks for the hook to be implemented on the decorator. ObservedDecisionService.ask runs the shared preflight and execution with the delegate’s hooks, so each provider call is observed once. A delegate’s own override of ask is therefore not called through ObservedDecisionService. A service customizes execution through capabilities() and the hook interfaces. Provider implementations put their execution logic in askQuestionSet, assess, classify or rate. Overriding ask does not customize execution through the observation wrapper.

Rating and ranking

RatingResult records evidence for one input against a caller-defined ordered scale, such as low, medium and high severity. It can carry a selected level, a distribution over levels, or a score with a named statistic, as well as inconclusive and failure outcomes. The prompted adapter supplies a selected level or an inconclusive outcome without inventing numeric evidence. The existing Ranking<T> scores candidates so Rankings<T> can order them, as Ranker does when choosing among agents.

Answer rules

A question-set provider must return a response for the request’s spec. Execution calls response.requireMatches(request.spec) before accepting it. Names, kinds and options or levels must match one for one in spec order; a mismatch throws IllegalStateException. Providers can use DecisionResponse.builder(request.spec) or assemble wire answers through DecisionResponseAssembler, which applies these rules:

A failed question leaves the answers to the other questions intact. An answer with an unknown name is ignored because it cannot change which answer a spec question receives. A repeated name can, so it fails the request.

Each anomaly is logged once per response at WARN with the service name, the spec question names and a fixed DecisionResponseAssembler.Anomaly kind: MISSING, WRONG_KIND, UNREADABLE, OUT_OF_DOMAIN, OUT_OF_RANGE, BAD_DISTRIBUTION, DUPLICATE, UNEXPECTED or UNSAFE_ENVELOPE. The line counts unknown answers and holds none of the keys, ids or outcomes received.

Both providers send questions under the keys q1 to qN in spec order. Question names stay with the caller, so no name character can break a provider’s format.

Service families

TypeSafe is a separate provider integration module, embabel-agent-typesafe, with its public factory and package-private adapter in com.embabel.agent.typesafe. This follows the provider-module layout used by com.embabel.agent.openai and com.embabel.agent.anthropic. The shared contracts live in com.embabel.common.ai.decision; Spring configuration is under com.embabel.agent.config.models.typesafe.

DecisionService has four implementations:

Jev and prompted services support every kind and answer a whole spec in one question-set call. They also implement the proposition and rating hooks for direct calls. Their classify answers one choice question with its own instructions and categories.

NoOpDecisionService(name) serves disabled configuration. Its provider is none, and it makes no provider call. It supports every kind and answers a whole spec in one question-set call, so a request reaches it and returns a request failure. Construction logs one INFO line, and each request logs the usual WARN line for a failed request.

StubDecisionService.builder(name) builds a service for tests. Its provider is stub. Outcomes for ask, rate and question assess calls are scripted by question name, and for assess(request) calls by proposition text. classify returns the choice scripted for the request’s question name, and otherwise the outcome scripted for its set of category ids. A call with no scripted outcome throws IllegalStateException naming the question. By default the stub answers a whole spec in one question-set call. Scripted question-set answers go through the same question checks as provider answers. After perQuestion() the builder builds a stub that answers each question through PropositionAssessment, classify and RatingAssessment. The stub supports every kind unless the builder’s capabilities(...) sets others. calls() returns the calls made, in order.

TypeSafe evidence

The Score value is the probability-weighted mean of the zero-based level positions. In typesafe-java-sdk 0.2.0, Score makes level 0 the first criteria entry, and ScoreAnswer keys its legend and probabilities by level index. The value 1.6 on the levels Calm, Frustrated and Very angry lies between the second and third level. A rated answer from Jev has no selected level. The confidence of a Score describes how concentrated the distribution is.

An unknown answer type is an unreadable answer. A value outside its range, or a distribution that fails its checks, fails only that question.

Prompted evidence

A prompted service asks the whole spec in one chat model call and reads a JSON array of answers. A proposition answer is TRUE, FALSE or INCONCLUSIVE. A choice answer is SELECTED with a category id, NO_MATCH or INCONCLUSIVE. A rating answer is RATED with a level id, or INCONCLUSIVE. Prompted outcomes carry no confidence, distribution or score.

The service’s retry settings apply to the model call. Parsing runs after the call returns, so a reply that cannot be read makes one call and returns INVALID_RESPONSE.

Prompted response parsing

The prompt uses q1..qN aliases and maps returned keys back to caller-owned questions, regardless of answer order. One malformed array element that cannot be matched safely rejects the whole envelope; it is never silently dropped. For choice and rating answers, an absent, null or blank id means no selection, while a non-string id is unreadable. SELECTED and RATED require an id from the question’s options or scale; NO_MATCH and INCONCLUSIVE require no id.

JsonResponseText in the common AI module unwraps one Markdown fence covering the complete reply, optionally tagged json. It leaves surrounding prose, other language tags and multiple or nested fences unchanged for the strict parser to reject.

Examples

The support-triage example registers a stub under the name triage-stub and the role support-triage. The routing policy lives in the application. Selected, NoMatch and Inconclusive are model judgments that the application maps to business routes. Failure reports an operational or invalid-response problem. The converter throws an application exception carrying its reason. The application handles that exception separately and chooses whether to retry or send the ticket for review. Confidence is optional and does not select a route.

data class Route(val queue: String, val sameDay: Boolean, val frustrationConfidence: Double?)

class DecisionUnavailableException(val reason: FailureReason) : IllegalStateException("Decision failed: $reason")

val triage = decisionSpec {
    proposition("urgent") { asking("Does the customer need help today?") }
    choice("department") {
        asking("Which team should handle the ticket?")
        option("billing", "Payments, invoices and refunds")
        option("technical", "Errors, outages and integrations")
    }
    rating("frustration") {
        asking("How frustrated is the customer?")
        level("calm")
        level("frustrated")
        level("very-angry")
    }
}

fun route(response: DecisionResponse): Route {
    val urgent = response.answer("urgent") as DecisionAnswer.Proposition
    val department = response.answer("department") as DecisionAnswer.Choice
    val frustration = response.answer("frustration") as DecisionAnswer.Rating
    val sameDay = when (val outcome = urgent.outcome) {
        is PropositionResult.Answered -> outcome.answer
        is PropositionResult.Inconclusive -> true
        is PropositionResult.Failure -> throw DecisionUnavailableException(outcome.reason)
    }
    val queue = when (val outcome = department.outcome) {
        is ClassificationResult.Selected -> outcome.categoryId
        is ClassificationResult.NoMatch -> "general"
        is ClassificationResult.Inconclusive -> "human-review"
        is ClassificationResult.Failure -> throw DecisionUnavailableException(outcome.reason)
    }
    // Confidence is optional. It is shown to the agent and does not change the route.
    val confidence = when (val outcome = frustration.outcome) {
        is RatingResult.Answered -> outcome.confidence
        is RatingResult.Inconclusive -> null
        is RatingResult.Failure -> throw DecisionUnavailableException(outcome.reason)
    }
    return Route(queue, sameDay, confidence)
}

fun triageTicket(ai: Ai, ticket: String): Route {
    val response = ai.decisions().byRole("support-triage").ask(ticket, triage)
    return try {
        route(response)
    } catch (failure: DecisionUnavailableException) {
        // Application policy schedules another attempt and same-day review.
        Route("retry-later", sameDay = true, frustrationConfidence = null)
    }
}

A Spring Converter maps a response to an application route:

record SupportRoute(String queue, boolean sameDay, Double confidence) {
}

static final class DecisionUnavailableException extends IllegalStateException {
    private final FailureReason reason;

    DecisionUnavailableException(FailureReason reason) {
        super("Decision failed: " + reason);
        this.reason = reason;
    }

    FailureReason reason() {
        return reason;
    }
}

static final class SupportRouteConverter implements Converter<DecisionResponse, SupportRoute> {

    @Override
    public SupportRoute convert(DecisionResponse response) {
        boolean sameDay = switch (response.answer(URGENT)) {
            case PropositionResult.Answered answered -> answered.getAnswer();
            // Without an answer, a person looks at the ticket the same day.
            case PropositionResult.Inconclusive inconclusive -> true;
            case PropositionResult.Failure failure -> throw new DecisionUnavailableException(failure.getReason());
        };
        // Confidence is carried for display when the provider reports it. Routing does not read it.
        return switch (response.answer(DEPARTMENT)) {
            case ClassificationResult.Selected selected ->
                new SupportRoute(selected.getCategoryId(), sameDay, selected.getConfidence());
            case ClassificationResult.NoMatch noMatch -> new SupportRoute("general", sameDay, null);
            case ClassificationResult.Inconclusive inconclusive -> new SupportRoute("human-review", sameDay, null);
            case ClassificationResult.Failure failure -> throw new DecisionUnavailableException(failure.getReason());
        };
    }
}

The retry policy handles operational failure outside the converter:

static SupportRoute routeOrRetry(DecisionResponse response) {
    try {
        return new SupportRouteConverter().convert(response);
    } catch (DecisionUnavailableException failure) {
        // Application policy schedules another attempt and same-day review.
        return new SupportRoute("retry-later", true, null);
    }
}

For an object containing only answered values, use DecisionProjection.of(response, type). Projection retains the original response and refuses outcomes with no representable value, including NoMatch, Inconclusive and Failure. A rating with only a score or distribution also has no selected level to project.

record AnsweredTriage(boolean urgent, String department) {
}

@Test
void projectionCreatesAnObjectOnlyWhenEveryAnswerHasAValue() {
    DecisionResponse response = respond(URGENT_ANSWER, BILLING);
    DecisionProjection<AnsweredTriage> projected = DecisionProjection.of(response, AnsweredTriage.class);
    assertEquals(new AnsweredTriage(true, "billing"), projected.getValue());
    assertEquals(response, projected.getResponse());

    for (DecisionResponse unresolved : List.of(
            respond(new PropositionResult.Inconclusive(MODEL), BILLING),
            respond(URGENT_ANSWER, new ClassificationResult.NoMatch(MODEL)),
            respond(URGENT_ANSWER, new ClassificationResult.Inconclusive(MODEL)),
            DecisionResponse.failed(TRIAGE, FailureReason.UNAVAILABLE))) {
        assertThrows(DecisionProjectionException.class,
            () -> DecisionProjection.of(unresolved, AnsweredTriage.class));
    }
}

The application stores the response JSON and each answer’s provenance:

record AnswerProvenance(String question, String outcome, String modelName, String provider) {
}

record TriageEvidence(
    String responseJson,
    List<AnswerProvenance> answers) {
}

private final JsonMapper mapper = JsonMapper.builder().build();

private final List<TriageEvidence> evidenceStore = new ArrayList<>();

TriageEvidence persist(DecisionResponse response) {
    List<AnswerProvenance> answers = response.getAnswers().stream()
        .map(SupportTriageExampleTest::provenanceOf)
        .toList();
    TriageEvidence evidence = new TriageEvidence(
        mapper.writeValueAsString(response),
        answers);
    evidenceStore.add(evidence);
    return evidence;
}

static AnswerProvenance provenanceOf(DecisionAnswer answer) {
    Object outcome = switch (answer) {
        case DecisionAnswer.Proposition proposition -> proposition.getOutcome();
        case DecisionAnswer.Choice choice -> choice.getOutcome();
        case DecisionAnswer.Rating rating -> rating.getOutcome();
    };
    ModelProvenance model = switch (outcome) {
        case PropositionResult.Answered answered -> answered.getProvenance();
        case PropositionResult.Inconclusive inconclusive -> inconclusive.getProvenance();
        case ClassificationResult.Selected selected -> selected.getProvenance();
        case ClassificationResult.NoMatch noMatch -> noMatch.getProvenance();
        case ClassificationResult.Inconclusive inconclusive -> inconclusive.getProvenance();
        case RatingResult.Answered answered -> answered.getProvenance();
        case RatingResult.Inconclusive inconclusive -> inconclusive.getProvenance();
        // A failure outcome has no model provenance.
        default -> null;
    };
    return new AnswerProvenance(
        answer.getName(),
        outcome.getClass().getSimpleName(),
        model == null ? null : model.getModelName(),
        model == null ? null : model.getProvider());
}

The grounding example checks a claim against a source excerpt and records the evidence. The policy decides what to persist from the answer kind and its own rules. Inconclusive and Failure are recorded as such.

record GroundingFacts(String claimText, String sourceExcerpt, String sourceId, String sourceRevision) {
}

// Only these fields enter the model input, in this order. Source ids and revisions stay out.
private static final List<Map.Entry<String, Function<GroundingFacts, String>>> INPUT_FIELDS = List.of(
    Map.entry("Claim", GroundingFacts::claimText),
    Map.entry("Source excerpt", GroundingFacts::sourceExcerpt));

static String render(GroundingFacts facts) {
    return INPUT_FIELDS.stream()
        .map(field -> field.getKey() + ": " + field.getValue().apply(facts))
        .collect(Collectors.joining("\n"));
}

static final PropositionQuestionSpec SUPPORTED =
    Questions.named("supported").proposition("Does the source excerpt support the claim?").build();

static final DecisionSpec GROUNDING = DecisionSpec.of(SUPPORTED);

static final String POLICY_VERSION = "grounding-policy-3";

enum Disposition { SUPPORTED, NOT_SUPPORTED, STALE_SOURCE, INCONCLUSIVE, FAILED }

record EvidenceRecord(
    String correlationId,
    String propositionId,
    String inputDigest,
    String sourceLocator,
    String sourceRevision,
    ModelProvenance model,
    Double pTrue,
    String policyVersion,
    Disposition proposedDisposition) {
}

// The policy reads the answer kind and its own source rule. The probability is recorded and not read.
static Disposition decide(PropositionResult outcome, GroundingFacts facts, String currentSourceRevision) {
    boolean sourceCurrent = facts.sourceRevision().equals(currentSourceRevision);
    return switch (outcome) {
        case PropositionResult.Answered answered when !sourceCurrent -> Disposition.STALE_SOURCE;
        case PropositionResult.Answered answered ->
            answered.getAnswer() ? Disposition.SUPPORTED : Disposition.NOT_SUPPORTED;
        case PropositionResult.Inconclusive inconclusive -> Disposition.INCONCLUSIVE;
        case PropositionResult.Failure failure -> Disposition.FAILED;
    };
}

final List<EvidenceRecord> evidenceLog = new ArrayList<>();

final List<String> groundedPropositions = new ArrayList<>();

EvidenceRecord ground(Ai ai, String correlationId, String propositionId, GroundingFacts facts,
                      String currentSourceRevision) {
    String input = render(facts);
    DecisionService grounding = ai.decisions().byRole("dice-grounding");
    DecisionResponse response = grounding.ask(input, GROUNDING);
    PropositionResult outcome = response.answer(SUPPORTED);

    Disposition disposition = decide(outcome, facts, currentSourceRevision);
    EvidenceRecord evidence = new EvidenceRecord(
        correlationId,
        propositionId,
        sha256(input),
        facts.sourceId(),
        facts.sourceRevision(),
        switch (outcome) {
            case PropositionResult.Answered answered -> answered.getProvenance();
            case PropositionResult.Inconclusive inconclusive -> inconclusive.getProvenance();
            case PropositionResult.Failure failure -> null;
        },
        outcome instanceof PropositionResult.Answered answered ? answered.getPTrue() : null,
        POLICY_VERSION,
        disposition);
    evidenceLog.add(evidence);
    // The application writes the grounding link. The decision service writes nothing.
    if (disposition == Disposition.SUPPORTED) {
        groundedPropositions.add(propositionId);
    }
    return evidence;
}

The revision example classifies a new proposition against scoped candidates and rechecks candidate versions before it applies a proposal. A missing or inconclusive answer leaves the proposal unapplied.

data class Candidate(val id: String, val version: Long, val text: String, val pinned: Boolean = false)

data class Proposal(
    val propositionId: String,
    val candidateId: String,
    val candidateVersion: Long,
    val relationId: String,
    val evidenceStrength: String?,
    val confidence: Double?,
    val model: ModelProvenance,
)

sealed interface Classification {
    data class Proposed(val proposals: List<Proposal>) : Classification
    data class NeedsReview(val reasons: List<String>) : Classification

    // Every candidate was answered as unrelated.
    data object New : Classification
}

sealed interface Application {
    data class Applied(val proposal: Proposal) : Application
    data class Invalidated(val proposal: Proposal, val reason: String) : Application
    data class Rejected(val proposal: Proposal, val reason: String) : Application
}

fun relationQuestion(candidate: Candidate) = "relation-$\{candidate.id}"

fun revisionSpec(candidates: List<Candidate>): DecisionSpec = decisionSpec {
    candidates.forEach { candidate ->
        choice(relationQuestion(candidate)) {
            asking("How does the new proposition relate to candidate $\{candidate.id}?")
            option("duplicate", "States the same fact")
            option("reinforces", "Adds support for the same fact")
            option("contradicts", "Cannot be true at the same time")
            option("unrelated", "Concerns a different fact")
        }
    }
    rating("evidence-strength") {
        asking("How strong is the evidence in the new proposition?")
        level("weak")
        level("moderate")
        level("strong")
    }
}

fun render(proposition: String, candidates: List<Candidate>): String =
    "New proposition: $proposition\n" + candidates.joinToString("\n") { "Candidate $\{it.id}: $\{it.text}" }

val proposalStore = mutableListOf<Proposal>()
val appliedRevisions = mutableListOf<Proposal>()

fun classify(propositionId: String, candidates: List<Candidate>, response: DecisionResponse): Classification {
    val strength = response.answers
        .filterIsInstance<DecisionAnswer.Rating>()
        .firstOrNull { it.name == "evidence-strength" }
        ?.let { (it.outcome as? RatingResult.Answered)?.selectedLevelId }
    val proposals = mutableListOf<Proposal>()
    val reasons = mutableListOf<String>()
    for (candidate in candidates) {
        val answer = response.answers
            .filterIsInstance<DecisionAnswer.Choice>()
            .firstOrNull { it.name == relationQuestion(candidate) }
        when (val outcome = answer?.outcome) {
            null -> reasons += "$\{candidate.id}: no answer"
            is ClassificationResult.Selected -> if (outcome.categoryId != "unrelated") {
                proposals += Proposal(
                    propositionId, candidate.id, candidate.version, outcome.categoryId,
                    strength, outcome.confidence, outcome.provenance,
                )
            }
            is ClassificationResult.NoMatch -> reasons += "$\{candidate.id}: no relation fits"
            is ClassificationResult.Inconclusive -> reasons += "$\{candidate.id}: inconclusive"
            is ClassificationResult.Failure -> reasons += "$\{candidate.id}: failed with $\{outcome.reason}"
        }
    }
    return when {
        reasons.isNotEmpty() -> Classification.NeedsReview(reasons)
        proposals.isEmpty() -> Classification.New
        else -> Classification.Proposed(proposals).also { proposalStore += proposals }
    }
}

fun classifyNew(ai: Ai, propositionId: String, text: String, candidates: List<Candidate>): Classification {
    val response = ai.decisions().byRole("dice-revision").ask(render(text, candidates), revisionSpec(candidates))
    return classify(propositionId, candidates, response)
}

// Runs in the application's transaction against the current candidate state.
fun apply(proposal: Proposal, current: Map<String, Candidate>): Application {
    val candidate = current[proposal.candidateId]
        ?: return Application.Invalidated(proposal, "candidate is no longer in scope")
    if (candidate.version != proposal.candidateVersion) {
        return Application.Invalidated(
            proposal,
            "candidate changed from version $\{proposal.candidateVersion} to $\{candidate.version}",
        )
    }
    if (proposal.relationId == "contradicts" && candidate.pinned) {
        return Application.Rejected(proposal, "a pinned candidate is not demoted")
    }
    if (proposal.relationId == "contradicts" && proposal.evidenceStrength != "strong") {
        return Application.Rejected(proposal, "demotion needs strong evidence")
    }
    appliedRevisions += proposal
    return Application.Applied(proposal)
}

In both examples the application authorizes every change. The framework returns evidence.

Observations

ObservedDecisionService records one logical observation per ask and one observation per provider call. The TypeSafe and LLM factories return observed services. Inside a workflow, using(service) wraps a service that is not already observed, so its asks are observed too. Outside a workflow, wrap a service you build in ObservedDecisionService(service, observationRegistry) to observe it.

Provider-call observations are children of the ask observation. When the Spring context has several ObservationRegistry beans, mark one @Primary. If none is primary, startup logs a WARN and decision observations use ObservationRegistry.NOOP. Inside a workflow the ask observation is a child of the action’s observation. A question-set ask records one ask and one ask_question_set observation. A per-question ask of three questions records one ask and three provider observations. A choice question among them records embabel.ai.classification with operation=classify. A request rejected by the preflight records one ask with outcome=unsupported and no provider observation.

partial means at least one question failed. request_failure and every thrown outcome mark the observation with a fixed error marker that carries no exception text.

Each answer event is named answer.<kind>.<outcome>, with the kind proposition, choice or rating and the outcome answered, selected, no_match, inconclusive or failure. A meter handler counts it as embabel.ai.ask.answer.<kind>.<outcome> with the ask’s tags. A tracing handler records its contextual name, <question name> <kind> <outcome>, as a span event.

service and provider come from configuration and code. The registry’s INFO summary maps each registration name to its service name and provider. Name services from configuration only. A service name taken from request or user data makes the tag unbounded. Question names, roles, input and model output never become tag values.

Diagnosing

TransportFailureDiagnostics in the common AI module categorizes HTTP failures, rate limits, connection failures and timeouts for logs. It traverses wrapped causes once and uses other for unknown exceptions. The decision adapter adds invalid-answer recognition; result mapping and retry policy remain separate from these diagnostic labels. A failed model call with interruption evidence escapes the retry callback as DecisionInterrupted, then becomes a CancellationException. An interrupted retry wait reaches the separate cause-chain check; both paths preserve the interrupt flag. problems

Loggers

Loggers are named after their classes. Setting a package level reaches every class under it.

A provider WARN line holds the service name, provider, operation, failure reason, cause category, HTTP status class or none, attempts, elapsed milliseconds and the exception’s simple class name. The cause category is one of timeout, connection, rate_limited, http_4xx, http_5xx, invalid_response and other. The Jev line starts with TypeSafe <operation label> failed with reason <reason>, makes one attempt, and ends with the provider request id or none. The prompted line starts with Decision call failed, counts the attempts made by its retry settings, and ends with a remedy that names the retry property prefix. A one-question call logs its operation: assess, classify or rate. Lines up to DEBUG hold no input, instructions, option or level text, received category ids, credentials or provider text. Question names and registration names appear in logs, exception messages and span events.

Sample lines

The registry summary at INFO:

Decision service registry built with 3 services: [typeSafeDecisionService (name jev-1, provider TypeSafe, type DECISION, capabilities kinds [PROPOSITION, CHOICE, RATING]), gpt (name gpt-x, provider OpenAI, type DECISION, capabilities kinds [PROPOSITION, CHOICE]), clf (name clf-1, provider acme, type CLASSIFICATION)]; decision default: 'typeSafeDecisionService' (explicit); decision roles: \{support-triage=gpt}; classification default: none set; candidates considered: []; classification roles: {}; default candidates: []

Routing and completion at DEBUG:

Decision ask started: service=log-service, provider=log-provider, questions=3
Decision ask completed: service=log-service, provider=log-provider, answers=['urgent' PROPOSITION answered, 'team' CHOICE selected, 'anger' RATING answered], elapsedMs=4

A failed request and a partial response at WARN:

Decision ask failed: service=question-set-service, provider=question-set-provider, requestFailure=UNAVAILABLE, elapsedMs=2. The provider's own log lines give the cause.
Decision ask partially failed: service=log-service, provider=log-provider, failedQuestions=['urgent' UNAVAILABLE], elapsedMs=3. The other answers are usable. The provider's own log lines give the cause.

Failed provider calls at WARN, from Jev and from a prompted service:

TypeSafe question set failed with reason UNAVAILABLE: service=jev-latest, provider=TypeSafe, operation=ask_question_set, cause=http_5xx, status=5xx, attempts=1, elapsedMs=7, exception=SafeHttpFailure, requestId=none
Decision call failed: service=gpt-test, provider=TestProvider, operation=ask, reason=UNAVAILABLE, cause=timeout, httpStatus=none, attempts=3, elapsedMs=12, exception=SocketTimeoutException. Check the model's availability, credentials and retry settings under embabel.agent.platform.decisions.test.

Answer anomalies at WARN, from question-set and per-question execution:

Decision service 'jev-latest' returned answers it could not use: is_urgent=WRONG_KIND, department=OUT_OF_DOMAIN, frustration=OUT_OF_RANGE, channel=MISSING; unexpected answers: 1
Decision answer anomaly: service=log-service, question='team', anomaly=OUT_OF_DOMAIN. The answer does not fit the question's options or levels and is recorded as INVALID_RESPONSE. Check the service's mapping of provider output to the question.

Content capture

Content capture logs the request and response content at TRACE. It needs two settings together:

• embabel.agent.platform.decisions.capture-content: true, or DecisionContentCapture.enable() in plain Java
• TRACE on the logger concerned

The switch alone logs nothing, and TRACE alone logs no content. With both, DecisionExecution logs the input, each question’s instructions, options or levels, and each answer with its evidence. The Jev service logs the response it received, and the prompted service logs the raw model reply. Spring logs one WARN at startup when the property is on and turns the switch off when the context closes.

Turn content capture on only for local diagnosis. DecisionContentCapture is a process-wide switch.

Experimental status

Every type on this page carries @ApiStatus.Experimental. The owner is James Dunnam (GitHub @jimador). Promotion to stable needs all of these:

• Service resolution by role, name, default and instance, used by a consumer application
• Interoperation of the Jev, prompted, no-op and stub families through one registry
• A compatibility review of DecisionService, Ai and PlatformServices
• Privacy checks on logs, tags and exception messages
• A runnable consumer proof
• A recorded run against the hosted Jev service

Promotion is revisited at the next release review after the consumer proof.