LLM Integration (LLM)

The umbrella product/feature whose agents share a base system prompt.

A configured caller within an application; runs actions to achieve a goal.

Classification used to group agents (for example "human", "autonomous", "review").

A repeatable LLM task. An action bundles all the metadata Spring AI needs to build a ChatClient: the model, configuration, persona, memory strategy, prompt text, plus the link tables for schemas, skills, tools, MCP clients, document collections, and advisors.

The hand-written entity adds the helpers getInputSchemas() (returns input/memory schemas) and getOutputAgentSchema() (returns the single output schema or raises a business exception if zero or many exist).

Classification with an optional global constraint prompt that is added to every action of that type.

Links an action to a DomainSchema for either input, output, or memory data, with cardinality and optional UI hints.

A validation rule enforces that, when a domainForm or domainGrid is given, it must reference the same domainSchema as this row.

These are pure link tables that attach reusable capabilities to an action.

Persona provides the voice/role prompt fragment.

Skill is a reusable capability prompt added to actions through LlmActionSkill.

Tool maps a Spring bean to a function the LLM can call.

MCP client describes a Model-Context-Protocol server connection that exposes one or more tools.

Advisor declares a Spring AI advisor bean that can be inserted into the chat-client pipeline of an action.

These three entities configure RAG sources.

Memory strategy controls how chat history is retained for a conversation.

Sampling, temperature, token-limit, and retry settings applied when building chat options.

Model identifies a specific LLM offered by a provider; provider is the vendor (Anthropic, OpenAI, …​).

Lists what a model is capable of (link table on the model, keyed by capability type).

One running instance of an agent calling an action. The conversation aggregates its messages, attachments, and tool calls and tracks lifecycle, finish reason, token usage, estimated cost, and any error detail.

One entry in a conversation’s transcript. Messages are recorded as immutable events ordered by messageSequence.

A file attached to a message (image, document, …​). Stored as an immutable event.

A function call the LLM emitted during a message.

LlmPromptAssemblyService builds the system prompt that is sent on every conversation. It concatenates contributions from the agent’s application, the agent itself, the action’s persona, every linked skill, every linked MCP client, the schema-prompt block (input/output data structures), and finally the action’s own prompt. Each contribution comes from IsPromptContributor.getPromptContribution() — implemented by LlmApplication, LlmAgent, LlmPersona, LlmSkill, McpClient, and LlmAction. Skills and MCP entries get short labels (Your skill: …​, Available tool '<name>': …​) before their text so the LLM can distinguish them.

LlmActionSchemaPromptService walks the action’s LlmActionSchema rows and produces an "Input data structure:" / "Output data structure:" block describing each schema’s fields. It is used as one segment of the assembled system prompt so the model knows the shape of the data it must consume and produce.

LlmChatOptionsFactory translates an action’s LlmConfiguration into a Spring AI ChatOptions (model name, max tokens, temperature, top-K, top-P, frequency penalty, presence penalty). Optional values are only set on the builder when present.

LlmChatClientFactory is the central wiring point. For each conversation it produces a Spring AI ChatClient configured with: the model bean for the action’s model, the assembled system prompt, the chat options, the resolved tool callbacks plus a per-conversation AskUserQuestion tool, the resolved advisors (action advisors + RAG advisors + a chat-memory advisor when a memory strategy is set), and any MCP tool callbacks. The resulting client is what LlmConversationService calls into.

LlmToolCallbackResolver looks up each LlmActionTool by its llmToolBeanName in the Spring application context and verifies that the bean is a ToolCallback. It supports an exclude set so the per-conversation AskUserQuestion callback can be inserted separately.

LlmAdvisorResolver reads LlmActionAdvisor rows ordered by llmAdvisorOrder and looks up each llmAdvisorBeanName in the Spring context. The resulting list of Advisor instances is added to the chat-client pipeline.

LlmDocumentCollectionResolver walks the action’s LlmActionDocument rows. For each linked LlmDocumentCollection it looks up the matching VectorStore bean (by the vector store’s code) and creates a QuestionAnswerAdvisor configured with the collection’s similarity threshold and result count. The default top-K is 4 when not specified on the collection.

LlmMcpClientResolver opens (and caches) one McpSyncClient per McpClient entity, choosing the transport (stdio, streamable HTTP, or SSE) based on mcpTransportType and the mcpConnectionUrl. Each connected client is wrapped in a SyncMcpToolCallbackProvider whose tool callbacks are merged into the chat client. Connections are kept across conversations and closed during shutdown.

LlmMemoryService materializes a ChatMemory per conversation based on the action’s LlmMemoryStrategy. FULL strategy uses an unbounded MessageWindowChatMemory; any other strategy uses a window of llmMemoryWindowSize messages. Memories are cached by conversation id and can be cleared explicitly.

LlmConversationService is the primary entry point for running an action. startConversation creates a LlmConversation row with a fresh GUID. sendMessage / sendMessageWithAttachments records the user message (and any attachments), flips the conversation to RUNNING, builds a chat client, and calls the model. On success it records the assistant message, persists every emitted tool call, captures token usage and estimated cost (using the model’s input/output token prices), and marks the conversation COMPLETED with the appropriate finish reason. On failure it records the error detail and marks the conversation FAILED.

LlmAskUserQuestionService together with ErpAskUserQuestionHandler implements the AskUserQuestion pattern. When the LLM calls the AskUserQuestion tool, the handler thread blocks on a CompletableFuture for up to five minutes while the conversation status is set to WAITING and a question message (role QU, falling back to TO if the generator has not yet produced the QU code) is persisted. A separate API call (submitAnswers) resolves the future and unblocks the handler so the LLM can continue. Each conversation gets its own handler instance, bound to its conversation id.

Read-side facade used by other modules and the UI.

Write-side facade used by other modules and the UI.

IsPromptContributor is the public seam for entities that contribute a fragment to the assembled system prompt. The submodule itself implements it on LlmApplication, LlmAgent, LlmPersona, LlmAction, LlmSkill, and McpClient. Other modules that introduce their own LLM-aware entities can implement the same interface to plug into prompt assembly.

ErpAskUserQuestionHandler implements the framework’s io.venlo.frame.server.ai.tools.AskUserQuestionHandler. It is constructed per conversation by LlmChatClientFactory and forwards the LLM’s structured question payload to LlmAskUserQuestionService.waitForUserAnswers, blocking until submitAnswers is called.

The submodule defines view models for the standard CRUD pages on every entity but does not declare any custom UI actions. Conversations, messages, tool calls, and attachments are surfaced as event streams under their parent conversation.