exo/lm-lib
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactored llm clients, removed maxContinuations feature, exposed finishReason instead;

Browse Source
This commit is contained in:
extio-de
2025-06-27 20:43:04 +02:00
parent f3706a408a
commit 47ef9ba436
26 changed files with 178 additions and 294 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
README.md
View File

@@ -4,7 +4,7 @@ The purpose of this library is to provide a comprehensive conversational AI syst
The program's key features include:
* Agent graph execution
* Agent graph execution (incl. template engine and branching capabilities)
* Integration with external AI models and services (e.g., Azure AI and OpenAi Api compatible)
* Model profiles (choose the right model and parameters for the task)
* Conversation management and text completion with support for multiple prompt strategies and tokenization implementations
@@ -143,40 +143,43 @@ Agentic flow: AgentTest#agenticFlow
# Announcement generated by AgentTest#agenticFlow
Title: Introducing a Powerful Framework for Building Intelligent Agents
## Introducing Advanced Agent Orchestration: Dynamic Conversations & Parallel Processing
We're excited to announce the development of a new Java codebase designed to build intelligent agents capable of interacting with language models like GPT. This framework offers a flexible and extensible architecture, empowering developers to create sophisticated conversational agents for a wide range of applications.
We're excited to announce significant enhancements to our agent framework, designed to empower developers building sophisticated conversational AI applications. This release introduces a powerful new system for orchestrating multiple agents, enabling more dynamic, efficient, and complex interactions.
**Key Features:**
Here's a breakdown of what's new:
* **Concurrent Text Processing:** Leveraging Java's `CompletableFuture` and `ExecutorService`, the framework enables parallel processing of text chunks, significantly speeding up agent execution.
**1. LLM Conversation Orchestration:**
* **Response-Driven Agent Selection:** The `chooseNextAgent` method intelligently selects the next agent in the chain based on the response received from the language model, allowing for dynamic and adaptive conversations.
Our framework now provides a robust system for chaining multiple agents together. Each agent can perform a specific task, and the framework intelligently manages the flow of information between them, creating a pipeline for complex conversational workflows.
* **Customizable Response Handling:** Each agent can be equipped with a custom `responseHandler` to parse and interpret responses from the language model, enabling tailored logic for specific tasks.
**2. Dynamic Prompt Templating with Branching:**
* **Dynamic Agent Chaining:** Agents can be chained together to create complex, multi-step interactions, with each agent specializing in a particular task and passing context to the next.
Say goodbye to rigid prompts! Our new templating engine allows for dynamic prompt generation based on the context. Critically, it now supports *branching*. This means a single input can trigger multiple prompt variations, allowing agents to explore different avenues of conversation or processing paths based on data within the input.
* **Contextualized Agent Execution:** The framework utilizes an `AgentContext` object to store and share information between agents, ensuring each agent has access to the relevant context for its task.
**3. Asynchronous Task Execution for Parallel Processing:**
**How it Works:**
Speed and efficiency are key. We've implemented asynchronous task execution using `CompletableFuture` and an `ExecutorService`. This allows agents to process multiple inputs (splits) *in parallel*, significantly reducing overall processing time and improving responsiveness.
The framework revolves around the `Agent` class, which defines an agent's behavior and interaction with the language model. Each agent receives an `AgentContext` containing relevant information, processes text using its defined template, interacts with the language model, and then uses its `responseHandler` to interpret the response. Based on the response, the `chooseNextAgent` method determines the next agent in the chain.
**4. Context Merging Across Multiple Agent Splits:**
**Benefits:**
When branching occurs, the framework intelligently merges the results from different execution paths. This ensures a unified context is maintained, allowing subsequent agents to access a complete and consolidated view of the conversation or task.
This framework offers several advantages for developers:
**5. Agent Chaining and Next Agent Selection:**
* **Flexibility:** Easily customize agent behavior through configurable parameters and custom response handlers.
* **Extensibility:** Add new agent types and functionalities as needed.
* **Efficiency:** Parallel processing and dynamic agent chaining optimize performance.
* **Adaptability:** Agents can respond intelligently to changing conversation contexts.
The framework dynamically determines the next agent to execute based on the current context. This allows for creating complex, stateful conversations where the flow adapts based on user input and agent responses. The system can also signal the end of the chain, indicating completion.
**Looking Ahead:**
**What does this mean for you?**
We believe this framework has the potential to revolutionize the way we build conversational agents. We're actively developing new features and functionalities, and we encourage the community to contribute and explore the possibilities.
These improvements unlock a new level of flexibility and power for building conversational AI applications. You can now:
Stay tuned for more updates and documentation!
* Create more complex and nuanced conversational flows.
* Handle a wider range of user inputs and scenarios.
* Improve the performance and responsiveness of your agents.
* Build more scalable and maintainable AI systems.
*Graph: M, CodeSummarizer, ↴, M, FeatureIdentifier, →, ⇉ 3/5, M, FeatureAnalyzer, →, 5↣1, M, AnswerGenerator*
*gemma2-27b requests=8, inTokens=19646, outTokens=2784, requestDuration=PT7M38.309006603S, getAbsoluteDuration()=PT3M54.653718763S, getTps()=95.59, getOutTps()=11.86*
We're confident that these advancements will significantly enhance your ability to build intelligent and engaging conversational experiences. Stay tuned for more updates and documentation!
*gemma3-12b-it-q5_k_m*
*M, CodeSummarizer, ↴, M, FeatureIdentifier, →, ⇉ 1/5, M, FeatureAnalyzer, 5↣1, →, M, AnswerGenerator*
*AgentRequestStatistic [requests=8, cachedPrompts=0, inTokens=27393, outTokens=4861, requestDuration=PT3M30.415881394S, getEffectiveDuration()=PT2M22.102597491S, getTps()=226.98, getOutTps()=34.21, getCost()=0]*