Commit Graph

56 Commits

Author SHA1 Message Date
Willem Jiang 4f7c5c9b71 fix: ensure web search is performed for research plans to fix #535 (#640)
* fix: ensure web search is performed for research plans to fix #535

          When using certain models (DeepSeek-V3, Qwen3, or local deployments), the
          agent framework failed to trigger web search tools, resulting in hallucinated
          data. This fix implements multiple safeguards:

          1. Add enforce_web_search configuration flag:
             - New config option to mandate web search in research plans
             - Defaults to False for backward compatibility

          2. Add plan validation function validate_and_fix_plan():
             - Validates that plans include at least one research step with web search
             - Enforces web search requirement when enabled
             - Adds default research step if plan has no steps

          3. Enhance coordinator_node fallback logic:
             - When model fails to call tools, fallback to planner instead of __end__
             - Ensures workflow continues even when tool calling fails
             - Logs detailed diagnostic info for debugging

          4. Update prompts for stricter requirements:
             - planner.md: Add MANDATORY web search requirement and clear warnings
             - coordinator.md: Add CRITICAL tool calling requirement
             - Emphasize consequences of missing web search (hallucinated data)

          5. Update tests to reflect new behavior:
             - test_coordinator_node_no_tool_calls: Expect planner instead of __end__
             - test_coordinator_empty_llm_response_corner_case: Same expectation

          Fixes #535 by ensuring:
          - Web search is always performed for research tasks
          - Workflow doesn't terminate on tool calling failures
          - Models with poor tool calling support can still proceed
          - No hallucinated data without real information gathering

* Update src/graph/nodes.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/graph/nodes.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* accept the review suggestion of getting configuration

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-10-22 08:27:06 +08:00
Willem Jiang 81110b7cb9 fix: add missing RunnableConfig parameter to human_feedback_node (#629)
* fix: add missing RunnableConfig parameter to human_feedback_node

This fixes issue #569 where interrupt() was being called outside of a runnable context.
The human_feedback_node was missing the config: RunnableConfig parameter that all other
node functions have, which caused RuntimeError when interrupt() tried to access the config.

- Add config: RunnableConfig parameter to function signature
- Add State type annotation to state parameter for consistency
- Maintains LangGraph execution context required by interrupt()

* test: update human_feedback_node tests to pass RunnableConfig parameter

Update all test functions that call human_feedback_node to include the new
required config parameter. These tests were failing because they were not
providing the RunnableConfig argument after the fix to add proper LangGraph
execution context.

Tests updated:
- test_human_feedback_node_auto_accepted
- test_human_feedback_node_edit_plan
- test_human_feedback_node_accepted
- test_human_feedback_node_invalid_interrupt
- test_human_feedback_node_json_decode_error_first_iteration
- test_human_feedback_node_json_decode_error_second_iteration
- test_human_feedback_node_not_enough_context

All tests now pass the mock_config fixture to human_feedback_node.
2025-10-19 17:35:06 +08:00
Willem Jiang 969919416f fix: improve error handling in researcher and coder nodes (#596)
- Wrap agent.ainvoke() calls in try-except blocks
- Log full exception tracebacks for better debugging
- Return detailed error messages to users instead of generic 'internal error'
- Include step title and agent name in error context
- Allow workflow to continue gracefully when agent execution fails
- Store error details in observations for audit trail
2025-10-19 16:33:14 +08:00
jimmyuconn1982 fabec4abb6 feat: Add intelligent clarification feature in coordinate step for research queries (#613)
* fix: support local models by making thought field optional in Plan model

- Make thought field optional in Plan model to fix Pydantic validation errors with local models
- Add Ollama configuration example to conf.yaml.example
- Update documentation to include local model support
- Improve planner prompt with better JSON format requirements

Fixes local model integration issues where models like qwen3:14b would fail
due to missing thought field in JSON output.

* feat: Add intelligent clarification feature for research queries

- Add multi-turn clarification process to refine vague research questions
- Implement three-dimension clarification standard (Tech/App, Focus, Scope)
- Add clarification state management in coordinator node
- Update coordinator prompt with detailed clarification guidelines
- Add UI settings to enable/disable clarification feature (disabled by default)
- Update workflow to handle clarification rounds recursively
- Add comprehensive test coverage for clarification functionality
- Update documentation with clarification feature usage guide

Key components:
- src/graph/nodes.py: Core clarification logic and state management
- src/prompts/coordinator.md: Detailed clarification guidelines
- src/workflow.py: Recursive clarification handling
- web/: UI settings integration
- tests/: Comprehensive test coverage
- docs/: Updated configuration guide

* fix: Improve clarification conversation continuity

- Add comprehensive conversation history to clarification context
- Include previous exchanges summary in system messages
- Add explicit guidelines for continuing rounds in coordinator prompt
- Prevent LLM from starting new topics during clarification
- Ensure topic continuity across clarification rounds

Fixes issue where LLM would restart clarification instead of building upon previous exchanges.

* fix: Add conversation history to clarification context

* fix: resolve clarification feature message to planer, prompt, test issues

- Optimize coordinator.md prompt template for better clarification flow
- Simplify final message sent to planner after clarification
- Fix API key assertion issues in test_search.py

* fix: Add configurable max_clarification_rounds and comprehensive tests

- Add max_clarification_rounds parameter for external configuration
- Add comprehensive test cases for clarification feature in test_app.py
- Fixes issues found during interactive mode testing where:
  - Recursive call failed due to missing initial_state parameter
  - Clarification exited prematurely at max rounds
  - Incorrect logging of max rounds reached

* Move clarification tests to test_nodes.py and add max_clarification_rounds to zh.json
2025-10-14 13:35:57 +08:00
Fancy-hjyp 364d12cc05 feat: add context compress (#590)
* feat:Add context compress

* feat: Add unit test

* feat: add unit test for context manager

* feat: add postprocessor param && code format

* feat: add configuration guide

* fix: fix the configuration_guide

* fix: fix the unit test

* fix: fix the default value

* feat: add test and log for context_manager
2025-09-27 21:42:22 +08:00
Chayton Bai cf7cc62c1b feat: support dify in rag module (#550)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-09-16 20:30:45 +08:00
CHANGXUBO 84c2eec410 feat: Implement Milvus retriver for RAG (#516)
* feat: Implement MilvusRetriever with embedding model and resource management

* chore: Update configuration and loader files for consistency

* chore: Clean up test_milvus.py for improved readability and organization

* feat: Add tests for DashscopeEmbeddings query and document embedding methods

* feat: Add tests for embedding model initialization and example file loading in MilvusProvider

* chore: Remove unused imports and clean up test_milvus.py for better readability

* chore: Clean up test_milvus.py for improved readability and organization

* chore: Clean up test_milvus.py for improved readability and organization

* fix: replace print statements with logging in recursion limit function

* Implement feature X to enhance user experience and optimize performance

* refactor: clean up unused imports and comments in AboutTab component

* Implement feature X to enhance user experience and fix bug Y in module Z

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-09-12 22:20:55 +08:00
Willem Jiang f1620489ca feat: creating mogodb and postgres mock instance in checkpoint test (#561)
* fix: using mongomock for the checkpoint test

* Add postgres mock setting to the unit test

* Added utils file of postgres_mock_utils

* fixed the runtime loading error of deerflow server
2025-09-09 22:49:11 +08:00
Willem Jiang 5ddbefeadf fix: the search content return tuple issue (#555) 2025-09-04 15:45:30 +08:00
Willem Jiang 7d8a0281a0 Fixed the deepseek v3 planning issue #545 (#554) 2025-09-04 10:09:49 +08:00
Anoyer-lzh 660a93338a fix: env parameters exception when configuring SSE or HTTP MCP server (#513)
* fix: _create_streamable_http_session() got an unexpected keyword argument 'env'

fix unit error

* update md

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-08-20 17:23:57 +08:00
zgjja 6bb96c9e7d feat: 1. replace black with ruff for fomatting and sort import (#489)
2. use tavily from`langchain-tavily` rather than the older one from `langchain-community`

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-08-17 22:57:23 +08:00
CHANGXUBO 88c642a683 feat: Enhance chat streaming and tool call processing (#498)
* feat: Enhance chat streaming and tool call processing

- Added support for MongoDB checkpointer in the chat streaming workflow.
- Introduced functions to process tool call chunks and sanitize arguments.
- Improved event message creation with additional metadata.
- Enhanced error handling for JSON serialization in event messages.
- Updated the frontend to convert escaped characters in tool call arguments.
- Refactored the workflow input preparation and initial message processing.
- Added new dependencies for MongoDB integration and tool argument sanitization.

* fix: Update MongoDB checkpointer configuration to use LANGGRAPH_CHECKPOINT_DB_URL

* feat: Add support for Postgres checkpointing and update README with database recommendations

* feat: Implement checkpoint saver functionality and update MongoDB connection handling

* refactor: Improve code formatting and readability in app.py and json_utils.py

* refactor: Clean up commented code and improve formatting in server.py

* refactor: Remove unused imports and improve code organization in app.py

* refactor: Improve code organization and remove unnecessary comments in app.py

* chore: use langgraph-checkpoint-postgres==2.0.21 to avoid the JSON convert issue in the latest version, implement chat stream persistant with Postgres

* feat: add MongoDB and PostgreSQL support for LangGraph checkpointing, enhance environment variable handling

* fix: update comments for clarity on Windows event loop policy

* chore: remove empty code changes in MongoDB and PostgreSQL checkpoint tests

* chore: clean up unused imports and code in checkpoint-related files

* chore: remove empty code changes in test_checkpoint.py

* chore: remove empty code changes in test_checkpoint.py

* chore: remove empty code changes in test_checkpoint.py

* test: update status code assertions in MCP endpoint tests to allow for 403 responses

* test: update MCP endpoint tests to assert specific status codes and enable MCP server configuration

* chore: remove unnecessary environment variables from unittest workflow

* fix: invert condition for MCP server configuration check to raise 403 when disabled

* chore: remove pymongo from test dependencies in uv.lock

* chore:  optimize the _get_agent_name method

* test: enhance ChatStreamManager tests for PostgreSQL and MongoDB initialization

* test: add persistence tests for ChatStreamManager with PostgreSQL and MongoDB

* test: add unit tests for ChatStreamManager initialization with PostgreSQL and MongoDB

* test: enhance persistence tests for ChatStreamManager with PostgreSQL and MongoDB to verify message aggregation

* test: add unit tests for ChatStreamManager with PostgreSQL and MongoDB

* test: add unit tests for ChatStreamManager initialization with PostgreSQL and MongoDB

* test: add unit tests for ChatStreamManager initialization with PostgreSQL and MongoDB

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-08-16 21:03:12 +08:00
Willem Jiang 762df2e279 fix: langchain-mcp-adapters version conflict (#500)
* fix: langchain-mcp-adapters version conflict

* fix the lint error
2025-08-04 10:36:31 +08:00
DanielWalnut 9c9390968d refine the research prompt (#459) 2025-07-22 14:13:10 +08:00
DanielWalnut 80bc8d14b8 fix: fix the bug introduced by coordinator messages update (#445) 2025-07-18 21:36:13 +08:00
道心坚定韩道友 911b8b947c fix:planner AttributeError 'list' object has no attribute 'get' (#436) 2025-07-18 09:27:15 +08:00
Kuro Akuta c0ad5e67ca fix: fix the coordinator's forgetting of its own messages. (#433) 2025-07-17 08:36:31 +08:00
DanielWalnut 034817cc4a refactor: human feedback doesn't need to check enough context (#423) 2025-07-15 18:51:41 +08:00
Willem Jiang 1a6ffd7556 fix: clean up the builder code (#417)
* fix: clean up the builder code

* fix:reformat the code
2025-07-15 17:22:50 +08:00
yihong ec78dc0962 fix: some lint fix using tools (#98)
* fix: some lint fix using tools

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: md lint

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: some lint fix using tools

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-12 13:59:02 +08:00
Willem Jiang 309e45f1b5 test: add unit tests for graph (#296)
* test: added unit test of builder

* test: Add unit tests for nodes.py

* test: add more unit tests in test_nodes

* test: try to fix the unit test error on GitHub

* test: reformate the code of test_nodes.py

* Fix the test error of reset the local argument

* Fixed the test error by setup args

* reformat the code
2025-06-18 10:05:02 +08:00
Willem Jiang b09dfa70c0 Revert "fix: solves the malformed json output and pydantic validation error p…" (#325)
This reverts commit a7315b46df.
2025-06-14 22:04:03 +08:00
DanielWalnut b6edc0fc8a feat: add deep think feature (#311)
* feat: implement backend logic

* feat: implement api/config endpoint

* rename the symbol

* feat: re-implement configuration at client-side

* feat: add client-side of deep thinking

* fix backend bug

* feat: add reasoning block

* docs: update readme

* fix: translate into English

* fix: change icon to lightbulb

* feat: ignore more bad cases

* feat: adjust thinking layout, and implement auto scrolling

* docs: add comments

---------

Co-authored-by: Henry Li <henry1943@163.com>
2025-06-14 13:12:43 +08:00
Tax f1b0454658 fix: solves the malformed json output and pydantic validation error produced by the 'planner' node by forcing the llm response to strictly comply with the pydantic 'Plan' model (#322) 2025-06-14 10:13:30 +08:00
DanielWalnut 310f08076a refactor: refine teh background check logic (#306) 2025-06-11 11:10:02 +08:00
DanielWalnut 01b1c21044 feat: support to adjust writing style (#290)
* feat: implment backend for adjust report style

* feat: add web part

* fix test cases

* fix: fix typing

---------

Co-authored-by: Henry Li <henry1943@163.com>
2025-06-07 20:48:39 +08:00
DanielWalnut 78125d5fd4 refactor: refine the graph structure (#283) 2025-06-05 12:47:17 +08:00
JeffJiang 96fd196baa feat: RAG Integration (#238)
* feat: add rag provider and retriever

* feat: retriever tool

* feat: add retriever tool to the researcher node

* feat: add rag http apis

* feat: new message input supports resource mentions

* feat: new message input component support resource mentions

* refactor: need_web_search to need_search

* chore: RAG integration docs

* chore: change example api host

* fix: user message color in dark mode

* fix: mentions style

* feat: add local_search_tool to researcher prompt

* chore: research prompt

* fix: ragflow page size and reporter with

* docs: ragflow integration and add acknowledgment projects

* chore: format
2025-05-28 14:13:46 +08:00
DanielWalnut d4f88e8a6e fix: fix unittes & background investigation search logic (#247) 2025-05-28 14:05:34 +08:00
wushiai1109 52c8cb0269 Update nodes.py (#242)
SELECTED_SEARCH_ENGINE impossible equal to SearchEngine.ARXIV, should be SearchEngine.ARXIV.value, or use the encapsulated get_web_search_tool
2025-05-27 18:58:14 +08:00
DanielWalnut bc95f4fca6 feat: config max_search_results for search engine (#192)
* feat: implement UI

* feat: config max_search_results for search engine via api

---------

Co-authored-by: Henry Li <henry1943@163.com>
2025-05-18 13:23:52 +08:00
changqingla caa4a2826d Fix :This PR can resolve the issue of exceeding the default tool invocation limit by setting the recursion limit through an environment variable.mit (#138)
* set ecursion limit

* set ecursion limit

* fix:check if the recession_limit within a reasonalbe range

* style: format code with black
2025-05-17 20:37:03 -07:00
DanielWalnut ce44dfcb7b refactor: upgrade langgraph version (#148) 2025-05-18 11:29:41 +08:00
Wang Hao 9891af752f fix: add model_dump (#137)
Co-authored-by: Willem Jiang <143703838+willem-bd@users.noreply.github.com>
2025-05-16 21:05:46 +08:00
DanielWalnut d865b7eba0 refactor: refine the step execute human message (#144) 2025-05-14 18:54:14 +08:00
DanielWalnut 17701599d4 refactor: add existing research findings into step human message (#140) 2025-05-14 18:40:14 +08:00
Zhao Longjie 6489831616 fix: background investigator node support more search engine (#75)
Change-Id: I030a2b9218dfbda2dd2383b7a73266dd7de589c7
2025-05-12 20:15:47 +08:00
DanielWalnut e0ef130cd9 feat: add necessary log when no tool calls (#16) 2025-05-09 14:22:07 +08:00
Zhao Longjie dab1ba4789 fix(planner): skip human feedback if context is sufficient
Change-Id: I2b5628a7a8ecb6a6bad2712a9ff81b9b1cd323c6
2025-04-29 18:17:27 +08:00
Zhao Longjie 899438eca0 feat(nodes): add background investigation node
Change-Id: I96e08e22fc7c52647edbf9be4f385a8fae9b449a
2025-04-27 20:15:42 +08:00
Wang Hao ada5e34eeb feat: add LOCALE variable to agent prompts 2025-04-26 20:57:38 -07:00
He Tao 3e4e551613 fix: add server name in desc 2025-04-23 18:02:58 +08:00
He Tao 0cf859b393 feat: support mcp settings 2025-04-23 16:00:01 +08:00
He Tao abdc740531 feat: add langgraph.json for langgraph studio debug 2025-04-22 15:34:06 +08:00
He Tao afb8571cbd feat: use doubao model by default 2025-04-22 11:04:36 +08:00
He Tao 52313b2405 fix: fix multi-round issue 2025-04-21 20:16:42 +08:00
Li Xin b67b04ff5d feat: support multi-language 2025-04-21 19:50:34 +08:00
He Tao 6937abcd91 chore: add license headers 2025-04-17 11:34:42 +08:00
He Tao 76fd04df22 chore: change the project name 2025-04-17 11:17:03 +08:00