* fix(threads): clean up local thread data after thread deletion
Delete DeerFlow-managed thread directories after the web UI removes a LangGraph thread.
This keeps local thread data in sync with conversation deletion and adds regression coverage for the cleanup flow.
* fix(threads): address thread cleanup review feedback
Encode thread cleanup URLs in the web client, keep cache updates explicit when no thread search data is cached, and return a generic 500 response from the cleanup endpoint while documenting the sanitized error behavior.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Add GuardrailMiddleware that evaluates every tool call before execution.
Three provider options: built-in AllowlistProvider (zero deps), OAP passport
providers (open standard), or custom providers loaded by class path.
- GuardrailProvider protocol with GuardrailRequest/Decision dataclasses
- GuardrailMiddleware (AgentMiddleware, position 5 in chain)
- AllowlistProvider for simple deny/allow by tool name
- GuardrailsConfig (Pydantic singleton, loaded from config.yaml)
- 25 tests covering allow/deny, fail-closed/open, async, GraphBubbleUp
- Comprehensive docs at backend/docs/GUARDRAILS.md
Closes#1213
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add Claude Code OAuth and Codex CLI providers
Port of bytedance/deer-flow#1136 from @solanian's feat/cli-oauth-providers branch.\n\nCarries the feature forward on top of current main without the original CLA-blocked commit metadata, while preserving attribution in the commit message for review.
* fix: harden CLI credential loading
Align Codex auth loading with the current ~/.codex/auth.json shape, make Docker credential mounts directory-based to avoid broken file binds on hosts without exported credential files, and add focused loader tests.
* refactor: tighten codex auth typing
Replace the temporary Any return type in CodexChatModel._load_codex_auth with the concrete CodexCliCredential type after the credential loader was stabilized.
* fix: load Claude Code OAuth from Keychain
Match Claude Code's macOS storage strategy more closely by checking the Keychain-backed credentials store before falling back to ~/.claude/.credentials.json. Keep explicit file overrides and add focused tests for the Keychain path.
* fix: require explicit Claude OAuth handoff
* style: format thread hooks reasoning request
* docs: document CLI-backed auth providers
* fix: address provider review feedback
* fix: harden provider edge cases
* Fix deferred tools, Codex message normalization, and local sandbox paths
* chore: narrow PR scope to OAuth providers
* chore: remove unrelated frontend changes
* chore: reapply OAuth branch frontend scope cleanup
* fix: preserve upload guards with reasoning effort wiring
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: normalize ToolMessage structured content in serialization
When models return ToolMessage content as a list of content blocks
(e.g. [{"type": "text", "text": "..."}]), the UI previously displayed
the raw Python repr string instead of the extracted text.
Replace str(msg.content) with the existing _extract_text() helper in
both _serialize_message() and stream() to properly normalize
list-of-blocks content to plain text.
Fixes#1149
Also fixes the same root cause as #1188 (characters displayed one per
line when tool response content is returned as structured blocks).
Added 11 regression tests covering string, list-of-blocks, mixed,
empty, and fallback content types.
* fix(memory): extract text from structured LLM responses in memory updater
When LLMs return response content as list of content blocks
(e.g. [{"type": "text", "text": "..."}]) instead of plain strings,
str() produces Python repr which breaks JSON parsing in the memory
updater. This caused memory updates to silently fail.
Changes:
- Add _extract_text() helper in updater.py for safe content normalization
- Use _extract_text() instead of str(response.content) in update_memory()
- Fix format_conversation_for_update() to handle plain strings in list content
- Fix subagent executor fallback path to extract text from list content
- Replace print() with structured logging (logger.info/warning/error)
- Add 13 regression tests covering _extract_text, format_conversation,
and update_memory with structured LLM responses
* fix: address Copilot review - defensive text extraction + logger.exception
- client.py _extract_text: use block.get('text') + isinstance check (prevent KeyError/TypeError)
- prompt.py format_conversation_for_update: same defensive check for dict text blocks
- executor.py: type-safe text extraction in both code paths, fallback to placeholder instead of str(raw_content)
- updater.py: use logger.exception() instead of logger.error() for traceback preservation
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: preserve chunked structured content without spurious newlines
* fix: restore backend unit test compatibility
---------
Co-authored-by: Exploreunive <Exploreunive@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: track token usage per conversation turn
Add token usage tracking to the streaming API so consumers can monitor
cost per turn without additional API calls.
Changes:
1. _serialize_message now includes usage_metadata for AI messages in
values events, exposing input_tokens/output_tokens/total_tokens
from LangChain's native metadata.
2. stream() accumulates token usage across all AI messages in a turn
and emits the cumulative totals in the end event:
{usage: {input_tokens: N, output_tokens: N, total_tokens: N}}
3. Each messages-tuple AI event with text content now includes a
per-message usage_metadata field for granular tracking.
This enables the frontend to display token consumption per turn,
support cost-aware UX, and let users monitor API spending.
10 tests added covering serialization passthrough and cumulative
aggregation logic.
Co-Authored-By: OpenClaw <noreply@openclaw.ai>
* fix: address Copilot review - use Mapping access for usage_metadata
- Replace getattr(usage, 'input_tokens', 0) with usage.get('input_tokens', 0)
since LangChain usage_metadata is a dict, not an object
- Remove unused 'import pytest' (fixes Ruff F401)
- Add proper stream() integration tests for cumulative usage in end event
and per-message usage_metadata in messages-tuple events
---------
Co-authored-by: Exploreunive <Exploreunive@users.noreply.github.com>
Co-authored-by: OpenClaw <noreply@openclaw.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This PR improves MiniMax Code Plan integration in DeerFlow by fixing three issues in the current flow: stream errors were not clearly surfaced in the UI, the frontend could not display the actual provider model ID, and MiniMax reasoning output could leak into final assistant content as inline <think>...</think>. The change adds a MiniMax-specific adapter, exposes real model IDs end-to-end, and adds a frontend fallback for historical messages.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(feishu): support @bot message in topic groups
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(feishu): preserve rich-text formatting and add parser unit tests
* chore(test): remove unused import to fix ruff lint error
* style: auto-format imports to satisfy ruff
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(manager): add bootstrap command to initialize soul.md in correct place
* feat(channels): add /bootstrap command to IM channels
Add a `/bootstrap` command that routes to the chat handler with
`is_bootstrap: True` in the run context, allowing the agent to invoke
its setup/initialization flow (e.g. `setup_agent`).
- The text after `/bootstrap` is forwarded as the chat message; when
omitted a default "Initialize workspace" message is used.
- Feishu channels use the streaming path as with normal chat.
- No changes to ChannelStore — bootstrap is stateless and triggered
purely by the command.
- Update /help output to include /bootstrap.
- Add 5 tests covering: text/no-text variants, Feishu streaming path,
thread creation, and help text.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: accept copilot suggestion
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(gateway): remove generated markdown on upload delete
Keep thread upload storage consistent by deleting the generated markdown companion when the original convertible upload is removed.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(harness): allow agent read access to /mnt/skills in local sandbox
Skill files under /mnt/skills/ were blocked by the path validator,
preventing agents from reading skill definitions. This change:
- Refactors `resolve_local_tool_path` into `validate_local_tool_path`,
a pure security gate that no longer resolves paths (left to the sandbox)
- Permits read-only access to the skills container path (/mnt/skills by
default, configurable via config.skills.container_path)
- Blocks write access to skills paths (PermissionError)
- Allows /mnt/skills in bash command path validation
- Adds `LocalSandbox.update_path_mappings` and injects per-thread
user-data mappings into the sandbox so all virtual-path resolution
is handled uniformly by the sandbox layer
- Covers all new behaviour with tests
Fixes#1177
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(sandbox): unify all virtual path resolution in tools.py
Move skills path resolution from LocalSandbox into tools.py so that all
virtual-to-host path translation (user-data and skills) lives in one
layer. LocalSandbox becomes a pure execution layer that receives only
real host paths — no more path_mappings, _resolve_path, or reverse
resolve logic.
This addresses architecture feedback that path resolution was split
across two layers (tools.py for user-data, LocalSandbox for skills),
making the flow hard to follow.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(sandbox): address Copilot review — cache-on-success and error path masking
- Replace @lru_cache with manual cache-on-success for _get_skills_container_path
and _get_skills_host_path so transient failures at startup don't permanently
disable skills access.
- Add _sanitize_error() helper that masks host filesystem paths in error
messages via mask_local_paths_in_output before returning them to the agent.
- Apply _sanitize_error() to all catch-all (Exception/OSError) handlers in
sandbox tool functions to prevent host path leakage in error output.
- Remove unused lru_cache import.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tools): add tool_search for deferred MCP tool loading
When multiple MCP servers are enabled, total tool count can exceed 30-50,
causing context bloat and degraded tool selection accuracy. This adds a
deferred tool loading mechanism controlled by `tool_search.enabled` config.
- Add ToolSearchConfig with single `enabled` field
- Add DeferredToolRegistry with regex search (select:, +keyword, keyword)
- Add tool_search tool returning OpenAI-compatible function JSON
- Add DeferredToolFilterMiddleware to hide deferred schemas from bind_tools
- Add <available-deferred-tools> section to system prompt
- Enable MCP tool_name_prefix to prevent cross-server name collisions
- Add 34 unit tests covering registry, tool, prompt, and middleware
* fix: reset stale deferred registry and bump config_version
- Reset deferred registry upfront in get_available_tools() to prevent
stale tool entries when MCP servers are disabled between calls
- Bump config_version to 2 for new tool_search config field
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(tests): mock get_app_config in prompt section tests for CI
CI has no config.yaml, causing TestDeferredToolsPromptSection to fail
with FileNotFoundError. Add autouse fixture to mock get_app_config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add citation/reference support to deep research reports (#1141)
- Enhance lead agent system prompt with mandatory citation requirements
after web_search/web_fetch tool usage
- Add citation examples and best practices to GitHub Deep Research skill
- Add citation hints to report template (Executive Summary, Key Analysis)
- Style regular markdown links in frontend for visual distinction
(color, underline, hover effect)
- Fix TitleMiddleware being registered when title generation is disabled
* fix: address PR review comments
- Revert TitleMiddleware conditional registration (agent.py) to avoid
sync/async incompatibility with DeerFlowClient
- Fix markdown link rendering: merge classNames instead of overwriting,
only set target=_blank for external http(s) URLs
- Remove unrelated package.json/pnpm-lock.yaml changes
* fix: use plain markdown links in Sources section for cleaner rendering
Inline citations in report body use [citation:Title](URL) for pill/badge style.
Sources section uses plain [Title](URL) for simple underlined link style.
* fix(frontend): render plain links as underlined text in artifact markdown
Only links with citation: prefix render as Badge pills.
Regular links in Sources section now render as underlined text links.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(harness): normalize structured content for titles
Flatten structured LangChain message content before prompting the title model so list/block payloads don't leak Python reprs into generated thread titles.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* Apply suggestions from code review
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* refactor: extract shared utils to break harness→app cross-layer imports
Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: split backend/src into harness (deerflow.*) and app (app.*)
Physically split the monolithic backend/src/ package into two layers:
- **Harness** (`packages/harness/deerflow/`): publishable agent framework
package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
models, MCP, skills, config, and all core infrastructure.
- **App** (`app/`): unpublished application code with import prefix `app.*`.
Contains gateway (FastAPI REST API) and channels (IM integrations).
Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure
Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: add harness→app boundary check test and update docs
Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.
Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add config versioning with auto-upgrade on startup
When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.
- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix comments
* fix: update src.* import in test_sandbox_tools_security to deerflow.*
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: handle empty config and search parent dirs for config.example.yaml
Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
looking next to config.yaml, fixing detection in common setups
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: correct skills root path depth and config_version type coercion
- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
after harness split, file lives at packages/harness/deerflow/skills/
so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
_check_config_version() to prevent TypeError when YAML stores value
as string (e.g. config_version: "1")
- tests: add regression tests for both fixes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update test imports from src.* to deerflow.*/app.* after harness refactor
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(feishu): stream updates on a single card
* fix(feishu): ensure final message on stream error and warn on missing card ID
- Wrap streaming loop in try/except/finally so a is_final=True outbound
message is always published, even when the LangGraph stream breaks
mid-way. This prevents _running_card_ids memory leaks and ensures the
Feishu card shows a DONE reaction instead of hanging on "Working on it".
- Log a warning when _ensure_running_card gets no message_id back from
the Feishu reply API, making silent fallback to new-card behavior
visible in logs.
- Add test_handle_feishu_stream_error_still_sends_final to cover the
error path.
- Reformat service.py dict comprehension (ruff format, no logic change).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Avoid blocking inbound on Feishu card creation
---------
Co-authored-by: songyaolun <songyaolun@bytedance.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add LoopDetectionMiddleware to break repetitive tool call loops
Adds a new AgentMiddleware that detects when the agent is stuck calling
the same tools with the same arguments repeatedly, which currently runs
until the recursion limit kills the run.
Detection: per-thread sliding window of tool call hashes (name + args).
- Warn threshold (default 3): injects a "wrap up" system message
- Hard limit (default 5): strips tool_calls, forcing final text output
Includes 13 unit tests covering hashing, thresholds, window sliding,
reset, and edge cases.
Closes#1055
* fix: address PR #1056 review feedback for LoopDetectionMiddleware
- Remove unused imports (Awaitable, Callable, ModelCallResult,
ModelRequest, ModelResponse, AIMessage) from loop_detection_middleware
- Remove unused pytest import from test file
- Fix _hash_tool_calls sort key: sort by (name, serialized args) for
deterministic hashing when multiple calls share the same tool name
- Revert subagent_enabled default to False in agent.py to match
DeerFlowClient and channel defaults
- Remove unrelated SearxNG tools and Next.js rewrite changes from PR
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address 2nd round review feedback on PR #1056
- Inject loop warning only once per thread (prevents context bloat)
- Add threading.Lock for thread-safe history mutations
- Use runtime.context thread_id instead of workspace_path
- Add LRU eviction for per-thread history (max 100 threads)
- Add 5 new tests covering warn-once, LRU eviction, thread isolation,
fallback thread_id, and lock presence
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: resolve lint errors in loop detection middleware tests
Sort imports (I001) and remove unused _WARNING_MSG import (F401)
to fix ruff lint failures in CI.
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add MiniMax as an OpenAI-compatible model provider
MiniMax offers high-performance LLMs (M2.5, M2.5-highspeed) with
204K context windows. This commit adds MiniMax as a selectable
provider in the configuration system.
Changes:
- Add MiniMax to SUPPORTED_MODELS with model definitions
- Add MiniMax provider configuration in conf/config.yaml
- Update documentation with MiniMax setup instructions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update README to remove MiniMax API details
Removed mention of MiniMax API usage and configuration examples.
---------
Co-authored-by: octo-patch <octo-patch@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: preserve conversation context in Telegram private chats
In private (1-on-1) chats, set topic_id=None so all messages map to a
single DeerFlow thread per chat instead of creating a new thread for
every message. Also fix _cmd_generic to use topic_id=None in private
chats so /new correctly targets the default thread.
Group chat behavior is unchanged (reply_to or msg_id as topic_id).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: preserve conversation context in Telegram private chats
Fixes#1101
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: mirror _on_text reply logic in _cmd_generic for group chats
_cmd_generic now prefers reply_to_message.message_id over msg_id in
group/supergroup chats, consistent with _on_text. This ensures commands
like /new and /status target the correct conversation thread when sent
as a reply in group chats.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(sandbox): harden local file access and mask host paths
- enforce local sandbox file tools to only accept /mnt/user-data paths
- add path traversal checks against thread workspace/uploads/outputs roots
- preserve requested virtual paths in tool error messages (no host path leaks)
- mask local absolute paths in bash output back to virtual sandbox paths
- update bash tool guidance to prefer thread-local venv + python -m pip
- add regression tests for path mapping, masking, and access restrictions
Fixes#968
* feat(sandbox): restrict risky absolute paths in local bash commands
- validate absolute path usage in local-mode bash commands
- allow only /mnt/user-data virtual paths for user data access
- keep a small allowlist for system executable/device paths
- return clear permission errors for unsafe command paths
- add regression tests for bash path validation rules
* test(sandbox): add success path test for resolve_local_tool_path (#992)
* Initial plan
* test(sandbox): add success path test for resolve_local_tool_path
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
* fix(sandbox): reject bare virtual root early with clear error in resolve_local_tool_path (#991)
* Initial plan
* fix(sandbox): reject bare virtual root early with clear error in resolve_local_tool_path
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
* fix(gateway): ignore archive metadata wrappers
Treat top-level __MACOSX and dotfile entries as packaging metadata so valid .skill archives still resolve to their real skill directory.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* Apply suggestions from code review
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(gateway): allow standard skill frontmatter metadata
Accept standard optional frontmatter fields during .skill installs so external skills with version, author, or compatibility metadata do not fail validation.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* docs: sync skill installer metadata behavior
Document the skill install allowlist so user-facing and backend contributor docs match the gateway validation contract.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* Apply suggestions from code review
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(gateway): normalize suggestion response content
Handle list-style model content before JSON parsing so provider wrappers do not silently drop follow-up suggestions.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* docs: sync suggestions endpoint behavior
Document the rich-content normalization path so the README and backend gateway notes stay aligned with the current router contract.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix(tracing): support LANGCHAIN_* env fallback for LangSmith config
- add backward-compatible env parsing in tracing_config.py
- support fallback keys:
LANGCHAIN_TRACING_V2 / LANGCHAIN_TRACING
LANGCHAIN_API_KEY
LANGCHAIN_PROJECT
LANGCHAIN_ENDPOINT
- keep LANGSMITH_* as preferred source when both are present
- add regression tests in test_tracing_config.py
* fix(tracing): correct LANGSMITH_* precedence over LANGCHAIN_* for enabled flag (#1067)
* Initial plan
* fix(tracing): use first-present-wins logic for enabled flag, add precedence docs and test
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
* Refactor sandbox state management and improve Docker integration
- Removed FileSandboxStateStore and SandboxStateStore classes for a cleaner architecture.
- Enhanced LocalContainerBackend to handle port allocation retries and introduced environment variable support for sandbox host configuration.
- Updated Paths class to include host_base_dir for Docker volume mounts and ensured proper permissions for sandbox directories.
- Modified ExtensionsConfig to improve error handling when loading configuration files and adjusted environment variable resolution.
- Updated sandbox configuration to include a replicas option for managing concurrent sandbox containers.
- Improved logging and context management in SandboxMiddleware for better sandbox lifecycle handling.
- Enhanced network port allocation logic to bind to 0.0.0.0 for compatibility with Docker.
- Updated Docker Compose files to ensure proper volume management and environment variable configuration.
- Created scripts to ensure necessary configuration files are present before starting services.
- Cleaned up unused MCP server configurations in extensions_config.example.json.
* Address Copilot review suggestions from PR #1068 (#9)
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
* fix(subagents): cleanup background tasks after completion to prevent memory leak
Added cleanup_background_task() function to remove completed subagent results
from the global _background_tasks dict. Found a small issue: completed tasks
were never removed, causing memory to grow indefinitely with each subagent
execution.
Alternative approaches considered:
- Future + SubagentHandle pattern: Not chosen due to requiring refactoring
Chose the simple cleanup approach for minimal code changes while effectively
resolving the memory leak.
Changes:
- Add cleanup_background_task() in executor.py
- Call cleanup in all task_tool return paths (completed, failed, timed out)
* fix(subagents): prevent race condition in background task cleanup
Address Copilot review feedback on memory leak fix:
- Add terminal state check in cleanup_background_task() to only remove
tasks that are COMPLETED/FAILED/TIMED_OUT or have completed_at set
- Remove cleanup call from polling safety-timeout branch in task_tool
since the task may still be running
- Add comprehensive tests for cleanup behavior including:
- Verification that cleanup is called on terminal states
- Verification that cleanup is NOT called on polling timeout
- Tests for terminal state check logic in executor
This prevents KeyError when the background executor tries to update
a task that was prematurely removed from _background_tasks.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(checkpointer): return InMemorySaver instead of None when not configured (#1016)
* fix(checkpointer): also fix get_checkpointer() to return InMemorySaver
Make all three checkpointer functions consistent:
- make_checkpointer() (async) → InMemorySaver
- checkpointer_context() (sync) → InMemorySaver
- get_checkpointer() (sync singleton) → InMemorySaver
This ensures DeerFlowClient always has a valid checkpointer.
* fix: address CI failure and Copilot review feedback
- Fix import order in test_checkpointer_none_fix.py (I001 ruff error)
- Fix type annotation: _checkpointer should be Checkpointer | None
- Update docstring: change "None if not configured" to "InMemorySaver if not configured"
- Ensure app config is loaded before checking checkpointer config to prevent incorrect InMemorySaver fallback
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add claude-to-deerflow skill for DeerFlow API integration
Add a new skill that enables Claude Code to interact with the DeerFlow
AI agent platform via its HTTP API, including chat streaming and status
checking capabilities.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: fix telegram channel
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add IM channels system for Feishu, Slack, and Telegram integration
Bridge external messaging platforms to DeerFlow via LangGraph Server with
async message bus, thread management, and per-channel configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review comments on IM channels system
Fix topic_id handling in store remove/list_entries and manager commands,
correct Telegram reply threading, remove unused imports/variables, update
docstrings and docs to match implementation, and prevent config mutation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* update skill creator
* fix im reply text
* fix comments
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add checkpointer configuration to config.example.yaml
- Introduced a new section for checkpointer configuration to enable state persistence for the embedded DeerFlowClient.
- Documented supported types: memory, sqlite, and postgres, along with examples for each.
- Clarified that the LangGraph Server manages its own state persistence separately.
* refactor(checkpointer): streamline checkpointer initialization and logging
* fix(uv.lock): update revision and add new wheel URLs for brotlicffi package
* feat: add langchain-anthropic dependency and update related configurations
* Fix checkpointer lifecycle, docstring, and path resolution bugs from PR #1005 review (#4)
* Initial plan
* Address all review suggestions from PR #1005
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
* Fix resolve_path to always return real Path; move SQLite special-string handling to callers
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
* feat: u may ask
* chore: adjust code according to CR
* chore: adjust code according to CR
* ut: test for suggestions.py
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(subagent): support async MCP tools in subagent executor
SubagentExecutor.execute() was synchronous and could not handle async-only tools like MCP tools. This caused failures when trying to use MCP tools within subagents.
Changes:
- Add _aexecute() async method using agent.astream() for async execution
- Refactor execute() to use asyncio.run() wrapping _aexecute()
- This allows subagents to use async tools (MCP) within ThreadPoolExecutor
* test(subagent): add unit tests for executor async/sync paths
Add comprehensive tests covering:
- Async _aexecute() with success/error cases
- Sync execute() wrapper using asyncio.run()
- Async tool (MCP) support verification
- Thread pool execution safety
* fix(subagent): subagent-test-circular-depend
- Use session-scoped fixture with delayed import to handle circular dependencies
without affecting other test modules
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
- replace with explicit runtime deps:
- regenerate after dependency changes
- make deterministic by patching
to avoid leaked global affecting expected paths
* feat(upload): implement optimistic UI for file uploads and enhance message handling
* feat(middleware): enhance file handling by collecting historical uploads from directory
* feat(thread-title): update page title handling for new threads and improve loading state
* feat(uploads-middleware): enhance file extraction by verifying file existence in uploads directory
* feat(thread-stream): update file path reference to use virtual_path for uploads
* feat(tests): add core behaviour tests for UploadsMiddleware
* feat(tests): remove unused pytest import from test_uploads_middleware_core_logic.py
* feat: enhance file upload handling and localization support
- Update UploadsMiddleware to validate filenames more robustly.
- Modify MessageListItem to parse uploaded files from raw content for backward compatibility.
- Add localization for uploading messages in English and Chinese.
- Introduce parseUploadedFiles utility to extract uploaded files from message content.
* fix(memory): prevent file upload events from persisting in long-term memory
Uploaded files are session-scoped and unavailable in future sessions.
Previously, upload interactions were recorded in memory, causing the
agent to search for non-existent files in subsequent conversations.
Changes:
- memory_middleware: skip human messages containing <uploaded_files>
and their paired AI responses from the memory queue
- updater: post-process generated memory to strip upload mentions
before saving to file
- prompt: instruct the memory LLM to ignore file upload events
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(memory): address Copilot review feedback on upload filtering
- memory_middleware: strip <uploaded_files> block from human messages
instead of dropping the entire turn; only skip the turn (and paired
AI response) when nothing remains after stripping
- updater: narrow the upload-scrubbing regex to explicit upload events
(avoids false-positive removal of "User works with CSV files" etc.);
also filter upload-event facts from the facts array
- prompt: move `import re` to module scope; skip upload-only human
messages (empty after stripping) rather than appending "User: "
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(memory): allow optional words between 'upload' and 'file' in scrub regex
The previous pattern required 'uploading file' with no intervening words,
so 'uploading a test file' was not matched and leaked into long-term memory.
Allow up to 3 modifier words between the verb and noun (e.g. 'uploading a
test file', 'uploaded the attachment').
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(memory): add unit tests for upload filtering in memory pipeline
Covers _filter_messages_for_memory and _strip_upload_mentions_from_memory
per Copilot review suggestion. 15 test cases verify:
- Upload-only turns (and paired AI responses) are excluded from memory queue
- User's real question is preserved when combined with an upload block
- Upload file paths are never present in filtered message content
- Intermediate tool messages are always excluded
- Multi-turn conversations: only the upload turn is dropped
- Multimodal (list-content) human messages are handled
- Upload-event sentences are removed from summaries and facts
- Legitimate file-related facts (CSV preferences, PDF exports) are preserved
- "uploading a test file" (words between verb and noun) is caught by regex
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: add agent management functionality with creation, editing, and deletion
* feat: enhance agent creation and chat experience
- Added AgentWelcome component to display agent description on new thread creation.
- Improved agent name validation with availability check during agent creation.
- Updated NewAgentPage to handle agent creation flow more effectively, including enhanced error handling and user feedback.
- Refactored chat components to streamline message handling and improve user experience.
- Introduced new bootstrap skill for personalized onboarding conversations, including detailed conversation phases and a structured SOUL.md template.
- Updated localization files to reflect new features and error messages.
- General code cleanup and optimizations across various components and hooks.
* Refactor workspace layout and agent management components
- Updated WorkspaceLayout to use useLayoutEffect for sidebar state initialization.
- Removed unused AgentFormDialog and related edit functionality from AgentCard.
- Introduced ArtifactTrigger component to manage artifact visibility.
- Enhanced ChatBox to handle artifact selection and display.
- Improved message list rendering logic to avoid loading states.
- Updated localization files to remove deprecated keys and add new translations.
- Refined hooks for local settings and thread management to improve performance and clarity.
- Added temporal awareness guidelines to deep research skill documentation.
* feat: refactor chat components and introduce thread management hooks
* feat: improve artifact file detail preview logic and clean up console logs
* feat: refactor lead agent creation logic and improve logging details
* feat: validate agent name format and enhance error handling in agent setup
* feat: simplify thread search query by removing unnecessary metadata
* feat: update query key in useDeleteThread and useRenameThread for consistency
* feat: add isMock parameter to thread and artifact handling for improved testing
* fix: reorder import of setup_agent for consistency in builtins module
* feat: append mock parameter to thread links in CaseStudySection for testing purposes
* fix: update load_agent_soul calls to use cfg.name for improved clarity
* fix: update date format in apply_prompt_template for consistency
* feat: integrate isMock parameter into artifact content loading for enhanced testing
* docs: add license section to SKILL.md for clarity and attribution
* feat(agent): enhance model resolution and agent configuration handling
* chore: remove unused import of _resolve_model_name from agents
* feat(agent): remove unused field
* fix(agent): set default value for requested_model_name in _resolve_model_name function
* feat(agent): update get_available_tools call to handle optional agent_config and improve middleware function signature
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat: Add reasoning effort configuration support
* Add `reasoning_effort` parameter to model config and agent initialization
* Support reasoning effort levels (minimal/low/medium/high) for Doubao/GPT-5 models
* Add UI controls in input box for reasoning effort selection
* Update doubao-seed-1.8 example config with reasoning effort support
Fixes & Cleanup:
* Ensure UTF-8 encoding for file operations
* Remove unused imports
* fix: set reasoning_effort to None for unsupported models
* fix: unit test error
* Update frontend/src/components/workspace/input-box.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
add oauth schema to MCP server config (extensions_config.json)
support client_credentials and refresh_token grants
implement token manager with caching and pre-expiry refresh
inject OAuth Authorization header for MCP tool discovery and tool calls
extend MCP gateway config models to read/write OAuth settings
update docs and examples for OAuth configuration
add unit tests for token fetch/cache and header injection
* fix: use shell fallback instead of hardcoded /bin/zsh in LocalSandbox
Replace hardcoded /bin/zsh executable with dynamic shell detection
that falls back through /bin/zsh → /bin/bash → /bin/sh. This fixes
skill execution failures in Docker containers (python:3.12-slim)
where zsh is not available.
Closes#935
* Update backend/src/sandbox/local/local_sandbox.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: atian8179 <atian8179@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Validate that all dict-returning client methods conform to Gateway
Pydantic response models (ModelsListResponse, ModelResponse,
SkillsListResponse, SkillResponse, SkillInstallResponse,
McpConfigResponse, UploadResponse, MemoryConfigResponse,
MemoryStatusResponse). Pydantic ValidationError in CI catches
schema drift between client and Gateway with zero production coupling.
Also includes prior review fixes: enhanced client methods, expanded
unit tests (67→77), live integration test improvements, and updated
documentation.
Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
Add `DeerFlowClient` class that provides direct in-process access to
DeerFlow's agent and Gateway capabilities without requiring LangGraph
Server or Gateway API processes. This enables users to import and use
DeerFlow as a Python library.
Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
* feat: add Novita AI as optional LLM provider
Adds Novita AI (https://novita.ai) as an optional, OpenAI-compatible
LLM provider.
Changes:
- Added Novita model configuration example in config.example.yaml
- Added NOVITA_API_KEY to .env.example
Usage: Set NOVITA_API_KEY in your environment and use novita-gpt-4
as the model name.
* update correct model info
* Update README.md
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix: recover from stale model context after config model changes
* fix: fail fast on missing model config and expand model resolution tests
* fix: remove duplicate get_app_config imports
* fix: align model resolution tests with runtime imports
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: remove duplicate model resolution test case
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Previously used before_model which returned {"messages": patches}, causing
LangGraph's add_messages reducer to append patches at the end of the message
list. This resulted in invalid ordering (ToolMessage after a HumanMessage)
that LLMs reject with tool call ID mismatch errors.
Switch to wrap_model_call/awrap_model_call to insert synthetic ToolMessages
immediately after each dangling AIMessage before the request reaches the LLM,
without persisting the patches to state.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Enforces config env var checks and improves startup handling
Ensures critical environment variables are validated during config resolution,
raising clear errors if missing. Improves server startup reliability by
verifying that backend services are listening and by terminating on
misconfiguration at launch. Adds more robust feedback to developers when
API startup fails, reducing silent misconfigurations and speeding up
troubleshooting.
* Initial plan
* Implement suggestions from PR #892: fix env var checks and improve error logging
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(subagents): make subagent timeout configurable via config.yaml
- Add SubagentsAppConfig supporting global and per-agent timeout_seconds
- Load subagents config section in AppConfig.from_file()
- Registry now applies config.yaml overrides without mutating builtin defaults
- Polling safety-net in task_tool is now dynamic (execution timeout + 60s buffer)
- Document subagents section in config.example.yaml
- Add make test command and enforce TDD policy in CLAUDE.md
- Add 38 unit tests covering config validation, timeout resolution, registry
override behavior, and polling timeout formula
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(subagents): add logging for subagent timeout config and execution
- Log loaded timeout config (global default + per-agent overrides) on startup
- Log debug message in registry when config.yaml overrides a builtin timeout
- Include timeout in executor's async execution start log
- Log effective timeout and polling limit when a task is dispatched
- Fix UnboundLocalError: move max_poll_count assignment before logger.info
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci(backend): add lint step and run all unit tests via Makefile
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix lint
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add LangSmith tracing integration
Add optional LangSmith tracing support that can be enabled via environment
variables (LANGSMITH_TRACING, LANGSMITH_API_KEY, LANGSMITH_PROJECT,
LANGSMITH_ENDPOINT). When enabled, a LangChainTracer callback is attached
to chat models and run metadata is injected for trace tagging.
Co-Authored-By: Claude <noreply@anthropic.com>
* Update backend/src/config/tracing_config.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update backend/src/agents/lead_agent/agent.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update backend/src/agents/lead_agent/agent.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update backend/src/models/factory.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add threading lock to ensure thread-safe access to tracing configuration
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Unifies market analysis, data analysis, and consulting reporting into a comprehensive consulting-analysis skill, enabling a two-phase workflow from analysis framework design to professional report generation. Introduces a DuckDB-based data analysis utility for Excel/CSV files and a chart-visualization skill with a flexible JS interface and extensive chart type documentation. Removes the legacy market analysis skill to streamline report generation and improve extensibility for consulting and data-driven workflows.
* Adds Kubernetes sandbox provisioner support
* Improves Docker dev setup by standardizing host paths
Replaces hardcoded host paths with a configurable root directory,
making the development environment more portable and easier to use
across different machines. Automatically sets the root path if not
already defined, reducing manual setup steps.
Support configuring max_concurrent_subagents (2-4, default 3) through
config.configurable, with automatic clamping and dynamic prompt generation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract get_skills_prompt_section() from apply_prompt_template() so
subagents can also receive the available skills list in their system
prompt. This allows subagents to discover and load skills via read_file,
just like the lead agent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: adds docker-based dev environment
* docs: updates Docker command help
* fix local dev
* feat(sandbox): add Kubernetes-based sandbox provider for multi-instance support
* fix: skills path in k8s
* feat: add example config for k8s sandbox
* fix: docker config
* fix: load skills on docker dev
* feat: support sandbox execution to Kubernetes Deployment model
* chore: rename web service name
- Backend: add citation format to lead_agent and general_purpose prompts
- Add CitationLink component (Badge + HoverCard) for citation cards
- MarkdownContent: detect citation: prefix in link text, render CitationLink
- Message/artifact/subtask: use MarkdownContent or Streamdown with CitationLink
- message-list-item: pass img via components prop (remove isHuman/img)
- message-group, subtask-card: drop unused imports; fix import order (lint)
Co-authored-by: Cursor <cursoragent@cursor.com>
- Keep upstream subagent HARD LIMIT (max 3 task calls, batching) in subagent_reminder
- Keep our removal of Citations: do not add back 'Citations when synthesizing'
Co-authored-by: Cursor <cursoragent@cursor.com>
Strengthen the SUBAGENT_SECTION prompt to prevent the model from launching
more than 3 subagents in a single response. When >3 sub-tasks are needed,
the model is now explicitly instructed to plan and execute in sequential
batches of ≤3. Reinforced at three prompt injection points: thinking style,
main subagent instructions, and critical reminders.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add two new middlewares to improve robustness of the agent pipeline:
- DanglingToolCallMiddleware injects placeholder ToolMessages for
interrupted tool calls, preventing LLM errors from malformed history
- SubagentLimitMiddleware truncates excess parallel task tool calls at
the model response level, replacing the runtime check in task_tool
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add "present_files" to disallowed_tools for bash and general-purpose
subagents to prevent them from presenting files directly. Also add the
new market-analysis skill for generating consulting-grade reports.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Updated documentation to accurately cover all backend subsystems including
subagents, memory, middleware chain, sandbox, MCP, skills, and gateway API.
Fixed broken MCP_SETUP.md link in root README.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevent resource exhaustion by capping the number of parallel subagents.
Adds runtime enforcement in task_tool and updates prompts/examples accordingly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enable task tool to capture and stream AI messages as they are generated by subagents. This replaces simple polling status updates with detailed message-level progress updates.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Reorder task tool parameters to prioritize description first for better usability
- Add tool_call_id injection for better task traceability
- Use tool_call_id as task_id in executor for consistent tracking
- Simplify event messages by removing redundant task_type field
- Update task examples in prompt to reflect new parameter order
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
AI was outputting bare brackets like [arXiv:xxx] without URLs,
which do not render as links. Updated prompt to explicitly show
correct vs wrong formats and require complete markdown links.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add subagents.enabled flag in config.yaml to control subagent feature:
- When disabled, task/task_status tools are not loaded
- When disabled, system prompt excludes subagent documentation
- Default is enabled for backward compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add accurate token counting using tiktoken library and significantly enhance
memory update prompts with detailed section guidelines, multilingual support,
and improved fact extraction. Update deep-research skill to be more proactive
for research queries.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Use citation.title for display text in CitationLink to ensure correct
titles show during streaming (instead of generic "Source" text)
- Render all external links as CitationLink badges for consistent styling
during streaming output
- Add removeAllCitations when copying message content to clipboard
- Simplify citations_format prompt for cleaner AI output
Co-authored-by: Cursor <cursoragent@cursor.com>
Add native Apple Container support for better performance on macOS while
maintaining full Docker compatibility. Enhance documentation with memory system
details, development guidelines, and sandbox setup instructions. Improve dev
experience with container image pre-pulling and unified cleanup tools.
Key changes:
- Auto-detect and prefer Apple Container on macOS with Docker fallback
- Add APPLE_CONTAINER.md with complete usage and troubleshooting guide
- Document memory system architecture in CLAUDE.md
- Add make setup-sandbox for pre-pulling container images
- Create cleanup-containers.sh for cross-runtime container cleanup
- Update all related documentation (README, SETUP, config examples)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implement automatic cache invalidation based on file modification time to ensure memory data consistency across Gateway API and agent prompts. The cache now automatically reloads when the memory file is updated externally.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add memory API endpoints for retrieving memory data:
- GET /api/memory - get current memory data
- POST /api/memory/reload - reload from file
- GET /api/memory/config - get memory configuration
- GET /api/memory/status - get config and data together
- Optimize MemoryMiddleware to only use user inputs and final
assistant responses, filtering out intermediate tool calls
- Add memory configuration example to config.example.yaml
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement a memory system that stores user context and conversation history
in memory.json, uses LLM to summarize conversations, and injects relevant
context into system prompts for personalized responses.
Key components:
- MemoryConfig for configuration management
- MemoryUpdateQueue with debounce for batch processing
- MemoryUpdater for LLM-based memory extraction
- MemoryMiddleware to queue conversations after agent execution
- Memory injection into lead agent system prompt
Note: Add memory section to config.yaml to enable (see config.example.yaml)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add README.md with project overview, quick start, and API reference
- Add CONTRIBUTING.md with development setup and contribution guidelines
- Add docs/ARCHITECTURE.md with detailed system architecture diagrams
- Add docs/API.md with complete API reference for LangGraph and Gateway
- Add docs/README.md as documentation index
- Update CLAUDE.md with improved structure and new features
- Update docs/TODO.md to reflect current status
- Update pyproject.toml description
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Enable previewing .skill files (ZIP archives) by extracting and displaying
their SKILL.md content. Add caching to avoid repeated ZIP extraction.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add POST /api/skills/install endpoint to install .skill files from
thread's user-data directory. The endpoint extracts the ZIP archive,
validates SKILL.md frontmatter, and installs to skills/custom/.
Frontend Install buttons now call the API instead of downloading.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change skills rendering from attribute-based format to nested element format
with <available_skills>, <skill>, <name>, <description>, and <location> tags
for better readability and structure.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add model-aware vision tool loading based on supports_vision flag
- Move view_image_tool from config to builtin tools for dynamic inclusion
- Add timeout to image search to prevent hanging requests
- Optimize image search results format using thumbnails
- Add image validation for reference images in generation
- Improve error handling with detailed messages
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add image viewing capability for vision-enabled models with ViewImageMiddleware and view_image_tool. Limit web_fetch tool output to 4096 characters to prevent excessive content. Update model config to support vision capability flag.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Backend:
- Handle both string and list format for message content in uploads middleware
- Extract text content from structured message blocks
- Add logging for debugging file upload flow
Frontend:
- Separate file display from message bubble for human messages
- Show uploaded files outside the message bubble for cleaner layout
- Improve file card border styling with subtle border color
- Add debug logging for message submission with files
Co-authored-by: Cursor <cursoragent@cursor.com>
Backend:
- Simplify citations prompt format and rules
- Add clear distinction between chat responses and file content
- Enforce full URL usage in markdown links, prohibit [cite-1] format
- Require content-first approach: write full content, then add citations at end
Frontend:
- Hide <citations> block in both chat messages and markdown preview
- Remove top-level Citations/Sources list for cleaner UI
- Auto-remove <citations> block in code editor view for markdown files
- Keep inline citation hover cards for reference details
This ensures citations are presented like Claude: clean content with inline reference badges.
Co-authored-by: Cursor <cursoragent@cursor.com>
Simplify clarification message handling by having the frontend detect and
display ask_clarification tool messages directly, instead of relying on
backend to add an extra AIMessage.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Citations:
- Add citations parsing utilities for extracting source references from AI responses
- Render inline citations as hover card badges in message content
- Display citation cards with title, URL, and description on hover
- Add citation badge rendering in artifact markdown preview
- Update prompt to guide AI to output citations in correct format
Thread Management:
- Add rename functionality for chat threads with dialog UI
- Add share functionality to copy thread link to clipboard
- Share links use Vercel URL for production accessibility
- Add useRenameThread hook for thread title updates
i18n:
- Add translations for rename, share, cancel, save, and linkCopied
Co-authored-by: Cursor <cursoragent@cursor.com>
When using thinking-enabled models (like Kimi K2.5, DeepSeek), the API
expects reasoning_content on all assistant messages. The original
ChatDeepSeek stores reasoning_content in additional_kwargs but doesn't
include it when making subsequent API calls, causing "reasoning_content
is missing" errors.
This adds PatchedChatDeepSeek which overrides _get_request_payload to
restore reasoning_content from additional_kwargs into the payload.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add web_search_tool and web_fetch_tool implementations using the official
firecrawl-py SDK as an alternative to Tavily/Jina AI integrations.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use ThreadPoolExecutor to generate audio for multiple script lines
concurrently, significantly speeding up podcast generation.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add podcast-generation skill for creating tech explainer podcasts
- Include generate.py script with TTS synthesis capabilities
- Add tech-explainer template for structured podcast content
- Increase sandbox command timeout from 30s to 600s to support
longer-running skill scripts
- Use ExtensionsConfig.from_file() instead of cached config to always
read latest configuration from disk in LangGraph Server
- Add mtime-based cache invalidation for MCP tools to detect config
file changes made through Gateway API
- Call reload_extensions_config() in Gateway API after updates to
refresh the global cache
- Remove unnecessary MCP initialization from Gateway startup since
MCP tools are only used by LangGraph Server
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add environment field to sandbox config for injecting env vars into container
- Support $VAR syntax to resolve values from host environment variables
- Refactor frontend API modules to use centralized getBackendBaseURL()
- Improve Doraemon skill with explicit input/output path arguments
- Add .env.example file
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a root-level Makefile to manage frontend, backend, and nginx services:
- `make check` validates required dependencies (Node.js 22+, pnpm, uv, nginx)
- `make install` installs all project dependencies
- `make dev` starts all services with unified port 2026
- `make stop` and `make clean` for cleanup
Update nginx configuration:
- Change port from 8000 to 2026
- Add frontend upstream and routing (port 3000)
- Add /api/langgraph/* routing with path rewriting to LangGraph server
- Keep other /api/* routes to Gateway API
- Route non-API requests to frontend
Update frontend configuration:
- Use relative URLs through nginx proxy by default
- Support environment variables for direct backend access
- Construct full URL for LangGraph SDK compatibility
Clean up backend Makefile by removing nginx and serve targets.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add type, url, and headers fields to MCP server config
- Update MCP client to handle stdio, sse, and http transports
- Add todos field to ThreadState
- Add Deerflow branding requirement to frontend-design skill
- Update extensions_config.example.json with SSE/HTTP examples
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add new MCP configuration management endpoint and enhance API documentation
with detailed descriptions, examples, and OpenAPI support for better
developer experience.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add comprehensive MCP integration using langchain-mcp-adapters to enable
pluggable external tools from MCP servers.
Features:
- MCP server configuration via mcp_config.json
- Automatic lazy initialization for seamless use in both FastAPI and LangGraph Studio
- Support for multiple MCP servers (filesystem, postgres, github, brave-search, etc.)
- Environment variable resolution in configuration
- Tool caching mechanism for optimal performance
- Complete documentation and setup guide
Implementation:
- Add src/mcp module with client, tools, and cache components
- Integrate MCP config loading in AppConfig
- Update tool system to include MCP tools automatically
- Add eager initialization in FastAPI lifespan handler
- Add lazy initialization fallback for LangGraph Studio
Dependencies:
- Add langchain-mcp-adapters>=0.1.0
Documentation:
- Add MCP_SETUP.md with comprehensive setup guide
- Update CLAUDE.md with MCP system architecture
- Update config.example.yaml with MCP configuration notes
- Update README.md with MCP setup instructions
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The proxy was creating a temporary httpx.AsyncClient within an async context manager.
When returning StreamingResponse for SSE endpoints, the client was being closed before
the streaming generator could use it, causing "client has been closed" errors.
This change introduces a shared httpx.AsyncClient that persists for the application
lifecycle, properly cleaned up during shutdown. This also improves performance by
reusing TCP connections across requests.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove .claude/settings.local.json from git tracking and add it to .gitignore to prevent future accidental commits of local settings.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Defer sandbox acquisition and thread directory creation until first use to improve performance and reduce resource usage.
Changes:
- Add lazy_init parameter to SandboxMiddleware (default: true)
- Add ensure_sandbox_initialized() helper for lazy sandbox acquisition
- Update all sandbox tools to use lazy initialization
- Add lazy_init parameter to ThreadDataMiddleware (default: true)
- Create thread directories on-demand in AioSandboxProvider
- LocalSandbox already creates directories on write (no changes needed)
Benefits:
- Saves 1-2s Docker container startup for conversations without tools
- Reduces unnecessary directory creation and file system operations
- Backward compatible with lazy_init=false option
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Add thread-safe port allocation and proper cleanup on process exit to
prevent port conflicts in concurrent environments and ensure containers
are stopped when the application terminates.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Implement a skills framework that enables specialized workflows for
specific tasks (e.g., PDF processing, web page generation). Skills are
discovered from the skills/ directory and automatically mounted in
sandboxes with path mapping support.
- Add SkillsConfig for configuring skills path and container mount point
- Implement dynamic skill loading from SKILL.md files with YAML frontmatter
- Add path mapping in LocalSandbox to translate container paths to local paths
- Mount skills directory in AIO Docker sandbox containers
- Update lead agent prompt to dynamically inject available skills
- Add setup documentation and expand config.example.yaml
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add .claude/ to .gitignore
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add gateway module with FastAPI server
- Add new gateway module with FastAPI app for API routing
- Add gateway and serve commands to Makefile
- Add fastapi, httpx, uvicorn, sse-starlette dependencies
- Fix model config retrieval in lead_agent (support both model_name and model)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Add AioSandboxProvider for Docker-based sandbox execution with
configurable container lifecycle, volume mounts, and port management
- Add TitleMiddleware to auto-generate thread titles after first
user-assistant exchange using LLM
- Add Claude Code documentation (CLAUDE.md, AGENTS.md)
- Extend SandboxConfig with Docker-specific options (image, port, mounts)
- Fix hardcoded mount path to use expanduser
- Add agent-sandbox and dotenv dependencies
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
amdoi7.
Uchi Uchibeke
knukn
infoquest-byteplus
Purricane
mxyhi
Gao Mingfei
haoliangxu
Ryanba
Chetan Sharma
Simon Su
JilongSun
greatmengqi
dependabot[bot]
DanielWalnut
lhd
lailoo
-Astraia-
YolenSong
virtaava
Octopus
Matt(허철진)
Willem Jiang
Frank
Liu Jice
JeffJiang
momorebi
aworki
Xinmin Zeng
null4536251
FangHao
Zhiyunyao
エイカク
atian8179
Alex
CHANGXUBO
hetao
LofiSu
Henry Li
amszuidas