deerflow2/backend/packages/harness/deerflow/agents/middlewares
Huixin615 64d923b0fd
fix(middleware): externalize oversized tool output into sandbox for non-mounted sandboxes (#3417)
* fix(middleware): externalize oversized tool output into sandbox for non-mounted sandboxes

ToolOutputBudgetMiddleware persisted oversized tool results to the host
filesystem and returned a /mnt/user-data/outputs virtual path. For sandboxes
that do not use thread-data mounts (e.g. remote AIO sandbox), that virtual
path does not exist inside the sandbox, so the model's read_file tool could
not read it back and reported 'file not found'.

Branch on SandboxProvider.uses_thread_data_mounts:

- Mounted sandboxes (local Docker, AIO + LocalContainerBackend) keep the
  original host-disk path; the host outputs dir is bind-mounted to the same
  virtual path inside the sandbox, so behavior is unchanged.

- Non-mounted (remote) sandboxes externalize into the sandbox itself via
  execute_command('mkdir -p ...') + write_file + 'test -s' validation. The
  validation step is required because AIO sandbox execute_command returns
  'Error: ...' as a string on failure instead of raising, so a silent mkdir
  failure would otherwise leak through.

Any failure (rejected subdir, mkdir/write/validate error) falls back to the
existing inline head+tail truncation, so an unreadable path is never returned
to the model.

The sandbox resolver reads the sandbox_id that SandboxMiddleware already
writes into runtime.state['sandbox']; it never calls provider.acquire(),
keeping the tool-call hot path free of blocking I/O. Tools that do not use a
sandbox (web_search, MCP, ...) resolve to None and fall through to inline
truncation, which is the safe behavior for them.

Fixes #3416

* fix(middleware): address Copilot review feedback on sandbox externalization

- Make get_sandbox_provider() lookup best-effort in _budget_content: only
  query when outputs_path or sandbox is available, and fall back to inline
  truncation if provider initialization raises rather than propagating
  the error. A resolved sandbox instance is sufficient on its own to take
  the non-mounted externalization branch.
- Strict-match the sandbox post-write validation echo
  (check.strip() == 'OK') to avoid false positives if execute_command
  ever surfaces unrelated stdout/stderr containing 'OK' as a substring.

Refs: #3417

* test: fix flaky tests relying on /nonexistent/... path under container root

Two tests in this module (test_returns_none_on_invalid_path and
test_fallback_when_disk_write_fails) used paths like
'/nonexistent/impossible/path' to trigger _externalize's OSError
fallback. These paths are creatable when the test process runs as root
inside the CI container: os.makedirs(..., exist_ok=True) successfully
creates the entire chain under /, so the OSError branch is never hit
and the tests fail. Reproducible on main independently of this PR.

Switch to '/dev/null/cannot-mkdir-here'. /dev/null is a character
device on both Linux and macOS, so os.makedirs always fails with
NotADirectoryError regardless of privileges, reliably exercising the
OSError fallback.

* fix(tool-output-budget): only consult sandbox provider when a sandbox is resolved

The previous revision called get_sandbox_provider() whenever externalization
was triggered, including on the legacy host-disk path. Environments without
a configured sandbox -- in particular CI runners without a config.yaml --
would raise FileNotFoundError there, get caught, and silently fall back to
inline truncation. That defeated the host-disk externalization path that
predates this PR and was the root cause of the regressing legacy tests.

Restructure the branching so the provider is only consulted when a sandbox
has actually been resolved for the current tool call:

  - sandbox resolved + provider.uses_thread_data_mounts: host-disk write
    (bind-mounted into the sandbox, equivalent to a sandbox-side write).
  - sandbox resolved + non-mounted provider:             sandbox write (#3416).
  - no sandbox + outputs_path:                           host-disk write
    (legacy / non-sandbox tools, no provider call at all).
  - otherwise:                                           inline fallback.

No test changes; the legacy externalization tests are provider-agnostic by
construction and now pass without monkeypatching.

Refs: #3416

* test(tool-output-budget): assert legacy path does not call sandbox provider

Lock in the contract introduced by d6e2d25b: when no sandbox is resolved
for a tool call, _budget_content must externalize to the host outputs
directory without consulting get_sandbox_provider(). Regressing this would
re-break legacy / non-sandbox tools in environments without a configured
sandbox (e.g. CI without config.yaml), which is the failure mode #3416's
fix avoids.

The test injects a get_sandbox_provider that raises on call, so any
future refactor that moves the provider lookup out of the sandbox-only
branch will fail loudly.

Refs: #3416
2026-06-08 12:24:48 +08:00
..
__init__.py feat: add create_deerflow_agent SDK entry point (Phase 1) (#1203) 2026-03-29 15:31:18 +08:00
clarification_middleware.py fix(backend): make clarification messages idempotent (#2350) (#2351) 2026-04-19 22:00:58 +08:00
dangling_tool_call_middleware.py fix(runtime): guide malformed write_file recovery (#3040) 2026-05-29 17:46:24 +08:00
deferred_tool_filter_middleware.py fix(tool-search): reliably hide deferred MCP schemas by removing the ContextVar (closures + graph state) (#3342) 2026-06-02 22:43:22 +08:00
dynamic_context_middleware.py fix(middleware): offload memory injection off event loop to prevent tiktoken blocking (#3402) (#3411) 2026-06-08 12:21:55 +08:00
llm_error_handling_middleware.py fix(#3189): prevent write_file streaming timeout on long reports (#3195) 2026-06-07 17:47:11 +08:00
loop_detection_middleware.py feat(loop-detection): defer warning injection (#2752) 2026-05-21 14:36:07 +08:00
memory_middleware.py refactor: thread app_config through lead and subagent task path (#2666) 2026-05-02 06:37:49 +08:00
safety_finish_reason_middleware.py fix(runtime): suppress tool execution when provider safety-terminates with tool_calls (#3035) 2026-05-22 21:20:28 +08:00
safety_termination_detectors.py fix(runtime): suppress tool execution when provider safety-terminates with tool_calls (#3035) 2026-05-22 21:20:28 +08:00
sandbox_audit_middleware.py feat(sandbox): strengthen bash command auditing with compound splitting and expanded patterns (#1881) 2026-04-07 17:15:24 +08:00
subagent_limit_middleware.py fix(middleware): sync raw tool call metadata (#2757) 2026-05-08 10:08:53 +08:00
summarization_middleware.py fix(summarization): tag summary LLM calls nostream to stop phantom stream messages (#2503) (#3378) 2026-06-07 17:55:04 +08:00
thread_data_middleware.py feat: enhance chat history loading with new hooks and UI components (#2338) 2026-04-26 11:20:17 +08:00
title_middleware.py fix(tracing): propagate session_id and user_id into Langfuse traces (#2944) 2026-05-21 16:49:31 +08:00
todo_middleware.py fix(todo): reuse thread state schema (#3206) 2026-05-26 23:58:08 +08:00
token_usage_middleware.py feat: stream subagent token usage to header via terminal task events (#2882) 2026-05-13 23:52:19 +08:00
tool_call_metadata.py fix(middleware): sync raw tool call metadata (#2757) 2026-05-08 10:08:53 +08:00
tool_error_handling_middleware.py fix(subagent): structured subagent_status field over text parsing (#3146) (#3154) 2026-06-07 22:49:55 +08:00
tool_output_budget_middleware.py fix(middleware): externalize oversized tool output into sandbox for non-mounted sandboxes (#3417) 2026-06-08 12:24:48 +08:00
uploads_middleware.py fix(agents): offload UploadsMiddleware uploads scan off the event loop (#3311) 2026-05-30 21:46:35 +08:00
view_image_middleware.py fix(backend): preserve viewed image reducer metadata (#1900) 2026-04-06 16:47:19 +08:00