deerflow2/backend/packages/harness/deerflow
Shengyuan Wang 2f3744f807
refactor: replace sync requests with async httpx in Jina AI client (#1603)
* refactor: replace sync requests with async httpx in Jina AI client

Replace synchronous `requests.post()` with `httpx.AsyncClient` in
JinaClient.crawl() and make web_fetch_tool async. This is part of the
planned async concurrency optimization for the agent hot path
(see docs/TODO.md).

* fix: address Copilot review feedback on async Jina client

- Short-circuit error strings in web_fetch_tool before passing to
  ReadabilityExtractor, preventing misleading extraction results
- Log missing JINA_API_KEY warning only once per process to reduce
  noise under concurrent async fetching
- Use logger.exception instead of logger.error in crawl exception
  handler to preserve stack traces for debugging
- Add async web_fetch_tool tests and warn-once coverage

* fix: mock get_app_config in web_fetch_tool tests for CI

The web_fetch_tool tests failed in CI because get_app_config requires
a config.yaml file that isn't present in the test environment. Mock
the config loader to remove the filesystem dependency.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-04-01 17:02:39 +08:00
..
agents feat(memory): structured reflection + correction detection in MemoryMiddleware (#1620) (#1668) 2026-04-01 16:45:29 +08:00
community refactor: replace sync requests with async httpx in Jina AI client (#1603) 2026-04-01 17:02:39 +08:00
config fix Windows Docker sandbox path mounting (#1634) 2026-03-31 22:19:27 +08:00
guardrails feat(guardrails): add pre-tool-call authorization middleware with pluggable providers (#1240) 2026-03-23 18:07:33 +08:00
mcp feat(harness): integration ACP agent tool (#1344) 2026-03-26 14:20:18 +08:00
models fix(oauth): Harden Claude OAuth cache-control handling (#1583) 2026-03-30 07:41:18 +08:00
reflection refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
runtime fix: surface configured sandbox mounts to agents (#1638) 2026-03-31 22:22:30 +08:00
sandbox fix(sandbox): anchor relative paths to thread workspace in local mode (#1522) 2026-03-29 23:21:06 +08:00
skills fix: replace print() with logging across harness package (#1282) 2026-03-27 23:15:35 +08:00
subagents fix: surface configured sandbox mounts to agents (#1638) 2026-03-31 22:22:30 +08:00
tools fix(tools): move sandbox.tools import in view_image_tool to break circular import (#1674) 2026-03-31 22:05:23 +08:00
uploads feat(harness): integration ACP agent tool (#1344) 2026-03-26 14:20:18 +08:00
utils refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
__init__.py refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
client.py feat: support memory import and export (#1521) 2026-03-30 17:25:47 +08:00