deerflow2/backend/packages/harness/deerflow
john lee cbf8b194e8
fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#3084)
* fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#2816)

- JsonlRunEventStore: offload all file I/O to asyncio.to_thread() so the
  event loop is never blocked; add per-thread asyncio.Lock to serialise
  concurrent puts and prevent interleaved JSONL lines
- Split _ensure_seq_loaded into a sync _compute_max_seq (runs in thread)
  and an async wrapper; seq counter is recovered from disk on fresh store init
- DbRunEventStore.put_batch: raise ValueError when events span multiple
  thread_ids (previously silently assumed same thread)
- Add test_jsonl_event_store_async_io.py: 12 tests covering lock reuse,
  concurrent seq monotonicity, disk recovery, and mixed-thread batch rejection

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address Copilot review comments

- delete_by_thread: pop _write_locks after releasing the lock to prevent
  unbounded growth when threads are repeatedly created and deleted
- tests: add regression guard asserting asyncio.to_thread is called for
  _write_record in put(); assert _write_locks entry removed on delete

* fix(lint): move patch import to local scope to fix ruff I001

* fix(lint): apply ruff check+format fixes to test file

* fix(runtime): address review feedback for JSONL async I/O hardening (#2816)

Use setdefault for atomic lock init in _get_write_lock; pop _write_locks
inside the held lock scope in delete_by_thread; update test docstring
and assert lock entry also cleared on delete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: rayhpeng <rayhpeng@gmail.com>
2026-05-29 09:27:53 +08:00
..
agents fix(memory): parse wrapped memory update json responses (#3252) 2026-05-28 07:46:44 +08:00
community fix(sandbox): avoid blocking sandbox readiness polling (#2822) 2026-05-21 14:44:34 +08:00
config fix(runtime): suppress tool execution when provider safety-terminates with tool_calls (#3035) 2026-05-22 21:20:28 +08:00
guardrails feat(guardrails): add pre-tool-call authorization middleware with pluggable providers (#1240) 2026-03-23 18:07:33 +08:00
mcp fix(mcp): skip session pooling for HTTP/SSE transports to avoid anyioRuntimeError (#3203) (#3224) 2026-05-27 08:32:57 +08:00
models feat(provider) Add patched MiMo reasoning content support (#3298) 2026-05-28 18:24:32 +08:00
persistence fix: harden run finalization persistence (#3155) 2026-05-23 00:09:06 +08:00
reflection refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
runtime fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#3084) 2026-05-29 09:27:53 +08:00
sandbox fix(sandbox): add group/other read permissions to uploaded files for Docker sandbox (#3127) (#3134) 2026-05-25 09:26:18 +08:00
skills Fix custom skill install permissions (#3241) 2026-05-28 15:48:32 +08:00
subagents fix(subagents): make subagent timeout terminal state atomic (#2583) 2026-05-18 22:19:32 +08:00
tools fix(stability): resolve P0 blockers from v2.0-m1-rc1 stability audit (#3107) (#3131) 2026-05-21 21:18:10 +08:00
tracing fix(tracing): propagate session_id and user_id into Langfuse traces (#2944) 2026-05-21 16:49:31 +08:00
uploads fix(uploads): add Windows support for safe symlink-protected uploads (#2794) 2026-05-09 18:21:54 +08:00
utils fix(gateway): return ISO 8601 timestamps from threads endpoints (#2599) 2026-05-02 15:16:16 +08:00
__init__.py refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
client.py fix(tracing): propagate session_id and user_id into Langfuse traces (#2944) 2026-05-21 16:49:31 +08:00