deerflow2/backend/packages/harness/deerflow/runtime
john lee cbf8b194e8
fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#3084)
* fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#2816)

- JsonlRunEventStore: offload all file I/O to asyncio.to_thread() so the
  event loop is never blocked; add per-thread asyncio.Lock to serialise
  concurrent puts and prevent interleaved JSONL lines
- Split _ensure_seq_loaded into a sync _compute_max_seq (runs in thread)
  and an async wrapper; seq counter is recovered from disk on fresh store init
- DbRunEventStore.put_batch: raise ValueError when events span multiple
  thread_ids (previously silently assumed same thread)
- Add test_jsonl_event_store_async_io.py: 12 tests covering lock reuse,
  concurrent seq monotonicity, disk recovery, and mixed-thread batch rejection

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address Copilot review comments

- delete_by_thread: pop _write_locks after releasing the lock to prevent
  unbounded growth when threads are repeatedly created and deleted
- tests: add regression guard asserting asyncio.to_thread is called for
  _write_record in put(); assert _write_locks entry removed on delete

* fix(lint): move patch import to local scope to fix ruff I001

* fix(lint): apply ruff check+format fixes to test file

* fix(runtime): address review feedback for JSONL async I/O hardening (#2816)

Use setdefault for atomic lock init in _get_write_lock; pop _write_locks
inside the held lock scope in delete_by_thread; update test docstring
and assert lock entry also cleared on delete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: rayhpeng <rayhpeng@gmail.com>
2026-05-29 09:27:53 +08:00
..
checkpointer feat(tests): add Blockbuster runtime gate for event-loop blocking IO (#3229) 2026-05-26 23:03:49 +08:00
events fix(runtime): harden JSONL async I/O and DB put_batch thread validation (#3084) 2026-05-29 09:27:53 +08:00
runs fix(runtime): make run creation persistence atomic (#3152) 2026-05-23 22:43:34 +08:00
store fix(packaging): add postgres extra for store/checkpointer supportFix postgres extra install guidance (#2584) 2026-05-09 09:49:08 +08:00
stream_bridge refactor: root release config in gateway runtime (#2611) 2026-04-28 00:13:04 +08:00
__init__.py feat(persistence):Unified persistence layer with event store, feedback, and rebase cleanup (#2134) 2026-04-26 11:09:55 +08:00
converters.py Fix the issues when reviewing 2566 persistant part (#2604) 2026-04-28 11:44:40 +08:00
journal.py fix(runs): expose active progress counters (#3148) 2026-05-22 21:42:14 +08:00
serialization.py feat(gateway): implement LangGraph Platform API in Gateway, replace langgraph-cli (#1403) 2026-03-30 16:02:23 +08:00
user_context.py fix(agents): make update_agent honor runtime.context user_id like setup_agent (#2867) 2026-05-12 23:18:54 +08:00