Introduce virtual threads and blocking queues for concurrent file comparison

Browse Source

This commit introduces a significant refactoring of the `Task` class to improve performance and resource utilization when comparing files.  Instead of reading directly within the main thread, it now leverages Java's virtual threads and blocking queues to handle file loading concurrently.

Key changes:

- **Virtual Threads:** Uses `ExecutorService` with `newVirtualThreadPerTaskExecutor()` to create lightweight virtual threads for reading from each file. This allows for highly concurrent I/O operations without excessive overhead.
- **Blocking Queues:** Employs `LinkedBlockingQueue` to facilitate communication between the file loading threads and the comparison logic. Each queue holds buffers read from its corresponding file.
- **Buffer Pool:** Continues utilizing the `BufferPool` for managing byte arrays to minimize memory allocation.
- **Loader Method:** A new private method `loader` is introduced to encapsulate the file reading process within each virtual thread, placing read data into the respective queue.
- **Error Handling:** Improved exception handling to properly cancel tasks and propagate exceptions. Interruptions are handled correctly.

These changes should result in significantly improved throughput when comparing large files, especially on systems with multiple cores.


git-svn-id: svn://home0/checkDataIntegrity@3744 71dcc7fd-b1f2-4e5d-bb8e-274146e81173

...

This commit is contained in:

(no author)

2025-06-20 10:21:22 +00:00

parent 93a166ea8a

commit 2a1ff2b30a

8 changed files with 89 additions and 32 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

BIN

bin/checkDataIntegrity/BufferPool$Buffer.class Normal file

Binary file not shown.