/claim #9810
ZStream.buffer(n) was buffering n + 1 elements instead of n. The upstream producer eagerly computes one element before offering it to the internal bounded queue, so with a queue of capacity n, the total in-flight count was always n + 1.
Reproducer (from the issue): buffer(1) allows 3 elements to be produced while the consumer is still processing the first one.
As described by @kyri-petrou in the issue discussion, the sequence is:
fakeNetworkCall(1) — computed eagerlyoffer — accepted, polled immediately downstreamfakeNetworkCall(2) — computed eagerlyoffer — added to queue (capacity 1), queue fullfakeNetworkCall(3) — computed eagerly ← this is the extra elementoffer — blocks because queue is fullThree cases, following the approach approved by @jdegoes:
| Capacity | Strategy | Why |
|---|---|---|
<= 0 |
No-op (return stream unchanged) | No buffering requested |
== 1 |
Synchronous Handoff |
Can’t use Queue.bounded(0). Handoff blocks the producer until the consumer takes, so exactly 1 element is buffered ahead. |
>= 2 |
Queue.bounded(capacity - 1) |
Queue capacity (n-1) + 1 eagerly computed element = exactly n total |
The processLoop helper is extracted to avoid duplicating the channel drain logic across the handoff and queue paths.
| Test | What it verifies |
|---|---|
buffer(1) does not prefetch a third element |
After pulling element 1, the producer offers element 2 into the handoff and blocks — element 3 does not start. Repeated 100x (@@ nonFlaky). |
buffer(2) allows a third element to start |
Confirms that buffer(2) does allow 3 elements in flight (correct behaviour for capacity 2). |
buffer(1) maintains elements and ordering |
Property-based test ensuring correctness is preserved with the new handoff path. |
sbt "streamsTestsJVM/testOnly zio.stream.ZStreamSpec -- -t buffer"
23 tests passed. 0 tests failed. 0 tests ignored.
All existing buffer tests continue to pass.
Update - Video recording of the fix: https://drive.google.com/file/d/1w3NtJsVE2xbn05m9LP0evoVKairaMDKA/view?usp=sharing Shows all 23 buffer tests passing, including 3 new regression tests:
Huntersreeni
@HunterSreeni
ZIO
@ZIO