This repository was archived by the owner on Feb 16, 2026. It is now read-only.
cache_req{,_fsm}: Add resp.storage control over synth body's storage#4358
Open
nigoroll wants to merge 198 commits intovarnishcache:masterfrom
Open
cache_req{,_fsm}: Add resp.storage control over synth body's storage#4358nigoroll wants to merge 198 commits intovarnishcache:masterfrom
nigoroll wants to merge 198 commits intovarnishcache:masterfrom
Conversation
nigoroll
force-pushed
the
sync_resp.storage
branch
from
de1b565 to
19d9528
Compare
Member
Author
|
Bugwash: Does |
nigoroll
force-pushed
the
sync_resp.storage
branch
from
19d9528 to
4585b8e
Compare
Member
Author
The answer is that we do not have a |
nigoroll
force-pushed
the
sync_resp.storage
branch
from
4585b8e to
0766d24
Compare
This reverts commit 0038dd1.
When calling out from configure to python to check compiler flags, make output from wflags.py consistent with the rest of autotools output rather than printing scary compiler errors.
sed -i 's:/github.com/varnishcache/varnish-cache/pull:/code.vinyl-cache.org/vinyl-cache/vinyl-cache/pulls:g' $(git grep -l github.com/varnishcache/varnish-cache/pull) And then manually edited two URLs pointing to individual comments
When translating VCL-to-C, VCL_BOOL values are translated verbatim, the stdbool.h file is not included in vgc.c. On Debian this causes the following error: ``` **** v1 CLI RX|Message from C-compiler: **** v1 CLI RX|vgc.c: In function 'VGC_function_vcl_init': **** v1 CLI RX|vgc.c:4402:26: error: 'false' undeclared (first use in this function) **** v1 CLI RX| 4402 | .hidevclname = false, **** v1 CLI RX| | ^~~~~ **** v1 CLI RX|vgc.c:702:1: note: 'false' is defined in header '<stdbool.h>'; this is probably fixable by adding '#include <stdbool.h>' **** v1 CLI RX| 701 | #include <stddef.h> // NULL, size_t **** v1 CLI RX| +++ |+#include <stdbool.h> **** v1 CLI RX| 702 | #include <stdint.h> // [u]int%d_t **** v1 CLI RX|vgc.c:4402:26: note: each undeclared identifier is reported only once for each function it appears in **** v1 CLI RX| 4402 | .hidevclname = false, **** v1 CLI RX|[1 lines truncated] **** v1 CLI RX|Running C-compiler failed, exited with 1 **** v1 CLI RX|VCL compilation failed ``` This commit fixes this by translating the boolean values (true/false) to int values (0/1). Since the strings are already checked to be valid identifiers we only check for the first letter in order to discriminate true from false.
sed -i 's:/github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst:/code.vinyl-cache.org/vinyl-cache/vinyl-cache/src/branch/main/doc/changes.rst:g' $(git grep -l github.com/varnishcache/varnish-cache) It is a bit annoying that forgejo does not render RST, so the document is essentially displayed as raw text. Because changes.rst is the basis for release docs and those are going to stay rst in sphinx, we can not sensibly migrate. We could check in an md copy, but that is also not nice...
sed -i 's:/github.com/varnishcache/varnish-cache:/code.vinyl-cache.org/vinyl-cache/vinyl-cache:g' $(git grep -l github.com/varnishcache/varnish-cache)
sed -i 's:/github.com/varnishcache/pkg-varnish-cache:/code.vinyl-cache.org/vinyl-cache/pkg-vinyl-cache:g' $(git grep -l github.com/varnishcache/)
The change broke literal default values, as exposed by m0.vtc Identified using git bisect Ref 2d5312c Ref #4452
The Vinyl Asynchronous Iterator (basically the code sending data to the client) requests data (by pulling into a vscarab), but the called filter or storage engine might not have it ready. In this case, -EAGAIN is returned to signal to the iterator "try again later", because no component of the VAI path must block ever. For the case of the storage engine returning -EAGAIN, we already have the notification callback provided by the iterator, which the storage engine must call when it either has data or an error to return. Upon reception of the notification, the iterator has to call into VDPIO and, consequently, ObjVAI again and pull the new state. None of this is new, but VDPs might also encounter situations where they can not serve the data _yet_, in which case they also return -EAGAIN. But they had, until now, no way to trigger a notification. So we add the simple mechanics of a notification function at the ObjVAI, Object and VPDIO layer, ultimately only leading to the notification callback registered by the iterator being called. We involve the storage engine because it might be interested to know when a VDP "above it" becomes ready to send data again, for example to start reading ahead.
I added a branch protection rule to forbid force-pushes.
Releasing 9.0.0
to keep in sync with the homepage
Partially reverts 879c9b3 This restores the option to set/keep localstatedir at ${prefix}/var and still have VINYL_STATE_DIR in {localstatedir}/lib/vinyl-cache, and not /var/run. Consequently, this fixes running vinyld with default options on systems where /var/run is mounted noexec (e.g. Debian) and restores the possibility to wholly contain the installation under ${prefix}. The patch does not use ${sharedstatedir}, because its default is ${prefix}/com Closes #4477
If VINYL_STATE_DIR could only be set to ${localstatedir}/lib/vinyl-cache, we can
not (reliably, without additional lifecycle management) use an ephemeral
${localstatedir} like /dev/shm, simply because we create a subdirectory of
VINYL_STATE_DIR for the actual state (see -n argument, defaulting to vinyld),
and ${localstatedir}/lib/vinyl-cache might not exist.
Adding recursive mkdir code to vinyld could also solve the problem, but would
add the complication of which permissions to recursively create directories
with, in particular considering that vinyld may or may not be started with
elevated privileges.
To avoid all these complications, we add a configure argument to directly set
VINYL_STATE_DIR
Related to #4477
When vsl_buffer was increased from 4k to 16k in 8869801, three tests with small workspaces were updated to explicitly set vsl_buffer to 4k. This test was missed, causing panics in VBO_GetBusyObj() and Req_New() when the 16k VSL buffer exceeds the 12000-byte pool allocation. The bug has been latent since July 2021 but was masked by mempool item reuse: the param.set happens mid-test after pool items have already been allocated at 96k, and those larger items keep getting recycled. A fresh 12k allocation (which deterministically panics) only occurs when the guard thread injects a new item before old ones are reused, which is timing-dependent. The recent move to github actions is likely the thing that made this visible. Committer edit: Adjusted to Vinyl Cache Fixes: varnish/varnish#8 Fixes: varnish/varnish#9
Until now we kept failed objects momentarily on the LRU. This is wasteful, because they never get inserted into EXP (which happens via HSH_Unbusy() -> EXP_Insert), and as such will be removed from cache when the last reference goes away. Why is this change correct? - in HSH_Lookup(), we explicitly skip OC_F_FAILED - the only place where OC_F_FAILED can be gained is HSH_Fail() - HSH_Fail() asserts that the object is still busy - LRU_Add() gets called via HSH_DerefBoc() -> ObjBocDone() when the boc's last reference goes away
I am sorry
(And make the FreeBSD runner -j2)
This adds the vcl variable resp.storage, available from vcl_synth {} to control
which storage the synth response body gets created on.
Rather than just remembering the stroage for use after VCL has returned, we
actually create an object on it when the variable changes.
This is the first step towards varnishcache#4344
nigoroll
force-pushed
the
sync_resp.storage
branch
from
0766d24 to
dc4ac44
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
draft PR of WIP for early review / feedback opportunities.
Next steps (planned, do not nail me down on this)
OBJ_VARATTRmore variable in length? #4359beresp.filtersVRT_l_{,be}resp_body()to write directly to the {body,vfp} without the VSB de-tourssy(synth) storage to remove last remaining memcpy forvcl_synth {}useThis adds the vcl variable resp.storage, available from vcl_synth {} to control which storage the synth response body gets created on.
Rather than just remembering the stroage for use after VCL has returned, we actually create an object on it when the variable changes.
This is the first step towards #4344