Introduced ETag annotations to support content caching from browser.

articles/ETag_HOWTO.md

@@ -0,0 +1,196 @@
+# HTTP ETag Support for WebAPI
+
+## Overview
+
+WebAPI supports HTTP ETag caching for selected GET endpoints. When enabled, the server generates an ETag (hash of the response body) and returns it with each response. On subsequent requests, if the client sends the ETag via `If-None-Match` header and the content hasn't changed, the server returns `304 Not Modified` instead of the full response body, saving bandwidth and improving performance.
+
+## Quick Start
+
+Add the `@UseEtag` annotation to any controller method:
+
+```java
+import org.ohdsi.webapi.util.UseEtag;
+
+@GetMapping("/{id}")
+@UseEtag
+public CohortDefinitionDTO getCohortDefinition(@PathVariable Integer id) {
+    return cohortDefinitionService.findById(id);
+}
+```
+
+That's it. No changes to return types or service layer required.
+
+## How It Works
+
+1. **First Request**: Client requests `/cohortdefinition/123`
+   - Server generates response JSON
+   - Filter computes SHA-256 hash of response body
+   - Response includes `ETag: "a1b2c3..."` header
+   - Client receives full response (200 OK)
+
+2. **Subsequent Requests**: Client requests same URL
+   - Browser automatically sends `If-None-Match: "a1b2c3..."` header
+   - Filter computes ETag of current response
+   - If ETags match → returns `304 Not Modified` (no body)
+   - If ETags differ → returns full response with new ETag (200 OK)
+
+## Implementation Components
+
+### `@UseEtag` Annotation
+
+Location: `org.ohdsi.webapi.util.UseEtag`
+
+Method-level annotation that marks endpoints for ETag support. Only methods with this annotation are processed by the ETag filter.
+
+```java
+@Target(ElementType.METHOD)
+@Retention(RetentionPolicy.RUNTIME)
+public @interface UseEtag {
+}
+```
+
+### `EtagUtil` Utility Class
+
+Location: `org.ohdsi.webapi.util.EtagUtil`
+
+Provides ETag generation and comparison:
+
+- `generateEtag(byte[] content)` - Computes SHA-256 hash, returns quoted string per RFC 7232 (e.g., `"a1b2c3..."`)
+- `matches(String ifNoneMatch, String etag)` - Compares `If-None-Match` header to generated ETag, handles multiple values and `*` wildcard
+
+### `EtagFilter` Servlet Filter
+
+Location: `org.ohdsi.webapi.util.EtagFilter`
+
+A servlet `Filter` that:
+
+1. Looks up the handler method via `RequestMappingHandlerMapping`
+2. Checks for `@UseEtag` annotation
+3. Wraps response with `ContentCachingResponseWrapper` to capture the body
+4. After response is written, computes ETag from cached bytes
+5. Compares with `If-None-Match` header
+6. Returns 304 or full response with appropriate headers
+
+**Key Design Decision**: Uses a Filter (not `ResponseBodyAdvice`) to avoid double-serialization. `ResponseBodyAdvice` receives the Java object before JSON serialization, so computing an ETag there would require serializing to JSON twice. The Filter intercepts after Spring has already serialized the response.
+
+## HTTP Headers
+
+For `@UseEtag` endpoints, the filter sets these response headers:
+
+| Header | Value | Purpose |
+|--------|-------|---------|
+| `ETag` | `"<sha256-hash>"` | Unique identifier for response content |
+| `Cache-Control` | `private, max-age=0, must-revalidate` | Allows browser caching but forces revalidation |
+| `Access-Control-Expose-Headers` | `ETag` | Exposes ETag to JavaScript in CORS contexts |
+
+## Testing
+
+### Using curl
+
+```bash
+# First request - get ETag
+curl -i -H "Authorization: Bearer <token>" http://localhost:8080/WebAPI/cohortdefinition/
+
+# Note the ETag header value, then:
+curl -i -H "Authorization: Bearer <token>" \
+     -H 'If-None-Match: "<etag-value>"' \
+     http://localhost:8080/WebAPI/cohortdefinition/
+
+# Should return: HTTP/1.1 304 Not Modified
+```
+
+### Using PowerShell
+
+```powershell
+$headers = @{ "Authorization" = "Bearer <token>" }
+$r1 = Invoke-WebRequest -Uri "http://localhost:8080/WebAPI/cohortdefinition/" -Headers $headers
+$etag = $r1.Headers["ETag"]
+Write-Host "ETag: $etag"
+
+$headers2 = @{ 
+    "Authorization" = "Bearer <token>"
+    "If-None-Match" = $etag 
+}
+$r2 = Invoke-WebRequest -Uri "http://localhost:8080/WebAPI/cohortdefinition/" -Headers $headers2
+Write-Host "Status: $($r2.StatusCode)"  # Should be 304
+```
+
+### Browser Testing
+
+1. Open Chrome DevTools → Network tab
+2. Ensure "Disable cache" is **unchecked**
+3. Make first request (200 OK with ETag header)
+4. Navigate away and back, or refresh normally (not hard refresh)
+5. Second request should show 304 Not Modified
+
+## Troubleshooting
+
+### Issue: Application fails to start with "required a single bean, but 2 were found"
+
+**Error:**
+```
+Parameter 0 of constructor in org.ohdsi.webapi.util.EtagFilter required a single bean, but 2 were found:
+- requestMappingHandlerMapping
+- controllerEndpointHandlerMapping
+```
+
+**Solution:** The `EtagFilter` constructor uses `@Qualifier("requestMappingHandlerMapping")` to disambiguate between Spring MVC's handler mapping and Spring Actuator's endpoint handler mapping.
+
+### Issue: Browser not sending `If-None-Match` header
+
+**Symptoms:** ETag header appears in response, but subsequent requests don't include `If-None-Match`.
+
+**Common Causes:**
+
+1. **`Cache-Control: no-store`** - Spring Security sets this by default. The filter overrides it with `private, max-age=0, must-revalidate` for `@UseEtag` endpoints.
+
+2. **CORS requests** - Cross-origin requests require special handling:
+   - The `Access-Control-Expose-Headers: ETag` header must be set (the filter does this)
+   - The `Vary: Origin` header can affect caching behavior
+   - `max-age=0, must-revalidate` works better than `no-cache` for some browsers in CORS scenarios
+
+3. **DevTools "Disable cache" enabled** - Check Chrome DevTools Network tab settings.
+
+4. **Hard refresh** - Ctrl+Shift+R bypasses cache. Use normal refresh (F5) or navigation.
+
+### Issue: curl command fails in PowerShell
+
+**Error:**
+```
+Invoke-WebRequest: ParameterBindingException
+```
+
+**Cause:** PowerShell aliases `curl` to `Invoke-WebRequest`.
+
+**Solution:** Use `curl.exe` explicitly, or use PowerShell's `Invoke-WebRequest` syntax.
+
+### Issue: ETag works with curl but not in browser (CORS)
+
+**Explanation:** When the frontend (e.g., Atlas at `localhost:80`) and backend (WebAPI at `localhost:8080`) are on different origins, browser caching behavior is more restrictive.
+
+**What helped:**
+- `Cache-Control: private, max-age=0, must-revalidate` (instead of just `no-cache`)
+- `Access-Control-Expose-Headers: ETag` to expose the header to JavaScript
+- Preserving Spring's default `Vary` header for CORS correctness
+
+**Alternative:** For maximum browser caching reliability, serve Atlas and WebAPI from the same origin.
+
+## Limitations
+
+1. **Response body required** - ETag is computed from response content; empty responses are skipped
+2. **Successful responses only** - ETag is only added for 2xx status codes
+3. **Per-request computation** - ETag is computed fresh for each request; no server-side caching of ETags
+4. **JSON responses** - Designed for JSON APIs; binary responses may work but are not the primary use case
+
+## Security Considerations
+
+- ETags use SHA-256 hashing, which is cryptographically strong
+- `Cache-Control: private` ensures responses are not stored in shared caches (proxies)
+- The `Vary: Origin` header (set by Spring CORS) ensures CORS responses are cached per-origin
+
+## References
+
+- [RFC 7232 - HTTP Conditional Requests](https://tools.ietf.org/html/rfc7232)
+- [MDN - ETag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag)
+- [MDN - If-None-Match](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-None-Match)
+- [Spring ContentCachingResponseWrapper](https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/util/ContentCachingResponseWrapper.html)

src/main/java/org/ohdsi/webapi/cohortdefinition/CohortDefinitionService.java

@@ -103,6 +103,7 @@
 import org.ohdsi.webapi.util.NameUtils;
 import org.ohdsi.webapi.util.PreparedStatementRenderer;
 import org.ohdsi.webapi.util.SessionUtils;
+import org.ohdsi.webapi.util.UseEtag;
 import org.ohdsi.webapi.versioning.domain.CohortVersion;
 import org.ohdsi.webapi.versioning.domain.Version;
 import org.ohdsi.webapi.versioning.domain.VersionBase;
@@ -437,6 +438,7 @@ public GenerateSqlResult generateSql(@RequestBody GenerateSqlRequest request) {
 	@GetMapping(produces = MediaType.APPLICATION_JSON_VALUE)
 	@Transactional(readOnly = true)
 	@Cacheable(cacheNames = CachingSetup.COHORT_DEFINITION_LIST_CACHE, key = "@authorizationService.getAuthenticatedPrincipal().getUserId()")
+	@UseEtag
 	public List<CohortMetadataDTO> getCohortDefinitionList() {
 		UserAuthorizations authz = authorizationService.getCurrentUserAuthorizations();
 		boolean globalRead = authorizationService.isPermitted("read:cohort-definition");

src/main/java/org/ohdsi/webapi/conceptset/ConceptSetService.java

@@ -56,6 +56,7 @@
 import org.ohdsi.webapi.util.CacheHelper;
 import org.ohdsi.webapi.util.ExportUtil;
 import org.ohdsi.webapi.util.NameUtils;
+import org.ohdsi.webapi.util.UseEtag;
 import org.ohdsi.webapi.util.ExceptionUtils;
 import org.ohdsi.webapi.versioning.domain.ConceptSetVersion;
 import org.ohdsi.webapi.versioning.domain.Version;
@@ -173,6 +174,7 @@ public ConceptSetDTO getConceptSet(@PathVariable("id") final int id) {
     @GetMapping(produces = MediaType.APPLICATION_JSON_VALUE)
     @Cacheable(cacheNames = ConceptSetService.CachingSetup.CONCEPT_SET_LIST_CACHE, key = "@authorizationService.getAuthenticatedPrincipal().getUserId()")
     @Transactional(readOnly = true)
+    @UseEtag
     public Collection<ConceptSetDTO> getConceptSets() {
         UserAuthorizations authz = authorizationService.getCurrentUserAuthorizations();
         boolean globalRead = authorizationService.isPermitted("read:conceptset");

src/main/java/org/ohdsi/webapi/util/EtagFilter.java

@@ -0,0 +1,131 @@
+package org.ohdsi.webapi.util;
+
+import jakarta.servlet.Filter;
+import jakarta.servlet.FilterChain;
+import jakarta.servlet.ServletException;
+import jakarta.servlet.ServletRequest;
+import jakarta.servlet.ServletResponse;
+import jakarta.servlet.http.HttpServletRequest;
+import jakarta.servlet.http.HttpServletResponse;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.core.Ordered;
+import org.springframework.core.annotation.Order;
+import org.springframework.beans.factory.annotation.Qualifier;
+import org.springframework.http.HttpHeaders;
+import org.springframework.http.HttpStatus;
+import org.springframework.stereotype.Component;
+import org.springframework.web.method.HandlerMethod;
+import org.springframework.web.servlet.HandlerExecutionChain;
+import org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping;
+import org.springframework.web.util.ContentCachingResponseWrapper;
+
+import java.io.IOException;
+
+/**
+ * Servlet filter that provides HTTP ETag support for endpoints annotated with {@link UseEtag}.
+ * <p>
+ * This filter:
+ * <ul>
+ *   <li>Looks up the handler method for the request</li>
+ *   <li>If annotated with {@code @UseEtag}, wraps the response to capture the body</li>
+ *   <li>After the response is written, computes an ETag from the body content</li>
+ *   <li>If the client's {@code If-None-Match} header matches, returns 304 Not Modified</li>
+ *   <li>Otherwise, adds the {@code ETag} header to the response</li>
+ * </ul>
+ * </p>
+ */
+@Component
+@Order(Ordered.LOWEST_PRECEDENCE - 10)
+public class EtagFilter implements Filter {
+
+    private static final Logger LOG = LoggerFactory.getLogger(EtagFilter.class);
+
+    private final RequestMappingHandlerMapping handlerMapping;
+
+    public EtagFilter(@Qualifier("requestMappingHandlerMapping") RequestMappingHandlerMapping handlerMapping) {
+        this.handlerMapping = handlerMapping;
+    }
+
+    @Override
+    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
+            throws IOException, ServletException {
+
+        HttpServletRequest httpRequest = (HttpServletRequest) request;
+        HttpServletResponse httpResponse = (HttpServletResponse) response;
+
+        // Check if the handler method is annotated with @UseEtag
+        if (!hasUseEtagAnnotation(httpRequest)) {
+            chain.doFilter(request, response);
+            return;
+        }
+
+        // Wrap response to capture the body
+        ContentCachingResponseWrapper wrappedResponse = new ContentCachingResponseWrapper(httpResponse);
+
+        // Execute the filter chain (controller writes to wrapped response)
+        chain.doFilter(request, wrappedResponse);
+
+        // Process ETag after response is written
+        processEtag(httpRequest, wrappedResponse);
+    }
+
+    private boolean hasUseEtagAnnotation(HttpServletRequest request) {
+        try {
+            HandlerExecutionChain handlerChain = handlerMapping.getHandler(request);
+            if (handlerChain == null) {
+                return false;
+            }
+
+            Object handler = handlerChain.getHandler();
+            if (handler instanceof HandlerMethod handlerMethod) {
+                return handlerMethod.hasMethodAnnotation(UseEtag.class);
+            }
+        } catch (Exception e) {
+            LOG.debug("Failed to look up handler for ETag check: {}", e.getMessage());
+        }
+        return false;
+    }
+
+    private void processEtag(HttpServletRequest request, ContentCachingResponseWrapper response)
+            throws IOException {
+
+        byte[] content = response.getContentAsByteArray();
+
+        // Only process ETag for successful responses with content
+        int status = response.getStatus();
+        if (!isSuccessStatus(status) || content.length == 0) {
+            response.copyBodyToResponse();
+            return;
+        }
+
+        String etag = EtagUtil.generateEtag(content);
+        if (etag == null) {
+            response.copyBodyToResponse();
+            return;
+        }
+
+        String ifNoneMatch = request.getHeader(HttpHeaders.IF_NONE_MATCH);
+
+        // Override Spring Security's default "no-store" with ETag-compatible caching
+        // "private, max-age=0, must-revalidate" forces browser to cache but always revalidate
+        response.setHeader(HttpHeaders.CACHE_CONTROL, "private, max-age=0, must-revalidate");
+        response.setHeader(HttpHeaders.ETAG, etag);
+        // Expose ETag header for CORS requests so browser/JS can access it
+        response.setHeader(HttpHeaders.ACCESS_CONTROL_EXPOSE_HEADERS, HttpHeaders.ETAG);
+
+        if (EtagUtil.matches(ifNoneMatch, etag)) {
+            // Client has current version - return 304 without body
+            response.resetBuffer();
+            response.setStatus(HttpStatus.NOT_MODIFIED.value());
+            response.flushBuffer();
+        } else {
+            // Return full response with ETag header
+            response.copyBodyToResponse();
+        }
+    }
+
+    private boolean isSuccessStatus(int status) {
+        return status >= 200 && status < 300;
+    }
+}