Skip to content

Eureka deregistration fails on graceful shutdown: "Connection pool shut down" #4569

@ZakariaYakoub

Description

@ZakariaYakoub

I am experiencing an issue where my Spring Boot microservices fail to deregister from Eureka during a graceful shutdown. Because the deregistration HTTP request fails,

During shutdown, it seems the HTTP connection pool is closed before the DiscoveryClient attempts to send the final DELETE request to the Eureka server, resulting in an IllegalStateException: Connection pool shut down.
Could you help me understand how to configure my applications so that the deregistration completes successfully before the connection pools are destroyed?

Environment
Spring Boot: 3.2.1
Spring Cloud: 2023.0.0 (eureka-client 2.0.2)
Deployment: Docker Swarm (using stop-first update config)

Configuration
Eureka Server (application.yml):

yaml
spring:
  application:
    name: service-registry
  cloud:
    inetutils:
      preferred-networks:
        - XXX.XXX.XXX
eureka:
  server:
    eviction-interval-timer-in-ms: 60000 
    response-cache-update-interval-ms: 10000 
  client:
    register-with-eureka: false
    fetch-registry: false
    service-url:
      defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
  instance:
    hostname: localhost

Eureka Client / Microservice (eureka-properties.yml via Config Server):

eureka:
  client:
    service-url:
      defaultZone: ${EUREKA_SERVICE_ADDRESS:http://localhost:8761/eureka/}
    register-with-eureka: true
    fetch-registry: true
  instance:
    prefer-ip-address: true
    instance-id: ${spring.application.name}:${spring.application.instance_id:${random.value}}
spring:
  cloud:
    inetutils:
      preferred-networks:
        - XXX.XXX.XXX

Shutdown Logs
Here are the logs during the shutdown of one of the microservices (order-service). It shows the sequence of events leading up to the failure:

13:50:42.773  INFO --- [ionShutdownHook] o.s.c.n.e.s.EurekaServiceRegistry        : Unregistering application ORDER-SERVICE with eureka with status DOWN
13:50:42.773  INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient    : Saw local status change event StatusChangeEvent [timestamp=1779112242773, current=DOWN, previous=UP]
13:50:42.773  INFO --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ORDER-SERVICE/order-service:373b0e4bac...: registering service...
13:50:42.777  INFO --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ORDER-SERVICE/order-service:373b0e4bac... - registration status: 204
13:50:42.781  INFO --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
13:50:42.782  INFO --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
13:50:42.786  INFO --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
13:50:42.786  INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient    : Shutting down DiscoveryClient ...
13:50:45.787  INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient    : Unregistering ...
13:50:45.791  INFO --- [ionShutdownHook] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://registry-dev:8761/eureka/} exception=Connection pool shut down stacktrace=java.lang.IllegalStateException: Connection pool shut down
13:50:45.792  WARN --- [ionShutdownHook] c.n.d.s.t.d.RetryableEurekaHttpClient    : Request execution failed with message: Connection pool shut down
13:50:45.793 ERROR --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ORDER-SERVICE/order-service:373b0e4bac... - de-registration failedCannot execute request on any known server
13:50:45.797  INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient    : Completed shut down of DiscoveryClient

order.logs.txt

Is there a specific configuration I am missing to ensure the deregistration happens before the connection pools are closed? Any guidance on how to resolve this would be greatly appreciated.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions