While developing spring cloud applications, we may have multiple micro-services communicating with each other.

A remote service can fail to return the response at any time. We need to create a resilient system, which should be able to handle these service call failures to avoid complete application failure.

In a client resilient application, a client service will recover from crashing even if the remote service is not responding. This pattern is called the client resiliency pattern.

In this article, we will learn how to implement client resiliency patterns like load balancing, circuit breakers, fall back, and bulkhead, etc.

Client side load balancing

In the client-side load balancing technique, the load balancer sits between service and the client. It handles the service invocation by using the list of available registered services.

We have learned about client-side load balancing using the Ribbon load balancer backed RestTemplate in the previous article.

Circuit breaker

Circuit breaker pattern monitors the call to remote service, detects any slowness in receiving the response and breaks the connection if the remote service is failing to respond within the time slot. It is a fail-fast approach.

Client side circuit breaker with Netflix Hystrix

Spring Cloud Netflix Hystrix library provides circuit breaker support with fallback method support.

Create a Eureka server

Create a Eureka server. We use the eureka server as a service discovery registry. Other services can register to this service registry as a eureka client so that they can be available for service discovery.

Check out the this article to know how to set up the eureka server.

Create a remote Eureka client service

Once the eureka server is set up, we can register other microservices into this eureka service registry.

Let us create a simple service with name spring-boot-eureka-client and add spring-cloud-starter-netflix-eureka-client, spring-boot-starter-web dependencies.

Below is the pom.xml file after creating the application.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.1.6.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.asb.example</groupId>
	<artifactId>spring-boot-eureka-client</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-boot-eureka-client</name>
	<description>Demo project for Spring Boot</description>

	<properties>
		<java.version>1.8</java.version>
		<spring-cloud.version>Greenwich.SR2</spring-cloud.version>
		<maven-jar-plugin.version>3.1.1</maven-jar-plugin.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>${spring-cloud.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>
</project>

Add the following properties to application.properties file under src/main/resources/ directory.

#set the application name, which is used for service registry
spring.application.name=my-client
#server port
server.port=8080

Update the spring boot bootstrap class with below code.

@SpringBootApplication
@EnableEurekaClient
@RestController
public class SpringBootEurekaClientApplication {

	@GetMapping("/call-me")
	public String method() throws InterruptedException {
		
		Thread.sleep(5000);
		return "You are calling me through service discovery!!";
	}

	public static void main(String[] args) {
		SpringApplication.run(SpringBootEurekaClientApplication.class, args);
	}
}

In the above code, we have created an endpoint called “/call-me“.
Also, notice the Thread.sleep() method, which adds a slight delay(delay of 5 seconds) before returning the response to the caller.

Create a service with Hystrix circuit breaker

Let us create another service to invoke the endpoint “/call-me” of the eureka client service, which we created earlier.

Create a service with name spring-boot-hystrix-example and spring-cloud-starter-netflix-hystrix, spring-cloud-starter-netflix-eureka-client and spring-boot-starter-web dependencies.

Below is the complete content of the pom.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.1.7.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.asb.example</groupId>
	<artifactId>spring-boot-hystrix-example</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-boot-hystrix-example</name>
	<description>Demo project for Spring Boot</description>

	<properties>
		<java.version>1.8</java.version>
		<spring-cloud.version>Greenwich.SR2</spring-cloud.version>
		<maven-jar-plugin.version>3.1.1</maven-jar-plugin.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>${spring-cloud.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

We have added eureka client dependency to enable Netflix ribbon load balancer. We are going to use the ribbon baked RestTemplate to invoke the external endpoint /call-me.

Enable Hystrix circuit breaker

The next step is to add the @EnableCircuitBreaker annotation to our spring boot bootstrap class.

@EnableCircuitBreaker annotation enables the circuit breaker functionality to use in any of our application methods.

@SpringBootApplication
@EnableCircuitBreaker
public class SpringBootHystrixExampleApplication {
	//main method
}

To enable the circuit breaker in any of the methods in our application, we can use @HystrixCommand annotation. The @HystrixCommand annotation wraps the method with the Hystrix circuit breaker.

Below is the complete code of our application’s bootstrap class.

@SpringBootApplication
@EnableCircuitBreaker
@RestController
public class SpringBootHystrixExampleApplication {

	@Bean
	@LoadBalanced
	public RestTemplate restTemplate() {
		return new RestTemplate();
	}

	@Autowired
	private RestTemplate restTemplate;

	@GetMapping("/call-hystrix-client")
	@HystrixCommand
	public String method() {

		String response = restTemplate.getForObject("http://my-client/call-me/", String.class);
		return "Response : " + response;
	}

	public static void main(String[] args) {
		SpringApplication.run(SpringBootHystrixExampleApplication.class, args);
	}
}

Now run the applications in the below order.

  • Run the Eureka server.
  • Run the Eureka client application(spring-boot-eureka-client).
  • Run the application spring-boot-hystrix-example.

We should be able to get the response as shown below.

Netflix hystrix circuit breaker example

This is happy flow. We got the response as the remote service responded within default timeout value.

Change the /call-me endpoint’s response delay to more higher value by modifying the code to Thread.sleep(11000);

Now hit the endpoint with the postman. We will receive the error as remote service is not responding within the default circuit breaker timeout period.

hystrix circuit breaker failed
Customizing the circuit breaker

We can customize the circuit breaker timeout period by using @HystrixProperty annotation. The below example shows how to set the circuit breaker timeout to 16 seconds.

@GetMapping("/call-hystrix-client-with-delay")
@HystrixCommand(commandProperties = {
@HystrixProperty(name = execution.isolation.thread.timeoutInMilliseconds", value = "16000") })
public String hystrixWithDelay() {

	String response = restTemplate.getForObject("http://my-client/call-me/", String.class);
	return "Response : " + response;
}

Client side fall back with Netflix Hystrix

Circuit breaker breaks the connection if the called service is not responding. We can use a fall back method to handle the circuit breaker exception and send a default response back to the service client.

To enable fallback functionality, we need to create a fallback method in our controller class. Then we have to set the property “fallbackMethod” with our fallback method name as value.

@GetMapping("/call-hystrix-client-with-fallback")
@HystrixCommand(commandProperties = {
	@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "16000") },
fallbackMethod = "fallBackMethod")
public String hystrixWithFallback() {

	String response = restTemplate.getForObject("http://my-client/call-me/", String.class);
	return "Response : " + response;
}

//fallback method:
public String fallBackMethod() {

	return "Looks like my-client service is not reponding!!";
}

Now check the service again. We should be able to get the fallback response.

Hystrix fallback example

We can set default circuit breaker properties at a class level using @DefaultProperties annotation.

@DefaultProperties(commandProperties = { @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "10000") })

Bulkhead with Netflix Hystrix

A microservice may call many external services. By default, these service calls use the current Java container’s thread for the program execution.

This may cause the entire application to crash if a high volume of the load is present on the system, and service calls are using too many resources.

Bulkhead pattern segregates the remote service calls into a separate thread pool and solves this problem.

Hystrix uses a thread pool for remote service calls. By default, this thread pool contains a size of 10 threads. This thread pool may not be sufficient for high volume applications.

Hystrix also supports a mechanism, where each service call can have a segregated thread pool.

The below example shows the customized thread pool setup for our external service call.

@GetMapping("/call-hystrix-client-with-fallback")
@HystrixCommand(commandProperties = {
	@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "10000") }, fallbackMethod = "fallBackMethod", 
threadPoolKey = "myServicePool", 
threadPoolProperties = {
	@HystrixProperty(name = "coreSize", value = "40"),
	@HystrixProperty(name = "maxQueueSize", value = "10") })
public String hystrixWithFallback() {

	String response = restTemplate.getForObject("http://my-client/call-me/", String.class);
	return "Response : " + response;
}
  • threadPoolProperties: Allows us to define our custom thread pool behavior.
  • threadPoolKey: Allows us to define unique name for the thread pool.
  • coreSize: Allows us to define maximum threads available in the thread pool.
  • maxQueueSize: Allows us to set size of the thread queue.

Conclusion

In this article, we learned about different client-side resiliency patterns like load balancing, circuit breaker, fallback, and bulkhead patterns. We learned how to use Netflix Hystrix library to achieve these patterns.

Sample example code is available on Github.

Happy coding 🙂 🙂

You may also interested in