Building a Simple Circuit Breaker

In the previous article, we went through basics of circuit breaker, discussed about its state cycle, now let’s head towards building a simple circuit breaker.

In this article, we’ll learn about:

• how to record success/failures
• how to switch between CLOSE and OPEN state (skipping HALF_OPEN for this article)
• how to compute current state of circuit breaker (compute based on last ’n’ number of requests, where n represents the window size for recording the results processed requests through circuit breaker)

Let’s begin with planning the interface for circuit breaker and decide which methods need to be exposed.

Circuit Breaker Interface

public interface CircuitBreaker {

void acquirePermission() throws PermissionNotAcquiredException;

void onSuccess();

void onError();

}

The following methods will be exposed:

• acquirePermission()
  Usage: Acquire permission before executing the incoming request.
  It throws PermissionNotAcquiredException when unable to acquire permission (sample scenario: if any request comes when circuit breaker is in OPEN state and still cooling period is not over).
• onSuccess()
  Usage: Register success after the incoming request is performed successfully.
• onError()
  Usage: Register failure after the incoming request got failed while processing.

Circuit Breaker States

public enum CircuitBreakerState {
CLOSE,
OPEN
}

A queue will be used to keep track of success/failure events.

Event Model and CircuitBreakerEventState

@Builder
@Getter
public class CircuitBreakerEvent {

private CircuitBreakerEventState state;
private Long timestamp;

}public enum CircuitBreakerEventState {
SUCCESS,
FAIL
}

Implementation of Circuit Breaker

public class CountBasedCircuitBreakerImpl implements CircuitBreaker {

 private String name;
 private Integer windowSize = 10;
 private Integer failureRateThreshold = 50;
 private CircuitBreakerState currentState;
 private Queue<CircuitBreakerEvent> eventWindow = new LinkedList<>();
 private long latestTimestampInOpenState;
 private Long waitDurationInOpenState = Long.valueOf(1000 * 60 * 5);
 private int successCountInQueue = 0;
 private int failCountInQueue = 0;
 private List<Class<? extends Exception>> failureExceptionList =
     Arrays.asList(MyTimeoutException.class);

 public CountBasedCircuitBreakerImpl(String name) {
   this.name = name;
 }

 @Override
 public void acquirePermission() throws PermissionNotAcquiredException {
   if (CircuitBreakerState.CLOSE.equals(currentState)) {
     return;
   }
   computePermission();
 }

 private void computePermission() throws PermissionNotAcquiredException {
   if (CircuitBreakerState.OPEN.equals(currentState)) {
     long durationElapsedInOpenState = System.currentTimeMillis() - latestTimestampInOpenState;
     if (durationElapsedInOpenState >= waitDurationInOpenState) {
       updateCurrentState(CircuitBreakerState.CLOSE);
     } else {
       String message = String.format(
           "CircuitBreaker name : '%s' ; state : %s ; failCountInQueue : %d ; "
               + "durationElapsedInOpenState : %d ", name, currentState.name(), failCountInQueue,
           durationElapsedInOpenState);
       throw new PermissionNotAcquiredException(message);
     }
   }
 }

 @Override
 public void onSuccess() {
   addEventToQueue(CircuitBreakerEventState.SUCCESS);
 }

 @Override
 public void onError() {
   addEventToQueue(CircuitBreakerEventState.FAIL);
 }

 @Override
 public <T> T performOperation(Supplier<T> computedResponse, Supplier<T> defaultResponse) {
   try {
     acquirePermission();
     T computedValue = computedResponse.get();
     onSuccess();
     return computedValue;
   } catch (PermissionNotAcquiredException e) {
     return defaultResponse.get();
   } catch (Exception e){
     if (failureExceptionList.contains(e.getClass())){
       onError();
     } else {
       onSuccess();
     }
     throw e;
   }
 }

 private CircuitBreakerEvent createCircuitBreakerEvent(CircuitBreakerEventState state) {
   return CircuitBreakerEvent.builder().state(state).timestamp(System.currentTimeMillis()).build();
 }

 private void addEventToQueue(CircuitBreakerEventState state) {
   if (eventWindow.size() == windowSize) {
     if (CircuitBreakerEventState.FAIL.equals(eventWindow.poll().getState())) {
       failCountInQueue--;
     } else {
       successCountInQueue--;
     }
   }
   eventWindow.add(createCircuitBreakerEvent(state));
   if (CircuitBreakerEventState.FAIL.equals(state)) {
     failCountInQueue++;
   } else {
     successCountInQueue++;
   }
   computeCurrentState();
 }

 private void computeCurrentState() {
   int currentFailureRate = (failCountInQueue * 100 / windowSize);
   if (currentFailureRate >= failureRateThreshold) {
     updateCurrentState(CircuitBreakerState.OPEN);
     latestTimestampInOpenState = System.currentTimeMillis();
   } else {
     updateCurrentState(CircuitBreakerState.CLOSE);
   }
 }

 private void updateCurrentState(CircuitBreakerState state) {
   currentState = state;
 }

}

In above implementation, for onSuccess()/onError(), we add an event at the end of the queue (FIFO — First In First Out) with respective event state and timestamp.

computeCurrentState() is used to compute current state of circuit breaker. It’s calculated using below formula:

This formula is arguable when our queue is not full. As per current formula, if 5 consecutive failures happen initially, in a windowSize of 10 with threshold of 50%, then the current state is changed to OPEN, but actually it’s 100% failure (as we had received only 5 requests and all had failed). Still, I would prefer above formula as it gives the initial breathing space for the circuit breaker before it actually starts to work. Another solution for this problem could be that, to avoid computing current state until thresholdEvents in circuitBreaker are received.

acquirePermission() — In this method, if current state is not CLOSE, then the permission is again computed. This is done because there can be a scenario in which current state is OPEN but request might be coming after cooling period, so it should be allowed and state should be updated to CLOSE. If the request arrives within cooling period, then PermissionNotAcquiredException is thrown.

performOperation() is added in circuit breaker interface to execute request. It takes Supplier for actual computation of the request and default response as arguments. If any exception is encountered during process of request, then we will also able to decide if the exception thrown should be considered as failure or success (example : TimeoutException can be considered as failure while calling other system but other exception which occurred while creation of request before calling other system, can be considered as success as we may not like to record such exception as failures in circuit breaker). Flow for this function is explained using below diagram.

Additionally we can also define a constructor or builder to update the circuit breaker properties, like failureRateThreshold, windowSize etc, which have been skipped for this demo project.

We have created CircuitBreakerRegistry for centrally managing all the circuit breaker in the current service. Below is the implementation for registry, all references to circuit breakers are stored in a map where key is name of circuit breaker and value is the reference to that circuit breaker.

public interface CircuitBreakerRegistry {

 CircuitBreaker circuitBreaker(String circuitBreakerName);

}

public class CircuitBreakerRegistryImpl implements CircuitBreakerRegistry {

 private Map<String, CircuitBreaker> circuitBreakerMap;

 public CircuitBreakerRegistryImpl() {
   circuitBreakerMap = new HashMap<>();
 }

 @Override
 public CircuitBreaker circuitBreaker(String circuitBreakerName) {
   return circuitBreakerMap.containsKey(circuitBreakerName) ?
       circuitBreakerMap.get(circuitBreakerName) :
       addToRegistry(circuitBreakerName, createCircuitBreaker(circuitBreakerName));
 }

 private CircuitBreaker addToRegistry(String circuitBreakerName, CircuitBreaker circuitBreaker) {
   circuitBreakerMap.put(circuitBreakerName, circuitBreaker);
   return circuitBreakerMap.get(circuitBreakerName);
 }

 private CircuitBreaker createCircuitBreaker(String circuitBreakerName) {
   return new CountBasedCircuitBreakerImpl(circuitBreakerName);
 }
}

Now let’s put this circuit breaker into action and use it in an endpoint and try to simulate different scenarios.

Below is the example how we can use circuit breaker to perform computation.

@RestController
public class TestController {

private CircuitBreaker circuitBreaker;

public TestController() {
circuitBreaker = new CountBasedCircuitBreakerImpl("MyCircuitBreaker");
 }

 @GetMapping
public String testEndpoint(@RequestParam("success") Boolean success) {
try {
circuitBreaker.acquirePermission();
return computedResponse(success);
   } catch (PermissionNotAcquiredException e) {
return defaultResponse();
   }
 }

private String computedResponse(Boolean success) {
if (success) {
circuitBreaker.onSuccess();
return "Success response";
   } else {
circuitBreaker.onError();
return "Error response";
   }
 }

private String defaultResponse() {
return "Circuit Breaker in OPEN state. This is default response";
 }
}

Below is another example to perform computation, this time we’ll use performOperation() method in circuitBreaker.

@RestController
public class TestController {

 @Autowired
private CircuitBreakerRegistry circuitBreakerRegistry;

private CircuitBreaker circuitBreaker;

public TestController(CircuitBreakerRegistry circuitBreakerRegistry) {
this.circuitBreakerRegistry = circuitBreakerRegistry;
circuitBreaker = circuitBreakerRegistry.circuitBreaker("TestCircuitBreaker");
 }

 @GetMapping
public String testEndpoint(@RequestParam(value = "success", required = false) Boolean success) {
try {
return circuitBreaker.performOperation(() -> computedResponse(success), this::defaultResponse);
   } catch (Exception e){
return e.getMessage();
   }
 }

private String computedResponse(Boolean success) {

if (Objects.nonNull(success)) {
if (success) {
circuitBreaker.onSuccess();
return "Success response";
     } else {
throw new MyTimeoutException("Failure in Circuit Breaker");
     }
   } else {
throw new RuntimeException("Not a failure in Circuit Breaker");
   }
 }

private String defaultResponse() {
return "Circuit Breaker in OPEN state. This is default response";
 }
}

That’s it !

We have our own working circuit breaker.

Code is available on github (branch : “count-based-circuit-breaker”)— https://github.com/rsbeoriginal/SimpleCircuitBreakerImplemenation/tree/count-based-circuit-breaker

‍