Exceeded limit on max bytes to buffer : 262144

 

에러 발생 개요

   스프링에서 Webclient를 이용하여 ElasticSearch에 질의를 하는 과정에서 다음 과 같은 에러가 발생

org.springframework.web.reactive.function.client.WebClientResponseException: 200 OK from POST http://111.111.111.111:9200/_msearch
	at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:341) ~[spring-webflux-6.0.11.jar:6.0.11]
	at org.springframework.web.reactive.function.client.DefaultClientResponse.lambda$createException$1(DefaultClientResponse.java:213) ~[spring-webflux-6.0.11.jar:6.0.11]
	at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106) ~[reactor-core-3.5.9.jar:3.5.9]
					.
					.
					.                                       
Suppressed: java.lang.Exception: #block terminated with an error
    at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:103) ~[reactor-core-3.5.9.jar:3.5.9]
    at reactor.core.publisher.Mono.block(Mono.java:1712) ~[reactor-core-3.5.9.jar:3.5.9]
					.
					.
					.                                                      
Caused by: org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
	at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99) ~[spring-core-6.0.11.jar:6.0.11]
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
	*__checkpoint ⇢ Body from POST http://111.111.111.111:9200/_msearch [DefaultClientResponse]

 

 

원인

- WebClient는 Spring 5에 도입된 비동기 및 논 블로킹 HTTP 클라이언트로, Spring WebFlux에서 제공되는 기능 중 하나로 Spring WebFlux는 애플리케이션 메모리 문제를 방지하기 위해 메모리에 버퍼링 할 최대 바이트 수가 기본적으로 256KB 으로 제한되어 있습니다.

 

 

해결

- pring WebFlux에서 256으로 제한되어 있던 버퍼 크기를 maxInMemorySize 설정을 통해 늘려 주면 된다.

    - 기존 코드

 WebClient.builder()
    .baseUrl(elasticsearchUrl)
    .defaultHeader("Authorization", "Basic " + encodedCredentials).build();

       

    - 변경 코드

WebClient.builder()
    .baseUrl(elasticsearchUrl)
    .defaultHeader("Authorization", "Basic " + encodedCredentials)
    .exchangeStrategies(ExchangeStrategies.builder().codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(500 * 1024)).build()).build();

 

※ 참고

• https://www.baeldung.com/spring-webflux-databufferlimitexception
• https://docs.spring.io/spring-framework/docs/5.2.6.RELEASE/spring-framework-reference/web-reactive.html#webflux-codecs-limits
• https://stackoverflow.com/questions/59735951/databufferlimitexception-exceeded-limit-on-max-bytes-to-buffer-webflux-error