如何变大 JSON 使用 REST 模板 Spring MVC java 中没有内存问题

Question

当我从 REST 服务变大 JSON 时，我收到 java.lang.OutOfMemoryError：Java 堆 space...

这是在 ~1000 调用 REST 服务后发生的

    HttpEntity<ProductRequest> requestUpdate = new HttpEntity<>(request, httpHeaders);

    ResponseEntity<String> messageEntity = restTemplate.exchange(getEndPointAddress(), HttpMethod.POST, requestUpdate, String.class);

    Map<String, Object> map = gson.fromJson(Helper.cleanJson(messageEntity.getBody()), Map.class);

<pre>
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOfRange(Arrays.java:2694) ~[?:1.7.0_181]
    at java.lang.String.<init>(String.java:203) ~[?:1.7.0_181]
    at java.lang.StringBuilder.toString(StringBuilder.java:405) ~[?:1.7.0_181]
    at org.springframework.util.StreamUtils.copyToString(StreamUtils.java:80) ~[stormjar.jar:4.0.3]
    at org.springframework.http.converter.StringHttpMessageConverter.readInternal(StringHttpMessageConverter.java:86) ~[stonjar.jar:4.0.3]
    at org.springframework.http.converter.StringHttpMessageConverter.readInternal(StringHttpMessageConverter.java:41) ~[stonjar.jar:4.0.3]
    at org.springframework.http.converter.AbstractHttpMessageConverter.read(AbstractHttpMessageConverter.java:193) ~[stonjar.jar:4.0.3]
    at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:104) ~[stonjar.jar:4.0.3]
    at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:884) ~[stonjar.jar:4.0.3]
    at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:868) ~[stonjar.jar:4.0.3]
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:622) ~[stonjar.jar:4.0.3]
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580) ~[stonjar.jar:4.0.3]
    at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498) ~[stonjar.jar:4.0.3]
    at com.myorg.wcf.PMiddle.retrieveP(PMiddle.java:95) ~[ssar.jar:4.0.3]
    at com.myorg.RetrieveP.getPJsonFromService(RetrieveP.java:188) ~[ssar.jar:4.0.3]
    at com.myorg.RetrieveP.execute(RetrieveP.java:107) ~[ssar.jar:4.0.3]
    at org.apache.storm.daemon.executor$fn__5044$tuple_action_fn__5046.invoke(executor.clj:727) ~[storm-core-1.1.0.jar:1.1.0]
    at org.apache.storm.daemon.executor$mk_task_receiver$fn__4965.invoke(executor.clj:459) ~[storm-core-1.1.0.jar:1.1.0]
    at org.apache.storm.disruptor$clojure_handler$reify__4480.onEvent(disruptor.clj:40) ~[ss-core-1.1.0.jar:1.1.0]
    at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) ~[ss-core-1.1.0.jar:1.1.0]
    at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) ~[ss-core-1.1.0.jar:1.1.0]
    at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) ~[ss-core-1.1.0.jar:1.1.0]
    at org.apache.storm.daemon.executor$fn__5044$fn__5057$fn__5110.invoke(executor.clj:846) ~[ss-core-1.1.0.jar:1.1.0]
    at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) [ss-core-1.1.0.jar:1.1.0]
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
    at java.lang.Thread.run(Thread.java:748) [?:1.7.0_181]</pre>

Answer 1

RestTemplate 和底层 Spring beans 不是为流处理而设计的，它们在内存中构建整个 ResponsEntity。根据您的示例，您没有按预期使用 RestTemplate 。您正在阅读 String 而不是将响应对象 Map<String, Object> 反序列化委托给底层转换器 bean。

如果 JSON 响应真的太大而无法放入您的堆中，您可以将 RestTemplate 替换为 URLConnection 或任何其他允许您访问响应的 HTTP 客户端库 InputStream直接。这将允许您手动读取和解析响应，例如by using com.google.gson.stream.JsonReader:

URLConnection conn = // create and open
JsonReader reader = new JsonReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
// read manually using reader

您将能够通过避免包含整个 JSON 主体的一个大 String 对象来节省内存。

如何变大 JSON 使用 REST 模板 Spring MVC java 中没有内存问题

How to get large JSON Using REST template Spring MVC without memory issues in java

java

spring

out-of-memory

resttemplate