下载文件时内存不足

Question

我正在尝试下载 android 中的 CSV 文件。下载后，我将使用 Jackson 进行映射，然后保存到 sqlite3 db。我面临的问题是，当我拨打下载文件的电话时，我收到 OutOfMemoryError。文件大小约为 12MB。当我在浏览器中尝试相同的文件时，我能够下载文件，所以我认为端点没有问题。

要下载，我使用的是 AsyncTask。

class DownloadCSVTask extends AsyncTask<Void, String, Void> {

    private final Call<ResponseBody> csvDump;
    private final Context context;

    public DownloadCSVTask(Call<ResponseBody> csvDump, Context context) {
        this.csvDump = csvDump;
        this.context = context;
    }

    @Override
    protected Void doInBackground(Void... voids) {
        try {
            Response<ResponseBody> response = csvDump.execute();
            File fileFromResponse = createFileFromResponse(response);
            mapFileContentsToModelAndPersist(fileFromResponse);
        } catch (IOException e) {
            Log.e(getClass().getSimpleName(), "CSV pull failed");
        }
        return null;
    }

    private File createFileFromResponse(Response<ResponseBody> response) throws IOException {
        File path = context.getFilesDir();
        File csvFile = File.createTempFile(TEMP_PREFIX, CSV_EXTENSION, path);
        FileOutputStream fileOutputStream = context.openFileOutput(csvFile.getName(), Context.MODE_PRIVATE);
        fileOutputStream.write(response.body().bytes());
        Log.i(getClass().getCanonicalName(), "Writing file completed" + csvFile.getAbsolutePath());
        fileOutputStream.close();
        return csvFile;
    }

    private void mapFileContentsToModelAndPersist(File fileFromResponse) {
        try {
            MappingIterator<City> cityIter = new CsvMapper().readerWithTypedSchemaFor(City.class).readValues(fileFromResponse);
            List<City> cities = cityIter.readAll();
            Stream.of(cities).forEach(City::persist);
        } catch (IOException e) {
            Log.e(getClass().getSimpleName(), "Failed to map contents of file to City Class");
        }
    }
}

我收到的错误日志：

 java.lang.RuntimeException: An error occured while executing doInBackground()
                         E      at android.os.AsyncTask.done(AsyncTask.java:300)
                         E      at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
                         E      at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
                         E      at java.util.concurrent.FutureTask.run(FutureTask.java:242)
                         E      at android.os.AsyncTask$SerialExecutor.run(AsyncTask.java:231)
                         E      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
                         E      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
                         E      at java.lang.Thread.run(Thread.java:841)
                         E  Caused by: java.lang.OutOfMemoryError
                         E      at java.lang.String.<init>(String.java:354)
                         E      at java.lang.String.<init>(String.java:393)
                         E      at okio.Buffer.readString(Buffer.java:616)
                         E      at okio.Buffer.readString(Buffer.java:599)
                         E      at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:263)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
                         E      at com.someclass.somepath.network.util.CustomHeadersInterceptor.intercept(CustomHeadersInterceptor.java:42)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
                         E      at com.someclass.somepath.network.util.NetworkInterceptor.intercept(NetworkInterceptor.java:23)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
                         E      at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:170)
                         E      at okhttp3.RealCall.execute(RealCall.java:60)
                         E      at retrofit2.OkHttpCall.execute(OkHttpCall.java:174)
                         E      at retrofit2.ExecutorCallAdapterFactory$ExecutorCallbackCall.execute(ExecutorCallAdapterFactory.java:89)
                         E      at com.someclass.somepath.service.DownloadCSVTask.doInBackground(CsvPullHandler.java:60)
                         E      at com.someclass.somepath.service.DownloadCSVTask.doInBackground(CsvPullHandler.java:47)
                         E      at android.os.AsyncTask.call(AsyncTask.java:288)
                         E      at java.util.concurrent.FutureTask.run(FutureTask.java:237)

我是不是做错了什么？还有其他方法吗？

Answer 1

Android 上 Java 代码的内存量限制为 16 Mb（因设备而异）。因此，您要么需要将这部分移动到本机 NDK 代码，您可以在其中 malloc() 与设备拥有的内存一样多，要么将文件直接保存到磁盘，然后在不将整个文件加载到 Java 缓冲区的情况下处理它. 在具有 16Kb 字节数组的循环中使用 URLConnection.getInputStream() 或 URLConnection.read()，而不是将整个文件复制到 response 数组中。或者您可以使用 BufferedReader.readLine() 直接从 HTTP 流中逐行读取它，然后逐行填充您的数据库。

下载文件时内存不足

Out of memory while downloading file

java

android

download

out-of-memory

android-asynctask