如何使用 Spring Batch 写入 Azure Blob？

Question

我正在尝试使用 Spring 批处理将逐行 csv 写入 Azure Blob。

自动装配 Azure 存储：

@Value("${azure.storage.ConnectionString}")
private String connectionString;

@Value("${azure.storage.container.name}")
private String containerName;


@Bean
@StepScope
public FlatFileItemWriter<Pojo> writer(
        @Value("#{stepExecutionContext['marketName']}") String marketName,
        @Value("#{jobParameters['startDate']}") String startDate,
        @Value("#{jobParameters['YYYY-MM']}") String yearMonth
) {
    // Create writer instance
    FlatFileItemWriter<Pojo> writer = new FlatFileItemWriter<>();

    String fullPath = yearMonth + "/" +  marketName + "/" +
            marketName + "_" + startDate;

    BlobContainerClient container = new BlobContainerClientBuilder()
            .connectionString(connectionString)
            .containerName(containerName)
            .buildClient();

    BlobClient blob = container.getBlobClient(fullPath);

    // Set output file location
    writer.setResource((Resource) blob);

    // All job repetitions should "append" to same output file
    writer.setAppendAllowed(true);

    // Name field values sequence based on object properties
    writer.setLineAggregator(new DelimitedLineAggregator<Pojo>() {
        {
            setDelimiter(",");
            setFieldExtractor(new BeanWrapperFieldExtractor<Pojo>() {
                {
                    setNames(new String[] {
                            "market",
                            "epsg",
                            "xbin",
                            "ybin",
                            "latitude",
                            "longitude",
                            "totalDownlinkVol",
                            "totalUplinkVol"
                    });
                }
            });
        }
    });
    return writer;
}

然后我得到错误：

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.writer' defined in class path resource [com/example/dbreader/configuration/BatchConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.item.file.FlatFileItemWriter]: Factory method 'writer' threw exception; nested exception is java.lang.ClassCastException: com.azure.storage.blob.BlobClient cannot be cast to org.springframework.core.io.Resource

Spring 抱怨我无法将 Blob 转换为资源。

有人知道如何在 Spring Batch Writer 中指向 Azure Blob 存储吗？

谢谢，马库斯。

Answer 1

FlatFileItemWriter需要org.springframework.core.io.Resource写入数据。如果您使用的 API 没有实现此接口，则它不能与 FlatFileItemWriter 一起使用。您需要为 Azure 提供 Resource 实现或寻找实现它的库，例如 Azure Spring Boot Starter Storage client library for Java.

如何使用 Spring Batch 写入 Azure Blob？

How to write to Azure Blob with Spring Batch?

spring

azure

azure-storage

spring-batch

azure-blob-storage