（解决）insert to bigquery not same row number input

Question

我使用 google bigquery API 将数据插入 bigquery

public void upload(Collection<Bucket> buckets){
log.finer("Uploading " + buckets.size() + " buckets to bigquery");
List<Rows> rows=new ArrayList<Rows>();
for (  Bucket bucket : buckets) {
TableRow row=new TableRow();
row.set("uploaded",System.currentTimeMillis() / 1000f);
row.set("codepoint",bucket.getKey().getCodepoint());
row.set("namespace",bucket.getKey().getNamespace());
row.set("kind",bucket.getKey().getKind());
row.set("op",bucket.getKey().getOp());
row.set("query",bucket.getKey().getQuery());
row.set("time",bucket.getKey().getTime() / 1000f);
row.set("reads",bucket.getReads());
row.set("writes",bucket.getWrites());
TableDataInsertAllRequest.Rows rowWrapper=new    TableDataInsertAllRequest.Rows();
rowWrapper.setJson(row);
rows.add(rowWrapper);
}
TableDataInsertAllRequest request=new      
TableDataInsertAllRequest().setRows(rows);
String tableId=tablePicker.pick();
try {
TableDataInsertAllResponse response=bigquery.tabledata().insertAll(insightDataset.projectId(),insightDataset.datasetId(),tableId,request).execute();
if (response.getInsertErrors() != null && !response.getInsertErrors().isEmpty()) {
  throw new RuntimeException("There were errors! " + response.getInsertErrors());
}
}
 catch (  IOException e) {
throw new RuntimeException(e);
}
}

我的问题：

我使用 2 个服务器调用函数同时插入到 bigquery，同一个帐户 bigquery。

示例：

服务器 A：有 500 行

服务器 B：有 400 行

插入到google bigquery 900行就对了

但是当我签入时 google bigquery 有 1300 行。

当我用 1 台服务器进行测试时，它向右插入（服务器 A 有 500 行，服务器 B 有 400 行）。

你能帮我解决这个问题吗？

Answer 1

您应该正确设置insertId 属性。您需要了解如何使用您使用的库执行此操作。

public TableDataInsertAllRequest.Rows setInsertId(java.lang.String insertId)
[Optional] A unique ID for each row. BigQuery uses this property to detect duplicate insertion requests on a best-effort basis.
Parameters:
insertId - insertId or null for none

https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/TableDataInsertAllRequest.Rows.html#setInsertId(java.lang.String)

（解决）insert to bigquery not same row number input

(Resolve) Insert to bigquery not same row number input

bigdata

google-bigquery