M Language Multiple json 查询总结果

Question

我休息一下 api 查询 returns 我最多 500 条记录。我也可以用 api 得到记录总数。

因此，我需要合并多个查询，以获取我的所有记录。我还需要该查询是动态的，并且始终与总记录保持同步。

这是最多 1000 条记录的示例代码：

let
    Source = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=0")),
    Source2 = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=500")),
    data = Source[data],
    data2 = Source2[data],
    #"Converted to Table" = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Converted to Table 2" = Table.FromList(data2, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"}),
    #"Expanded Column2" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"}),
    #"Appended Query" = Table.Combine({#"Expanded Column1", #"Expanded Column2"})
in
    #"Expanded Column1"

我的总记录大约是 11000 条，而且每周都在变化。

我可以使用data.additional_data.total_records获得记录总数。

我可以在变量上使用这个值，并使用一段时间来进行总计（总记录数 / 500）的查询吗？

有什么想法吗？

Answer 1

您可以将查询参数化并将其转换为函数，而不是获取数据并一一合并查询。

让我们假设您的原始查询如下：

let
    Source = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=0")),
    data = Source[data],
    #"Converted to Table" = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"})
in
    #"Expanded Column1"

您可以参数化 start 参数并将查询更改为 GetDeal 函数，如下所示：

(start as text) =>
let
    Source = Json.Document(Web.Contents("https://example.pipedrive.com/v1/deal:(id,title,value,currency)?api_token=12345&limit=500&start=" & start)),
    data = Source[data],
    #"Converted to Table" = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "title", "value", "currency"}, {"Column1.id", "Column1.title", "Column1.value", "Column1.currency"})
in
    #"Expanded Column1"

假设您通过导航到 data.additional_data.total_records:

将 total_records 存储为来自另一个查询的值

您现在可以创建新查询并生成从 0 到 total_records 的列表，增量为 500:

let
    Source = List.Generate(() => 0, each _ <= (total_records), each _ + 500)
in
    Source

将其转换为 table 并将列更改为 text:

let
    Source = List.Generate(() => 0, each _ <= (total_records), each _ + 500),
    #"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "start"}}),
    #"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"start", type text}})
in
    #"Changed Type"

现在我们可以使用列 start 调用自定义函数 GetDeal 作为 :

你应该有一个 table 的列表作为一个新的列并且可以从那里开始工作。您可以删除 start 列，因为它不再需要了。

M Language Multiple json 查询总结果

M Language Multiple json queries for total results

json

m

powerquery

powerbi