雪花数据透视属性值到 objects 数组中的列

Question

编辑：我给出了错误的示例数据。更新了一些细节并为经过清理的实际数据切换了虚拟数据。

来源系统：来自 Stitch 的 Freshdesk

Table结构：

create or replace TABLE TICKETS (
    CC_EMAILS VARIANT,
    COMPANY VARIANT,
    COMPANY_ID NUMBER(38,0),
    CREATED_AT TIMESTAMP_TZ(9),
    CUSTOM_FIELDS VARIANT,
    DUE_BY TIMESTAMP_TZ(9),
    FR_DUE_BY TIMESTAMP_TZ(9),
    FR_ESCALATED BOOLEAN,
    FWD_EMAILS VARIANT,
    ID NUMBER(38,0) NOT NULL,
    IS_ESCALATED BOOLEAN,
    PRIORITY FLOAT,
    REPLY_CC_EMAILS VARIANT,
    REQUESTER VARIANT,
    REQUESTER_ID NUMBER(38,0),
    RESPONDER_ID NUMBER(38,0),
    SOURCE FLOAT,
    SPAM BOOLEAN,
    STATS VARIANT,
    STATUS FLOAT,
    SUBJECT VARCHAR(16777216),
    TAGS VARIANT,
    TICKET_CC_EMAILS VARIANT,
    TYPE VARCHAR(16777216),
    UPDATED_AT TIMESTAMP_TZ(9),
    _SDC_BATCHED_AT TIMESTAMP_TZ(9),
    _SDC_EXTRACTED_AT TIMESTAMP_TZ(9),
    _SDC_RECEIVED_AT TIMESTAMP_TZ(9),
    _SDC_SEQUENCE NUMBER(38,0),
    _SDC_TABLE_VERSION NUMBER(38,0),
    EMAIL_CONFIG_ID NUMBER(38,0),
    TO_EMAILS VARIANT,
    PRODUCT_ID NUMBER(38,0),
    GROUP_ID NUMBER(38,0),
    ASSOCIATION_TYPE NUMBER(38,0),
    ASSOCIATED_TICKETS_COUNT NUMBER(38,0),
    DELETED BOOLEAN,
    primary key (ID)
);

注意变体字段，"custom_fields"。它经历了 api 和雪花之间的不幸转变。结果字段包含 3 个或更多 objects 的数组，每个字段都是一个自定义字段。我无法更改数据格式。示例：

# values could be null
[
  {
    "name": "cf_request",
    "value": "none"
  },
  {
    "name": "cf_related_with",
    "value": "none"
  },
  {
    "name": "cf_question",
    "value": "none"
  }
]

# or values could have a combination of null and non-null values
[
  {
    "name": "cf_request",
    "value": "none"
  },
  {
    "name": "cf_related_with",
    "value": "none"
  },
  {
    "name": "cf_question",
    "value": "concern"
  }
]

# or they could all have non-null values
[
  {
    "name": "cf_request",
    "value": "issue with timer"
  },
  {
    "name": "cf_related_with",
    "value": "timer stopped"
  },
  {
    "name": "cf_question",
    "value": "technical problem"
  }
]

我基本上想将它们转换为 select 查询中的字段，其中 name 属性的值变成列 header。使输出类似于以下内容：

+----+------------------+-----------------+-------------------+-----------------------------+
| id |    cf_request    | cf_related_with |    cf_question    |      all_other_fields       |
+----+------------------+-----------------+-------------------+-----------------------------+
|  5 | issue with timer | timer stopped   | technical problem | more data about this ticket |
|  6 | hq               | laptop issues   | some value        | more data                   |
|  7 | a thing          | about a thing   | about something   | more data                   |
+----+------------------+-----------------+-------------------+-----------------------------+

是否有一个函数可以用符合条件的值搜索数组 objects 和 returns objects 的值？类似于：

select
id,
get_object_where(name = 'category', value) as category,
get_object_where(name = 'subcategory', value) as category,
get_object_where(name = 'subsubcategory', value) as category
from my_data_table

不幸的是，PIVOT 需要一个聚合函数，我尝试使用最小值和最大值，但只得到一个 return 个空值。如果有另一种不需要聚合的语法，类似于这种方法的东西会很棒。

with arr as (
  select
  id, 
  cs.value:name col_name, 
  cs.value:value col_value
  from my_data_table,
    lateral flatten(input => custom_fields) cs
)
select 
*
from arr 
pivot(col_name for col_value in ('category', 'subcategory', 'subsubcategory')
as p (id, category, subcategory, subsubcategory);

可以使用以下方法，但它的缺陷在于每次添加新的自定义字段时我都必须添加大小写以说明数组中的新位置。

select
    id, 
    case 
      when custom_fields[0]:name = 'cf_request' then custom_fields[0]:value
      when custom_fields[1]:name = 'cf_request' then custom_fields[1]:value
      when custom_fields[2]:name = 'cf_request' then custom_fields[2]:value
      when custom_fields[2]:name = 'cf_request' then custom_fields[3]:value
      else null
    end cf_request,
    case 
      when custom_fields[0]:name = 'cf_related_with' then custom_fields[0]:value
      when custom_fields[1]:name = 'cf_related_with' then custom_fields[1]:value
      when custom_fields[2]:name = 'cf_related_with' then custom_fields[2]:value
      when custom_fields[2]:name = 'cf_related_with' then custom_fields[3]:value
      else null
    end cf_related_with,
    case 
      when custom_fields[0]:name = 'cf_question' then custom_fields[0]:value
      when custom_fields[1]:name = 'cf_question' then custom_fields[1]:value
      when custom_fields[2]:name = 'cf_question' then custom_fields[2]:value
      when custom_fields[2]:name = 'cf_question' then custom_fields[3]:value
      else null
    end cf_question,
    created_at
from my_db.my_schema.tickets;

Answer 1

我想你几乎已经成功了。您只需要在 col_name 周围添加一个 max() 或 min() 即可。正如您所说，它需要一个聚合函数，像 max() 或 min() 这样的函数在这里可以工作，因为它聚合了您拥有的 name/value 对。例如，如果您有 2 个子类别值，它将选择 min/max 值。从您的示例来看，这似乎不是问题，因此它始终会选择您想要的值。我能够使用此查询复制您的场景：

WITH x AS (
    SELECT parse_json('[{"name": "category","value": "Bikes"},{"name": "subcategory","value": "Mountain Bikes"},{"name": "subsubcategory","value": "hardtail bikes"}]')::VARIANT as field_var
    ),
arr as (
  select
  seq,
  cs.value:name::varchar col_name, 
  cs.value:value::varchar col_value
  from x,
    lateral flatten(input => x.field_var) cs
)
select 
*
from arr
pivot(max(col_value) for col_name in ('category','subcategory','subsubcategory')) as p (seq, category, subcategory, subsubcategory);

雪花数据透视属性值到 objects 数组中的列

snowflake pivot attribute values into columns in array of objects

snowflake-cloud-data-platform