解析 Databricks 中嵌套的 XML

Question

我正在尝试 p

我正在尝试将 XML 读入数据框并尝试使用如下所示的爆炸来展平数据。

val df = spark.read.format("xml").option("rowTag","on").option("inferschema","true").load("文件路径" ) val parsxml= df .withColumn("exploded_element", explode(("prgSvc.element"))).

我收到以下错误。

command-5246708674960:4: error: type mismatch;
found   : String("prgSvc.element")
required: org.apache.spark.sql.Column
.withColumn("exploded_element", explode(("prgSvc.element")))**

Before reading the XML into the data frame, I also tried to manually assign a custom schema and read the XML file. But the output is all NULL. Could you please let me know if my approach is valid and how to resolve this issue and achieve the output.
Thank you.

Answer 1

使用这个

import spark.implicits._

val parsxml= df .withColumn("exploded_element", explode($"prgSvc.element"))

解析 Databricks 中嵌套的 XML

Parsing nested XML in Databricks

scala

apache-spark

apache-spark-sql

azure-databricks