如何使用 Sqoop 将 Postgres 表（几何格式）导入 Hive？

Question

我有 Geometry 类型的 Postgres 表 data.Now 我正在尝试使用 Sqoop.Then 将 Postgres 表导入 Hive，它给出了这个错误，

sqoop import --connect jdbc:postgresql://192.168.1.130:5432/Analyst_Bangalore --username postgres --password track@123 --table state -m 1 --warehouse-dir /user/hive/warehouse/analytics.db

 ERROR orm.ClassWriter (ClassWriter.java:generateFields(560)) - Cannot resolve SQL type 1111
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateEquals(645)) - Cannot resolve SQL type 1111
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateEquals(645)) - Cannot resolve SQL type 1111
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbRead(718)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbRead(718)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateLoadLargeObjects(798)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateLoadLargeObjects(798)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbWrite(884)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,116 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbWrite(884)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,116 ERROR orm.ClassWriter (ClassWriter.java:myGenerateHadoopRead(955)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,116 ERROR orm.ClassWriter (ClassWriter.java:myGenerateHadoopWrite(1595)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR orm.ClassWriter (ClassWriter.java:myGenerateHadoopWrite(1595)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR orm.ClassWriter (ClassWriter.java:myGenerateToString(1314)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR orm.ClassWriter (ClassWriter.java:myGenerateToString(1314)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR sqoop.Sqoop (Sqoop.java:runSqoop(181)) - Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.sqoop.orm.ClassWriter.parseNullVal(ClassWriter.java:1377)
        at org.apache.sqoop.orm.ClassWriter.parseColumn(ClassWriter.java:1402)
        at org.apache.sqoop.orm.ClassWriter.myGenerateParser(ClassWriter.java:1528)

任何人都知道如何将几何数据导入配置单元，请建议me.Any 不胜感激。

Answer 1

Sqoop 默认不支持几何数据类型。您需要为 Java（ 对于 POJO class）手动添加它的映射。

在您的代码中添加此标记：

--map-column-java the_geom=String

最终命令：

sqoop import \
 --connect jdbc:postgresql://192.168.1.130:5432/Analyst_Bangalore \
 --username postgres \
 --password track@123 \
 --map-column-java the_geom=String \
 --table state \
 -m 1 \
 --warehouse-dir /user/hive/warehouse/analytics.db

如果您使用 hive import.

添加 --map-column-hive the_geom=string

如何使用 Sqoop 将 Postgres 表（几何格式）导入 Hive？

How to import Postgres tables(Geometry format) into Hive using Sqoop?

postgresql

hadoop

sqoop