-- --schema 不适用于 Sqoop create-hive-table

Question

HDP-2.5.0.0 使用 Ambari 2.4.0.1

我可以从 SQL 服务器源数据库在 HCatalog 中创建一个 table，例如：

sqoop import --null-string '\N' --null-non-string '\N' --hive-delims-replacement '[=11=]D' --hcatalog-home /usr/hdp/current/hive-webhcat --hcatalog-database MS_Management_Coaching --hcatalog-table TripAggregate --create-hcatalog-table --hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="ZLIB")' --validate --connect 'jdbc:sqlserver://<DB server>;database=Management' --username uname--password pwd--table TripAggregate -- --schema Coaching

但是当我尝试使用 --create-hive-table 时，--schema 选项不起作用，无论我将其放置在何处：

-bash-4.2$ sqoop create-hive-table --hive-database test --connect 'jdbc:sqlserver://<DB Server>;database=Management' --username uname--password pwd--table TripAggregate -- --schema Coaching
Warning: /usr/hdp/2.5.0.0-1245/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/10/12 21:28:13 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.5.0.0-1245
16/10/12 21:28:13 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/10/12 21:28:13 ERROR tool.BaseSqoopTool: Error parsing arguments for create-hive-table:
16/10/12 21:28:13 ERROR tool.BaseSqoopTool: Unrecognized argument: --
16/10/12 21:28:13 ERROR tool.BaseSqoopTool: Unrecognized argument: --schema
16/10/12 21:28:13 ERROR tool.BaseSqoopTool: Unrecognized argument: DriverCoaching
Try --help for usage instructions.

Answer 1

If the argument -- is given on the command-line, then subsequent arguments are sent directly to the underlying tool.

在查看 sqoop 代码后，我发现在 --create-hive-table 流程中并没有转到底层工具。这就是您无法在命令中使用 -- --schema 的原因。

source code 对 ImportTool 有用的部分：

 public void validateOptions(SqoopOptions options)
      throws InvalidOptionsException {

    // If extraArguments is full, check for '--' followed by args for
    // mysqldump or other commands we rely on.
    options.setExtraArgs(getSubcommandArgs(extraArguments));
    int dashPos = getDashPosition(extraArguments);

    if (hasUnrecognizedArgs(extraArguments, 0, dashPos)) {
      throw new InvalidOptionsException(HELP_STR);
    }

    validateImportOptions(options);
    validateIncrementalOptions(options);
    validateCommonOptions(options);
    validateCodeGenOptions(options);
    validateOutputFormatOptions(options);
    validateHBaseOptions(options);
    validateHiveOptions(options);
    validateHCatalogOptions(options);
    validateAccumuloOptions(options);
  }

source code 对 CreateHiveTable 有用的部分：

 public void validateOptions(SqoopOptions options)
      throws InvalidOptionsException {

    if (hasUnrecognizedArgs(extraArguments)) {
      throw new InvalidOptionsException(HELP_STR);
    }

    validateCommonOptions(options);
    validateOutputFormatOptions(options);
    validateHiveOptions(options);

    if (options.getTableName() == null) {
      throw new InvalidOptionsException(
          "--table is required for table definition importing." + HELP_STR);
    }
  }

你看不到 -- args 的检查在后面完成。

编辑：

--hive-import 默认创建配置单元 table，您可以使用 -- --schema 和 import 命令。如果您希望 sqoop 为您创建配置单元 table 并在其中导入数据 table。它应该适合你。

-- --schema 不适用于 Sqoop create-hive-table

-- --schema doesn't work with Sqoop create-hive-table

sqoop

hortonworks-data-platform