可以通过直线连接到配置单元，但不能 java。错误 Could not open client transport with JDBC Uri GSS initiate failed
Can connect to hive through beeline, but not java. Error Could not open client transport with JDBC Uri GSS initiate failed
我可以访问启用了 kerberos 的远程 hdfs 集群。当我尝试使用直线连接时
beeline -u "jdbc:hive2://host:port/arstel;principal=principal"
连接成功。
但是当我编写一个 java 应用程序并使用相同的 JDBC Uri 时，我得到
ERROR transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
jdbc.HiveConnection: Could not open client transport with JDBC Uri: jdbc:hive2://host:port/arstel;principal=principal
Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://host:port/arstel;principal=principal: GSS initiate failed
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
我的应用程序
public class RecordController {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException, ClassNotFoundException {
Class.forName(driverName);
Connection con = DriverManager
.getConnection("jdbc:hive2://host:port/arstel;principal=principal");
Statement stmt = con.createStatement();
String tableName = "evkuzmin_testHiveDriverTable";
stmt.execute("drop table if exists " + tableName);
stmt.execute("create table " + tableName + " (key int, value string)");
}
}
为什么会出现这种情况，我该如何解决？
这个我也试过了。相同的结果。
getConnection("jdbc:hive2://host:port/arstel;principal=principal;" +
"auth=kerberos;" +
"kerberosAuthType=fromSubject");
编辑
我没有密钥表。在堆栈上阅读它和 here 后，我按照说明创建了一个。然后我以这种方式将属性添加到我的 java 文件中
Class.forName(driverName);
System.setProperty("java.security.auth.login.config","gss-jaas.conf");
System.setProperty("sun.security.jgss.debug","true");
System.setProperty("javax.security.auth.useSubjectCredsOnly","false");
System.setProperty("java.security.krb5.conf","krb5.conf");
我添加后resources
gss-jaas.conf
com.sun.security.jgss.initiate {
com.sun.security.auth.module.Krb5LoginModule=required
useKeyTab=true
useTicketCache=false
principal="hive/host@DOMAIN"
doNotPrompt=true
keyTab="EvKuzmin.keytab"
debug=true;
}
和krb5.conf
[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = DOMAIN
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
[realms]
DOMAIN = {
admin_server = ms-dcs.DOMAIN
kdc = ms-dcs.DOMAIN
}
[domain_realm]
.DOMAIN = DOMAIN
DOMAIN = DOMAIN
然后错误改了
java.sql.SQLException: Could not create secure connection to jdbc:hive2://host:port/arstel;principal=hive/principal;auth=kerberos;kerberosAuthType=fromSubject: Failed to open client transport
Caused by: javax.security.sasl.SaslException: Failed to open client transport [Caused by java.io.IOException: Could not instantiate SASL transport]
Caused by: java.io.IOException: Could not instantiate SASL transport
Caused by: javax.security.sasl.SaslException: Failure to initialize security context [Caused by GSSException: Invalid name provided (Mechanism level: Cannot locate default realm)]
Caused by: GSSException: Invalid name provided (Mechanism level: Cannot locate default realm)
还有
这里已经有一个密钥表
Keytab name: FILE:/etc/krb5.keytab
但我不知道它是什么领域或如何使用它 + 我无权访问该文件夹。
编辑 2 2017-08-09
尝试通过 UserGroupInformation 进行操作。出现错误
Exception in thread "main" java.lang.IllegalArgumentException: Invalid attribute value for hadoop.security.authentication of Kerberos
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:200)
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:227)
at com.hive.connect.controller.RecordController.main(RecordController.java:20)
EDIT3 2017-08-10
使用用户组
public class RecordController {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException, ClassNotFoundException, IOException {
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab("hive/principal", "EvKuzmin.keytab");
Class.forName("org.apache.hive.jdbc.HiveDriver");
Connection con = DriverManager
.getConnection("jdbc:hive2://host:port/arstel;" +
"principal=hive/principal;" +
"tez.queue.name=adhoc;" +
"hive.execution.engine=tez;" +
"mapreduce.job.reduces=1;");
错误
java.io.IOException: Login failure for hive/principal from keytab EvKuzmin.keytab
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:695)
at com.hive.connect.controller.RecordController.main(RecordController.java:22)
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
最后我把我的krb5.conf改成这样
[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = DOMAIN
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
[realms]
DOMAIN = {
admin_server = ms-dcs.DOMAIN
kdc = ms-dcs.DOMAIN
}
[domain_realm]
.DOMAIN = DOMAIN
DOMAIN = DOMAIN
还有我的java代码
public class RecordController {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException, ClassNotFoundException, IOException {
Class.forName(driverName);
System.setProperty("javax.security.auth.useSubjectCredsOnly","false");
System.setProperty("java.security.krb5.conf","krb5.conf");
Connection con = DriverManager
.getConnection("jdbc:hive2://host:port/arstel;" +
"principal=hive/host@DOMAIN;" +
"auth=kerberos;" +
"kerberosAuthType=fromSubject");
编辑
当我在集群上启动它时，我不需要输入密码和登录，但在 windows 的本地机器上我需要。
我可以访问启用了 kerberos 的远程 hdfs 集群。当我尝试使用直线连接时
beeline -u "jdbc:hive2://host:port/arstel;principal=principal"
连接成功。
但是当我编写一个 java 应用程序并使用相同的 JDBC Uri 时，我得到
ERROR transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
jdbc.HiveConnection: Could not open client transport with JDBC Uri: jdbc:hive2://host:port/arstel;principal=principal
Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://host:port/arstel;principal=principal: GSS initiate failed
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
我的应用程序
public class RecordController {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException, ClassNotFoundException {
Class.forName(driverName);
Connection con = DriverManager
.getConnection("jdbc:hive2://host:port/arstel;principal=principal");
Statement stmt = con.createStatement();
String tableName = "evkuzmin_testHiveDriverTable";
stmt.execute("drop table if exists " + tableName);
stmt.execute("create table " + tableName + " (key int, value string)");
}
}
为什么会出现这种情况，我该如何解决？
这个我也试过了。相同的结果。
getConnection("jdbc:hive2://host:port/arstel;principal=principal;" +
"auth=kerberos;" +
"kerberosAuthType=fromSubject");
编辑
我没有密钥表。在堆栈上阅读它和 here 后，我按照说明创建了一个。然后我以这种方式将属性添加到我的 java 文件中
Class.forName(driverName);
System.setProperty("java.security.auth.login.config","gss-jaas.conf");
System.setProperty("sun.security.jgss.debug","true");
System.setProperty("javax.security.auth.useSubjectCredsOnly","false");
System.setProperty("java.security.krb5.conf","krb5.conf");
我添加后resources
gss-jaas.conf
com.sun.security.jgss.initiate {
com.sun.security.auth.module.Krb5LoginModule=required
useKeyTab=true
useTicketCache=false
principal="hive/host@DOMAIN"
doNotPrompt=true
keyTab="EvKuzmin.keytab"
debug=true;
}
和krb5.conf
[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = DOMAIN
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
[realms]
DOMAIN = {
admin_server = ms-dcs.DOMAIN
kdc = ms-dcs.DOMAIN
}
[domain_realm]
.DOMAIN = DOMAIN
DOMAIN = DOMAIN
然后错误改了
java.sql.SQLException: Could not create secure connection to jdbc:hive2://host:port/arstel;principal=hive/principal;auth=kerberos;kerberosAuthType=fromSubject: Failed to open client transport
Caused by: javax.security.sasl.SaslException: Failed to open client transport [Caused by java.io.IOException: Could not instantiate SASL transport]
Caused by: java.io.IOException: Could not instantiate SASL transport
Caused by: javax.security.sasl.SaslException: Failure to initialize security context [Caused by GSSException: Invalid name provided (Mechanism level: Cannot locate default realm)]
Caused by: GSSException: Invalid name provided (Mechanism level: Cannot locate default realm)
还有
这里已经有一个密钥表
Keytab name: FILE:/etc/krb5.keytab
但我不知道它是什么领域或如何使用它 + 我无权访问该文件夹。
编辑 2 2017-08-09
尝试通过 UserGroupInformation 进行操作。出现错误
Exception in thread "main" java.lang.IllegalArgumentException: Invalid attribute value for hadoop.security.authentication of Kerberos
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:200)
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:227)
at com.hive.connect.controller.RecordController.main(RecordController.java:20)
EDIT3 2017-08-10 使用用户组
public class RecordController {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException, ClassNotFoundException, IOException {
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab("hive/principal", "EvKuzmin.keytab");
Class.forName("org.apache.hive.jdbc.HiveDriver");
Connection con = DriverManager
.getConnection("jdbc:hive2://host:port/arstel;" +
"principal=hive/principal;" +
"tez.queue.name=adhoc;" +
"hive.execution.engine=tez;" +
"mapreduce.job.reduces=1;");
错误
java.io.IOException: Login failure for hive/principal from keytab EvKuzmin.keytab
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:695)
at com.hive.connect.controller.RecordController.main(RecordController.java:22)
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
最后我把我的krb5.conf改成这样
[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = DOMAIN
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
[realms]
DOMAIN = {
admin_server = ms-dcs.DOMAIN
kdc = ms-dcs.DOMAIN
}
[domain_realm]
.DOMAIN = DOMAIN
DOMAIN = DOMAIN
还有我的java代码
public class RecordController {
private static String driverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException, ClassNotFoundException, IOException {
Class.forName(driverName);
System.setProperty("javax.security.auth.useSubjectCredsOnly","false");
System.setProperty("java.security.krb5.conf","krb5.conf");
Connection con = DriverManager
.getConnection("jdbc:hive2://host:port/arstel;" +
"principal=hive/host@DOMAIN;" +
"auth=kerberos;" +
"kerberosAuthType=fromSubject");
编辑
当我在集群上启动它时，我不需要输入密码和登录，但在 windows 的本地机器上我需要。