Dataproc 尝试通过 JDBC 连接到 Postgres，缺少权限

Question

提前致谢...

我想 connect/write 使用 JDBC API 以编程方式使用 Cloud SQL 到 Postgres SQL 实例运行。我使用了以下罐子：
1. postgresql
2. postgres 套接字工厂
3. postgres-socket-factory-1.0.11-jar-with-dependencies.jar

我是运行 Dataproc，它将尝试使用步骤 #1 进行连接，但出现以下异常：

2019-04-01 11:05:03.998 IST
Something unusual has occurred to cause the driver to fail. Please report this exception.
    at org.postgresql.Driver.connect(Driver.java:277)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:270)
    at rdsConnector$.getConnection(rdsConnector.scala:33) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.yarn.ApplicationMaster$$anon.run(ApplicationMaster.scala:721)
Caused by: java.lang.RuntimeException: Unable to retrieve information about Cloud SQL instance [projectID:us-east1:dB]
    at com.google.cloud.sql.core.SslSocketFactory.obtainInstanceMetadata(SslSocketFactory.java:459)
    at com.google.cloud.sql.core.SslSocketFactory.fetchInstanceSslInfo(SslSocketFactory.java:333)
    at com.google.cloud.sql.core.SslSocketFactory.getInstanceSslInfo(SslSocketFactory.java:313)
    at com.google.cloud.sql.core.SslSocketFactory.createAndConfigureSocket(SslSocketFactory.java:194)
    at com.google.cloud.sql.core.SslSocketFactory.create(SslSocketFactory.java:160)
    at com.google.cloud.sql.postgres.SocketFactory.createSocket(SocketFactory.java:96)
    at org.postgresql.core.PGStream.<init>(PGStream.java:62)
    at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
    at org.postgresql.Driver.makeConnection(Driver.java:454)
    at org.postgresql.Driver.connect(Driver.java:256)
    ... 11 more
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden

2019-04-01 11:05:03.000 IST
User class threw exception: org.postgresql.util.PSQLException: Something unusual has occurred to cause the driver to fail. Please report this exception.

我将此理解为权限问题，但由于我使用 Dataproc 连接到 Postgres，缺少什么权限？如果我是来自本地笔记本电脑的运行，那么我必须将 GOOGLE_APPLICATION_CREDENTIALS 设置为 json 文件。但如果是 Dataproc，流程是什么？

Answer 1

JDBC SocketFactory 使用 Application Default Credentials 策略来访问帐户凭据。

对于 Cloud Dataproc，在 [project-number]-compute@developer.gserviceaccount.com 处为您提供了一个默认服务帐户。您授予此帐户 "Cloud SQL Client" IAM 角色，JDBC SocketFactory 将使用它进行身份验证，因此您将能够连接到您的应用程序。

Answer 2

为以后的访问者留下进一步的注释：

如果您遇到的错误如下：

{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "Insufficient Permission",
    "reason" : "insufficientPermissions"
  } ],
  "message" : "Request had insufficient authentication scopes.",
  "status" : "PERMISSION_DENIED"
}

那么这可能是由 VM 实例的范围（云 API 访问范围）问题引起的。解决此问题的一种方法是在创建 Dataproc 集群时添加 sql-admin 的范围。例如：

gcloud dataproc clusters create <your-cluster-name> \
    --region=<your-region> \
    --zone=<your-zone> \
    --scopes=https://www.googleapis.com/auth/sqlservice.admin \
    ...

另一种方式是在 VM 实例中编辑。

详情见gcloud documentation

Dataproc 尝试通过 JDBC 连接到 Postgres，缺少权限

Dataproc trying to connect to Postgres through JDBC, missing permissions

postgresql

jdbc

google-cloud-sql

google-cloud-platform

google-cloud-dataproc