在 Atlas 实体之间添加关系的简单示例？

Question

使用 REST API 在 apache atlas 中添加实体之间的关系的正确方法是什么？查看 REST API 的 docs，我发现很难说出某些字段的含义，哪些是必需的，哪些不是（如果不输入会发生什么），或者默认值是什么应该是（因为示例使用的似乎是占位符值（例如，什么是 provenanceType 或 propagateTags 字段似乎期望某种枚举值，但从未指定有效选项））。

有人可以提供任何示例来说明这在真实/有效值中的样子吗？例如。如果已经将 2 个实体 E1 和 E2 添加到 Atlas 中并希望在两者之间建立关系，则需要做类似...

curl -X POST --header 'Content-Type: application/json;charset=UTF-8' --header 'Accept: application/json' -d '{<simplified json>}' 'https://atlas-server-hostname:21000/v2/relationship'

正在尝试

[hph_etl@HW03 ~]$ curl -vv -u admin:admin -X POST --header 'Content-Type: application/json;charset=UTF-8' --header 'Accept: application/json' -d '{ \
   "createTime": 1565135406, \
   "createdBy": "hph_etl", \
   "end1": { \
     "guid": "2ddcda5b-2489-4636-a9ab-12b199c02422", \
     "typeName": "hdfs_path" \
   }, \
   "end2": { \
     "guid": "a33f45de-13d0-4a30-9df7-b0e02eb0dfd5", \
     "typeName": "hdfs_path" \
   }, \
   "guid": "2ddcda5b-2489-4636-a9ab-12b199c02422", \
   "propagateTags": "TWO_TO_ONE", \
   "status": "ACTIVE", \
   "typeName": "hdfs_path" \
 }' 'http://HW03.co.local:21000/api/atlas/v2/relationship'


* About to connect() to HW03.co.local port 21000 (#0)
*   Trying 172.18.4.48...
* Connected to HW03.co.local (172.18.4.48) port 21000 (#0)
* Server auth using Basic with user 'admin'
> POST /v2/relationship HTTP/1.1
> Authorization: Basic xxxxxx
> User-Agent: curl/7.29.0
> Host: HW03.co.local:21000
> Content-Type: application/json;charset=UTF-8
> Accept: application/json
> Content-Length: 442
>
* upload completely sent off: 442 out of 442 bytes
< HTTP/1.1 404 Not Found
< Date: Wed, 07 Aug 2019 01:07:44 GMT
< Set-Cookie: ATLASSESSIONID=xxxxxx;Path=/;HttpOnly
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Content-Type: text/html;charset=utf-8
< Content-Length: 2265
< Server: Jetty(9.3.14.v20161028)
<
<!doctype html>
<!--
....
*
*     http://www.apache.org/licenses/LICENSE-2.0
....
-->
<!--[if gt IE 8]>
<script type="text/javascript">
function Redirect() {
window.location.assign("login.jsp");
}
Redirect();
</script>
<![endif]-->
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if gt IE 7]>
<script src="js/external_lib/es5-shim.min.js"></script>
<script src="js/external_lib/respond.min.js"></script>
<![endif]-->
<html lang="en">

<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Atlas</title>
    ....
</head>

<body>
<div id="wrapper">
    <!-- Page Content Begin -->
    <div id="page-content-wrapper">
        <div class="page-title clearfix">
            <h3>Looking for something?</h3>
            <p>We're sorry. The web address you're looking for is not a functioning page in Apache Atlas. Please try navigating from <a href="index.html">Apache Atlas Home</a></p>
        </div>
    </div>
    <!-- Page Content End -->
</div>
</body>

</html>
* Connection #0 to host HW03.co.local left intact

不起作用，并且不确定如何处理错误消息。即使是

的简单示例

curl -vv -u admin:admin -X POST --header 'Content-Type: application/json;charset=UTF-8' --header 'Accept: application/json' -d '{ \
   "end1": { \
     "guid": "2ddcda5b-2489-4636-a9ab-12b199c02422" \
   }, \
   "end2": { \
     "guid": "a33f45de-13d0-4a30-9df7-b0e02eb0dfd5" \
   }, \
   "typeName": "AtlasRelationshipDef" \
 }' 'http://HW03.ucera.local:21000/api/atlas/v2/relationship'

抛出一个类似的无信息错误

* upload completely sent off: 211 out of 211 bytes
< HTTP/1.1 500 Internal Server Error
< Date: Mon, 12 Aug 2019 19:57:44 GMT
< Set-Cookie: ATLASSESSIONID=xxxxxx;Path=/;HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Server: Jetty(9.3.14.v20161028)
<
* Connection #0 to host HW03.co.local left intact
There was an error processing your request. It has been logged (ID 6d64bc3a1a910e46)

检查 atlas 主机服务器 /var/logs/atlas/application.log 中的日志，我可以看到另一个无意义的错误消息...

[hph_etl@HW03 atlas]$ cat application.log | grep -C 2 6d64bc3a1a910e46
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2019-08-12 09:57:44,880 ERROR - [pool-2-thread-10 - 8a5535b1-6544-4f9b-b3ad-8bec5e8d6fcd:] ~ Error handling a request: 6d64bc3a1a910e46 (ExceptionMapperUtil:32)
javax.ws.rs.WebApplicationException
        at com.sun.jersey.server.impl.uri.rules.TerminatingRule.accept(TerminatingRule.java:66)

请注意，为了获得关系链接所需的 guid，甚至 Hortonworks 似乎也只提供 poor solution。

这里可能出了什么问题？有没有比链接到的文档更好的文档来理解 API?

Answer 1

可以使用实体 GUID 或 end1 和 end2 中的 uniqueAttributes 来创建现有实体之间的 Atlas 关系，它可以是 qualifiedName 或任何其他唯一属性。

请注意，顶级 typeName 是关系 def typeName，而 end1 和 end2 中的 typeName 是实体 typeName。

如果 hive_table 和 hive_db 之间的关系 def typeName 是：hive_table_db

因此，如果您想在 hive_table 和 hive_db 之间创建关系，请求将是：

 POST: /api/atlas/v2/relationship
        {
        "typeName": "hive_table_db",
        "end1": {
            "typeName": "hive_table",
            "uniqueAttributes": {
                "qualifiedName": "db.table@cluster"
            }
        },
        "end2": {
            "typeName": "hive_db",
            "uniqueAttributes": {
                "qualifiedName": "db@cluster"
            }
        }
    }

对于预定义的 Atlas 类型，您可以从其在 relationshipAttributeDefs 字段

中的定义中找到关系 typeName

GET: /api/atlas/v2/types/typedef/name/hive_db

给出以下响应：

{
    "category": "ENTITY",
    "guid": "78c44290-2ed8-461b-953d-3965d9bb44ca",
    "createdBy": "root",
    "updatedBy": "root",
    "createTime": 1548175553859,
    "updateTime": 1548175822249,
    "version": 2,
    "name": "hive_db",
    "description": "hive_db",
    "typeVersion": "1.2",
    "serviceType": "hive",
    "attributeDefs": [
        {
            "name": "clusterName",
            "typeName": "string",
            "isOptional": false,
            "cardinality": "SINGLE",
            "valuesMinCount": 1,
            "valuesMaxCount": 1,
            "isUnique": false,
            "isIndexable": true,
            "includeInNotification": true,
            "searchWeight": -1
        },
        {
            "name": "location",
            "typeName": "string",
            "isOptional": true,
            "cardinality": "SINGLE",
            "valuesMinCount": 0,
            "valuesMaxCount": 1,
            "isUnique": false,
            "isIndexable": false,
            "includeInNotification": false,
            "searchWeight": -1
        },
        {
            "name": "parameters",
            "typeName": "map<string,string>",
            "isOptional": true,
            "cardinality": "SINGLE",
            "valuesMinCount": 0,
            "valuesMaxCount": 1,
            "isUnique": false,
            "isIndexable": false,
            "includeInNotification": false,
            "searchWeight": -1
        },
        {
            "name": "ownerType",
            "typeName": "hive_principal_type",
            "isOptional": true,
            "cardinality": "SINGLE",
            "valuesMinCount": 0,
            "valuesMaxCount": 1,
            "isUnique": false,
            "isIndexable": false,
            "includeInNotification": false,
            "searchWeight": -1
        }
    ],
    "superTypes": [
        "Asset"
    ],
    "subTypes": [],
    "relationshipAttributeDefs": [
        {
            "name": "tables",
            "typeName": "array<hive_table>",
            "isOptional": true,
            "cardinality": "SET",
            "valuesMinCount": -1,
            "valuesMaxCount": -1,
            "isUnique": false,
            "isIndexable": false,
            "includeInNotification": false,
            "searchWeight": -1,
            "constraints": [
                {
                    "type": "ownedRef"
                }
            ],
            "relationshipTypeName": "hive_table_db",
            "isLegacyAttribute": false
        },
        {
            "name": "ddlQueries",
            "typeName": "array<hive_db_ddl>",
            "isOptional": true,
            "cardinality": "SET",
            "valuesMinCount": -1,
            "valuesMaxCount": -1,
            "isUnique": false,
            "isIndexable": false,
            "includeInNotification": false,
            "searchWeight": -1,
            "constraints": [
                {
                    "type": "ownedRef"
                }
            ],
            "relationshipTypeName": "hive_db_ddl_queries",
            "isLegacyAttribute": false
        },
        {
            "name": "meanings",
            "typeName": "array<AtlasGlossaryTerm>",
            "isOptional": true,
            "cardinality": "SET",
            "valuesMinCount": -1,
            "valuesMaxCount": -1,
            "isUnique": false,
            "isIndexable": false,
            "includeInNotification": false,
            "searchWeight": -1,
            "relationshipTypeName": "AtlasGlossarySemanticAssignment",
            "isLegacyAttribute": false
        }
    ]
}

此外，您需要确保end1和end2中的typeName符合关系def，您可以在类型定义中查看：

GET: /api/atlas/v2/types/typedef/name/hive_table_db
    {
        "category": "RELATIONSHIP",
        "guid": "79257a2c-407c-4c0b-b3ae-04b1b3a8d649",
        "createdBy": "root",
        "updatedBy": "root",
        "createTime": 1548175553894,
        "updateTime": 1548175553894,
        "version": 1,
        "name": "hive_table_db",
        "description": "hive_table_db",
        "typeVersion": "1.0",
        "serviceType": "hive",
        "attributeDefs": [],
        "relationshipCategory": "COMPOSITION",
        "propagateTags": "NONE",
        "endDef1": {
            "type": "hive_table",
            "name": "db",
            "isContainer": false,
            "cardinality": "SINGLE",
            "isLegacyAttribute": true
        },
        "endDef2": {
            "type": "hive_db",
            "name": "tables",
            "isContainer": true,
            "cardinality": "SET",
            "isLegacyAttribute": false
 }
}

Answer 2

curl -i -X POST  -H 'Content-Type: application/json' -H 'Accept: application/json' -u admin:admin 'http://localhost:21000/api/atlas/v2/relationship' -d '
        {
        "typeName": "???",
        "end1": {"guid" : ""},
        "end2": {"guid" : ""}
    }
'



1. How to fullfil <???>
 go to http://localhist:21000/api/atlas/v2/types/typedef/name/<YOUR ENTITY> 
to find relationshipAttributeDefs for this ENTITY

You need "relationshipTypeName" where  "rdbms_foreign_key_column_references"="???"

2. than exec

curl -i -X POST  -H 'Content-Type: application/json' -H 'Accept: application/json' -u admin:admin 'http://localhost:21000/api/atlas/v2/relationship' -d '
        {
        "typeName": "rdbms_foreign_key_column_references",
        "end1": {"guid" : ""},
        "end2": {"guid" : ""}
    }
'
you will have error
"errorMessage":"Relationship end is invalid. Expected rdbms_foreign_key but is NULL"

where you will know the needs ENTITY

在 Atlas 实体之间添加关系的简单示例？

Simple example for adding relationships between Atlas entities?

apache-atlas

hdp