pig latin - 不同计数和分组依据

Question

我有一个数据集(userid, resH, resW)

数据如

(1001, 800, 600)
(1001, 800, 600)
(1002, 900, 700)
(1003, 900, 700)
(1004, 1800, 600)
(1005, 1800, 1600)

我想获取每组 resH-reshW 中不同用户的数量。

例如，上述数据的输出将是

800,  600, 1
900,  700, 2
1800, 600, 1
1800, 1600, 1

我试过

D = group data by (resH,resW);
E = foreach D { 
    unique = DISTINCT data.userId;
    generate group, COUNT(unique) as unique_cnt;
};

但我没有得到我所期待的。

Answer 1

加载数据然后区分它以删除重复项，然后按感兴趣的两列分组并计算用户 ID。

A = LOAD 'data.csv' USING PigStorage(',') AS (userid:int,resH:int,resW:int);
B = DISTINCT A;
C = GROUP B BY (resH,resW);
D = FOREACH C GENERATE FLATTEN(group) AS (resH,resW),COUNT(A.userid);
DUMP D;

Answer 2

要求的输出方式：-

Alias1 = LOAD 'input.txt' USING PigStorage(',') AS (userid:int,resH:int,resW:int);
Alias2 = DISTINCT Alias1;
Alias3 = GROUP Alias2 BY (resH,resW);
Alias4 = foreach Alias3 GENERATE flatten(group),flatten(COUNT(Alias2.userid)) ;
DUMP Alias4;
or 
STORE Alias4 INTO 'output.txt' USING PigStorage(',');

pig latin - 不同计数和分组依据

pig latin - count distinct and group by

apache-pig