在 Pig 中操作数据结构

Question

之前我问过 Hive 或 Pig 中的。我能够在 SQL 中得到答案，并从那里找到了 Hive 的答案。我仍在寻找 Pig 的解决方案。

我想更改我的表格：

进入 myTable2：

我试过：

myTable2 = FOREACH myTable GENERATE item, year, 
'jan' AS month, jan AS value, 
'feb' AS month, feb AS value,  
'mar' AS month, mar AS value;

在 Hive 中或多或少是有效的，但是 Pig 给我：

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1108:
<line 2, column 35> Duplicate schema alias: month

Answer 1

我弄明白了，虽然我希望看到更简洁的版本：

JAN = FOREACH myTable GENERATE item, year, 'jan' AS month, jan AS value;
FEB = FOREACH myTable GENERATE item, year, 'feb' AS month, feb AS value;
MAR = FOREACH myTable GENERATE item, year, 'mar' AS month, mar AS value;
myTable2 = union JAN, FEB, MAR;

Answer 2

猪脚本:

data = LOAD '/pigsamples/sampledata'  USING PigStorage(',') 
       AS (item:CHARARRAY, year:INT, jan:DOUBLE, feb:DOUBLE, mar:DOUBLE);

--concatenating month name to its value so that they won't get separated when i perform a flatten on the tuple.
concat_data =  FOREACH data GENERATE item, year, CONCAT('jan:', (CHARARRAY)jan) AS jan, 
               CONCAT('feb:', (CHARARRAY)feb) AS feb, CONCAT('mar:', (CHARARRAY)mar) AS mar;

--convert the month (name,value) pairs to a bag and flatten them
flatten_values = FOREACH concat_data GENERATE item, year, FLATTEN (TOBAG (jan, feb, mar)) AS month_values;

--split the string based on the delimiter that we used above to concat
split_flatten_values = FOREACH flatten_values GENERATE item, year, FLATTEN (STRSPLIT (month_values, ':')) AS (month:CHARARRAY, value:CHARARRAY);

在 Pig 中操作数据结构

Manipulating a data structure in Pig

hadoop

hive

apache-pig