如何为所有用户推荐 Spark ALS 中排名前 10 的产品？

Question

我们如何在 PySpark 中获得前 10 个推荐产品。我知道有 recommendProducts 之类的方法可以为单个用户推荐产品，而 predictAll 可以预测 {user,item} 对的评级。但是有没有一种有效的方法可以为所有用户输出每个用户的前 10 个项目？

Answer 1

我编写了这个函数，它将用户特征和产品特征乘以分区，以便它得到分布，然后它获得用户对每个产品的评分，并按评分对它们进行排序，并输出 8 个最热门推荐产品的列表。

#Collect product feature matrix
 productFeatures = bestModel.productFeatures().collect() 
 productArray=[]
 productFeaturesArray=[]
 for x in productFeatures:
    productArray.append(x[0])
    productFeaturesArray.append(x[1])  
 matrix=np.matrix(productFeaturesArray)
 productArrayBroadCast=sc.broadcast(productArray)
 productFeaturesArraybroadcast=sc.broadcast(matrix.T)

 def func(iterator):
      userFeaturesArray = []
      userArray = []
      for x in iterator:
          userArray.append(x[0])
          userFeaturesArray.append(x[1])
          userFeatureMatrix = np.matrix(userFeaturesArray)
          userRecommendationArray = userFeatureMatrix*(productFeaturesArraybroadcast.value)
          mappedUserRecommendationArray = []
          #Extract ratings from the matrix
          i=0
          for i in range(0,len(userArray)):
              ratingdict={}
              j=0
              for j in range(0,len(productArrayBroadcast.value)):
                   ratingdict[str(productArrayBroadcast.value[j])]=userRecommendationArray.item((i,j))
                   j=j+1
              #Take the top 8 recommendations for the user
              sort_apps=sorted(ratingdict.keys(), key=lambda x: x[1])[:8]
              sort_apps='|'.join(sort_apps)
              mappedUserRecommendationArray.append((userArray[i],sort_apps))
              i=i+1
      return [x for x in mappedUserRecommendationArray]


recommendations=model.userFeatures().repartition(2000).mapPartitions(func)

Answer 2

PySpark >= 1.6.0 提供 MatrixFactorizationModel.recommendProductsForUsers:

>> model.recommendProductsForUsers(3).take(2)
[(1,
  (Rating(user=1, product=2975, rating=0.003626774788608227),
   Rating(user=1, product=1322, rating=0.002494393082165332),
   Rating(user=1, product=8746, rating=0.002176665942528324))),
 (2,
  (Rating(user=2, product=4060, rating=0.011020947406565042),
   Rating(user=2, product=2332, rating=0.009479319983658458),
   Rating(user=2, product=1979, rating=0.004587168057824856)))]

如何为所有用户推荐 Spark ALS 中排名前 10 的产品？

How to recommend top 10 products in Spark ALS for all the users?

apache-spark

pyspark