A Distributed Parallel Computation For Weighted Tensor Approximation

This paper is aimed at implementing weighted tensor approximation(WTA)on Apache Spark,and achieving less processing time.Unlike traditional tensor approximation,this paper takes the validity of the input data into consideration during the data compression process.By giving different weights to valid and invalid data,we can eliminate the impact of invalid data and obtain better approximation results.However,because of the large amount of raw data,the compression process of WTA using single node not only takes a long time,but also poses a great challenge for hardware.In this paper,we investigate on how to parallelize the compression process for less processing time.We choose to implement WTA on Spark,whose computational performance is often faster than other distributed computing platforms,such as Apache Hadoop.The feasibility of WTA on Spark is achieved by transforming the original multilinear problem into a common linear one.The input tensor is also partitioned into small blocks to further reduce compression time.And we established a set of fast operation methods that are suitable for distribute computing on Spark.The experimental results show that WTA achieves better rendering effect than TA,and the results fully show that the multi-nodes distribute computing on Spark is much faster than single node.