Spark Unique Pair In Cartesian Product

I have this: In [1]:a = sc.parallelize([a,b,c]) In [2]:a.cartesian(a).collect() Out[3]: [(a, a), (a, b), (a, c), (b, a), (c, a), (b, b), (b, c), (c, b), (c, c)] I want the followi

Solution 1:

Not sure about the python syntax, but in scala you could write:

a.cartesian(a).filter{ case (a,b) => a <= b }.collect()

My guess is in python it would be something like:

a.cartesian(a).filter(lambda a, b: a <= b).collect()