lacucinadiadine

I'm currently working with Spark 2.1 and have a main script that calls a helper module that con… Read more Pyspark 2.1: Importing Module With Udf's Breaks Hive Connectivity

I'm running spark via pycharm and respectively pyspark shell. I've stacked with this error:… Read more Pyspark Application Fail With Java.lang.outofmemoryerror: Java Heap Space

I have a data frame in pyspark. In this data frame I have column called id that is unique. Now I wa… Read more How To Find Maximum Value Of A Column In Python Dataframe

I'm using spark version 2.0.1 & python 2.7. I'm running following code # This will retu… Read more Typeerror: 'groupeddata' Object Is Not Iterable In Pyspark

I am trying to solve the following error (I am using the databricks platform and spark 2.0) tweets_… Read more Pyspark Error With Udf: Py4j.py4jexception: Method __getnewargs__([]) Does Not Exist Error

I would like to transform the values of a column into multiple columns of a dataframe in pyspark on… Read more Transform Columns Values To Columns In Pyspark Dataframe

Hy, I'm trying build a recommendation system with Spark I have a data frame with users email an… Read more Pyspark Add New Column Field With The Data Frame Row Number

I have a need to fetch list of holidays in a given range, i.e., if start date is 20/12/2016 & e… Read more Python List Of Usa Holidays Between A Range