Skip to content Skip to sidebar Skip to footer
Showing posts with the label Scala
How To Use Scala Udf In Pyspark?

How To Use Scala Udf In Pyspark?

Post a Comment
I want to be able to use a Scala function as a UDF in PySpark package com.test object ScalaPySpark… Read more How To Use Scala Udf In Pyspark?
How To Split A Text File Into Multiple Columns With Spark

How To Split A Text File Into Multiple Columns With Spark

Post a Comment
I'm having difficulty on splitting a text data file with delimiter '|' into data frame … Read more How To Split A Text File Into Multiple Columns With Spark
Why Is This Simple Spark Program Not Utlizing Multiple Cores?

Why Is This Simple Spark Program Not Utlizing Multiple Cores?

Post a Comment
So, I'm running this simple program on a 16 core multicore system. I run it by issuing the foll… Read more Why Is This Simple Spark Program Not Utlizing Multiple Cores?