Use It or Lose It Notes
Posted: | Categories: living, tech | Tags: java, snippets, spark, use-it-or-lose-it
Because I mostly don’t use it, and then end up losing it. This is my living blog of quick, forgotten patterns. Not profound, just practical. Table of Contents Spark Java Spark 1. Create a SparkSession (Boilerplate I always forget) SparkSession spark = SparkSession.builder() .appName("UILI") .master("local[*]") .getOrCreate(); 2. Create a Dataset from Strings (not from files) Dataset<String> ds = spark.createDataset( Arrays.asList("Abc", "xyz"), Encoders.STRING() ); 3. SparkConf and RDD creation options SparkConf conf = new SparkConf(). Read more...