datasets of spark Row into string

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public class SparkSample {
public static void main(String[] args) {
SparkSession spark = SparkSession
.builder()
.appName("SparkSample")
.master("local[*]")
.getOrCreate();
//create df
List<String> myList = Arrays.asList("one", "two", "three", "four", "five");
Dataset<Row> df = spark.createDataset(myList, Encoders.STRING()).toDF();
df.show();
//using df.as
List<String> listOne = df.as(Encoders.STRING()).collectAsList();
System.out.println(listOne);
//using df.map
List<String> listTwo = df.map(row -> row.mkString(), Encoders.STRING()).collectAsList();
System.out.println(listTwo);
}
}

Service ‘sparkDriver’ could not bind on port 0. 解决方案

hostname命令看下hostname 添加到 /etc/hosts https://stackoverflow.com/questions/34601554/mac-spark-shell-error-initializing-sparkcontext

未完待续…

Ref