Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
Welcome To Ask or Share your Answers For Others

Categories

I have a dataset with so many columns and I want to cast all columns to the string using Java.

I tried below steps, I want to know if there is any better way to achieve this?

Dataset<Row> ds = ...;
JavaRDD<String[]> stringArrRDD = ds.javaRDD().map(row->{
          int length = row.length();
          String[] columns = new String[length];
          for(int i=0; i<length;i++){
              columns[i] = row.get(i) !=null? row.get(i).toString():"";
          }
       return  columns;});
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
178 views
Welcome To Ask or Share your Answers For Others

1 Answer

You can iterate over columns:

for (String c: ds.columns()) {
    ds = ds.withColumn(c, ds.col(c).cast("string"));
}

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
...