Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
Welcome To Ask or Share your Answers For Others

Categories

I have the following schema:

root
  |-- id: string (nullable = true)
  |-- text: string (nullable = true)  
  |-- user: string (nullable = true)

How can I create StructType schema from this String?

I know that I can use .schema() method in my Dataset but I asking If its possible to create Schema from string.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
355 views
Welcome To Ask or Share your Answers For Others

1 Answer

I was need something like that and i wrote a method, this able to creating the schema from string. But you must change this method for your scenario.

val schemaString = "val1#Int val2#String val3#Int"

val schema = StructType(schemaString.split(" ").map(fieldNameTypeStr => {
  val fieldNameType = fieldNameTypeStr.split("#")
  val dataType: DataType = fieldNameType(1) match {
    case "String" => StringType
    case "Int" => IntegerType
      /* other cases */
  }
  StructField(fieldNameType(0), dataType, true)
}))

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
...