I have a spark data frame which can have duplicate columns, with different row values, is it possible to coalesce those duplicate columns and get a dataframe without any duplicate columns
example :
|name |upload| name| upload1|
| null| null|alice| 101|
| null| null| bob| 231|
|alice| 100| null| null|
| bob| 23| null| null|
should become -
|name |upload| upload1|
| alice| null| 101|
| bob | null| 231|
|alice| 100| null|
| bob| 23| null|
See Question&Answers more detail:os