I have a Dataset with the following columns:
Id, seed
my dataset has 10M per 100k rows and every 100k rows the seed change. I would like to add a column "random" as follows:
data.withColumn("random",rand(col("seed")))
the problem is that function org.apache.spark.sql.functions.rand accept only a value not a column. Is there a way to generate randoms with sparksql and a column as seed?
Thanks
Aucun commentaire:
Enregistrer un commentaire