格式转换以dataframe1通过分割它
+-----+---------------+------------------------------------------------------------------------------------------------------------------+
|table| err_timestamp| err_message |
+-----+---------------+------------------------------------------------------------------------------------------------------------------+
| t1|7/26/2017 13:56|[error = RI_VIOLATION, field = user_id, value = 'null'] |
| t2|7/26/2017 13:58|[error = NULL_CHECK, field = geo_id, value = 'null'] [error = DATATYPE_CHECK, field = emp_id, value = 'FIWOERE8'] |
+-----+---------------+------------------------------------------------------------------------------------------------------------------+
到输出dataframe2作为整个列的转置和列如下所示,创建从现有列新列。
+-----+--------------+---------+--------------+-----------+
|table| err_date|err_field| err_type| err_value|
+-----+--------------+---------+--------------+-----------+
| t1|7/26/2017 0:00| user_id| RI_VOILATION| null|
| t2|7/26/2017 0:00| geo_id| NULL_CHECK| null|
| t2|7/26/2017 0:00| emp_id|DATATYPE_CHECK|FDSADFSDA68|
+-----+--------------+---------+--------------+-----------+
认真配合?你试图问什么? – ayush
嗨,我想转换dataframe1到dataframe2不使用火花SQL,请帮我 – prakash
提供它是非常重要的莫我 – prakash