TAJO-2164: SequenceFile print wrong values with TextSerializerDeserializer#1033
TAJO-2164: SequenceFile print wrong values with TextSerializerDeserializer#1033blrunner wants to merge 7 commits intoapache:masterfrom
Conversation
| if (delim == null || delim.isEmpty()) { | ||
| delim = meta.getProperty(StorageConstants.TEXT_DELIMITER, StorageConstants.DEFAULT_FIELD_DELIMITER); | ||
| } | ||
| meta.getPropertySet().set(StorageConstants.TEXT_DELIMITER, delim); |
There was a problem hiding this comment.
The property order should be TEXT_DELIMITER, SEQUENCEFILE_DELIMITER and please use default value of getProperty.
Why you set the property ? is there any reason ?
There was a problem hiding this comment.
If users only set SEQUENCEFILE_DELIMITER, TextLineSerDe can't delimit records because TextLineSerDe use TEXT_DELIMITER as the delimiter. I modified codes again as follows. The value of SEQUENCEFILE_DELIMITER will only use if TEXT_DELIMITER doesn't exit and SEQUENCEFILE_DELIMITER exists only.
There was a problem hiding this comment.
I think you can change to simplify code like this
TableMeta tableMeta = meta.clone();
if (!tableMeta.containsProperty(StorageConstants.TEXT_DELIMITER)) {
tableMeta.putProperty(StorageConstants.TEXT_DELIMITER, tableMeta.getProperty(StorageConstants.SEQUENCEFILE_DELIMITER);
}
also you should consider the null char and you should remove useless code
…CEFILE_DELIMITER exists.
| keyValueSet.set(StorageConstants.TEXT_DELIMITER, meta.getProperty(StorageConstants.SEQUENCEFILE_DELIMITER)); | ||
| } | ||
|
|
||
| String delim = meta.getProperty(StorageConstants.TEXT_DELIMITER, StorageConstants.DEFAULT_FIELD_DELIMITER); |
|
Thanks @jinossy , |
|
+1 LGTM |
|
Jaehwa looks busy, so I'll commit this patch soon. |
|
Thanks guys, I'll ship it tonight. |
…into TAJO-2164
3 participants
See https://issues.apache.org/jira/browse/TAJO-2164