-
Notifications
You must be signed in to change notification settings - Fork 415
Closed
Milestone
Description
Apache Iceberg version
0.6.0 (latest release)
Please describe the bug 🐞
This bug was reported by Jorge Arada on slack: https://apache-iceberg.slack.com/archives/C029EE6HQ5D/p1712309474600169
Reported example to reproduce:
catalog = load_hive("iceberg_catalog", {"uri": "some_url"})
fields = [NestedField( field_id=0, name="teste_tz", field_type=TimestamptzType(), required=False)]
schema = Schema(*fields)
catalog.create_table(identifier="raw.test_table", schema=schema)Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/revolut/app/.venv/lib/python3.11/site-packages/pyiceberg/catalog/hive.py", line 315, in create_table
sd=_construct_hive_storage_descriptor(schema, location),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/revolut/app/.venv/lib/python3.11/site-packages/pyiceberg/catalog/hive.py", line 155, in _construct_hive_storage_descriptor
[FieldSchema(field.name, visit(field.field_type, SchemaToHiveConverter()), field.doc) for field in schema.fields],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/revolut/app/.venv/lib/python3.11/site-packages/pyiceberg/catalog/hive.py", line 155, in <listcomp>
[FieldSchema(field.name, visit(field.field_type, SchemaToHiveConverter()), field.doc) for field in schema.fields],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/functools.py", line 909, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/revolut/app/.venv/lib/python3.11/site-packages/pyiceberg/schema.py", line 847, in _
return visitor.primitive(obj)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/revolut/app/.venv/lib/python3.11/site-packages/pyiceberg/catalog/hive.py", line 228, in primitive
return HIVE_PRIMITIVE_TYPES[type(primitive)]
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: <class 'pyiceberg.types.TimestamptzType'>
It seems we miss the TimestampzTypein the Iceberg to Hive type conversion map.
A similar issue has been observed and fixed in glue: #366
It would be good to include the fix in 0.6.1
FokkoFokko
Metadata
Metadata
Assignees
Labels
No labels