Allow non-stringly typed table properties

[jonashaag]

Feature Request / Improvement

Currently table properties like write.parquet.compression-level need to be stringly typed in PyIceberg. Should we allow for other types as well?

[Fokko]

@jonashaag Can you give an example? Table properties are encoded as strings by default.

[jonashaag]

See the example in my post. The pydantic Schema doesn't allow for non strings currently

[Fokko]

But we convert it to an int when setting the level? I'm sorry if I'm missing a detail. The spec states that the properties are encoded as a string: https://iceberg.apache.org/spec/#table-metadata-and-snapshots Changing that should be raised on the mailing list.

[jonashaag]

Do you think we should automatically convert them to a string if safe? It is just really awkward to set eg the compression level as a string

[Fokko]

Yes, that's what we are doing here:

iceberg-python/pyiceberg/io/pyarrow.py

Lines 1766 to 1773 in 853a77c

	def _get_int(key: str, default: Optional[int] = None) -> Optional[int]:
	if value := table_properties.get(key):
	try:
	return int(value)
	except ValueError as e:
	raise ValueError(f"Could not parse table property {key} to an integer: {value}") from e
	else:
	return default

It is just really awkward to set eg the compression level as a string

It is a string map, so I don't think we have any other options.

[jonashaag]

convert them to a string

The code you are showing converts them from a string.

I meant the other end:

create_table(..., properties={"write.parquet.compression-level": 42})

[Fokko]

@jonashaag Ahh, thanks, it already felt like I was missing a piece. We should accept everything that can coerce into a string. I like to follow the idea of being liberal in what you accept and being strict when writing.

[Fokko]

@jonashaag is this something that you're interested in contributing? :)

[kevinjqliu]

@jonashaag this should work now!