Adding a new dev dep, deptry

[kevinjqliu]

This PR adds a new dev dependency named deptry, which is a "command line tool to check for issues with dependencies in a Python project".

poetry install
deptry .

Furthermore, this PR changes dev depdencies group from tool.poetry.dev-dependencies (legacy/pre-1.2.x) to tool.poetry.group.dev.dependencies (current).
See the, "A note about defining a dev dependencies group" section of poetry doc and python-poetry/poetry#6638

Deptry

Using deptry found a few dependency issues, which we can address in a seperate PR

deptry .

(venv) ➜  iceberg-python git:(kevinjqliu/dev-dep) deptry .
Assuming the corresponding module name of package 'pyarrow' is 'pyarrow'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'pandas' is 'pandas'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'duckdb' is 'duckdb'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'ray' is 'ray'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'python-snappy' is 'python_snappy'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'thrift' is 'thrift'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'mypy-boto3-glue' is 'mypy_boto3_glue'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 's3fs' is 's3fs'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'adlfs' is 'adlfs'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'gcsfs' is 'gcsfs'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'psycopg2-binary' is 'psycopg2_binary'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'sqlalchemy' is 'sqlalchemy'. Install the package or configure a package_module_name_map entry to override this behaviour.
Assuming the corresponding module name of package 'getdaft' is 'getdaft'. Install the package or configure a package_module_name_map entry to override this behaviour.
Scanning 69 files...

build-module.py:26:12: DEP004 'Cython' imported but declared as a dev dependency
build-module.py:27:5: DEP004 'Cython' imported but declared as a dev dependency
build-module.py:28:5: DEP003 'setuptools' imported but it is a transitive dependency
build-module.py:29:5: DEP003 'setuptools' imported but it is a transitive dependency
dev/provision.py:18:1: DEP004 'pyspark' imported but declared as a dev dependency
dev/provision.py:19:1: DEP004 'pyspark' imported but declared as a dev dependency
mkdocs/gen_doc_stubs.py:20:8: DEP001 'griffe' imported but missing from the dependency definitions
mkdocs/gen_doc_stubs.py:21:8: DEP001 'mkdocs_gen_files' imported but missing from the dependency definitions
pyiceberg/avro/codecs/snappy_codec.py:27:12: DEP001 'snappy' imported but missing from the dependency definitions
pyiceberg/catalog/hive.py:33:1: DEP001 'hive_metastore' imported but missing from the dependency definitions
pyiceberg/catalog/hive.py:34:1: DEP001 'hive_metastore' imported but missing from the dependency definitions
pyiceberg/catalog/hive.py:50:1: DEP001 'hive_metastore' imported but missing from the dependency definitions
pyiceberg/catalog/hive.py:51:1: DEP001 'hive_metastore' imported but missing from the dependency definitions
pyiceberg/catalog/rest.py:418:9: DEP003 'botocore' imported but it is a transitive dependency
pyiceberg/catalog/rest.py:419:9: DEP003 'botocore' imported but it is a transitive dependency
pyiceberg/io/fsspec.py:33:1: DEP003 'botocore' imported but it is a transitive dependency
pyiceberg/io/fsspec.py:34:1: DEP003 'botocore' imported but it is a transitive dependency
pyiceberg/io/pyarrow.py:57:8: DEP001 'numpy' imported but missing from the dependency definitions
pyiceberg/partitioning.py:41:1: DEP004 'typing_extensions' imported but declared as a dev dependency
pyiceberg/table/__init__.py:48:1: DEP004 'typing_extensions' imported but declared as a dev dependency
pyiceberg/table/__init__.py:134:12: DEP001 'daft' imported but missing from the dependency definitions
pyiceberg/table/__init__.py:1217:16: DEP001 'daft' imported but missing from the dependency definitions
pyiceberg/table/metadata.py:33:1: DEP004 'typing_extensions' imported but declared as a dev dependency
pyiceberg/table/refs.py:21:1: DEP004 'typing_extensions' imported but declared as a dev dependency
pyiceberg/table/sorting.py:35:1: DEP004 'typing_extensions' imported but declared as a dev dependency
pyiceberg/types.py:53:1: DEP003 'pydantic_core' imported but it is a transitive dependency
pyproject.toml: DEP002 'python-snappy' defined as a dependency but not used in the codebase
pyproject.toml: DEP002 'psycopg2-binary' defined as a dependency but not used in the codebase
pyproject.toml: DEP002 'getdaft' defined as a dependency but not used in the codebase
vendor/hive_metastore/ThriftHiveMetastore.py:28:8: DEP001 'fb303' imported but missing from the dependency definitions
Found 30 dependency issues.

For more information, see the documentation: https://fpgmaas.github.io/deptry/

[Fokko]

While there are some false positives, I do like this. At some point we should also add this to the CI 👍