metadata.iceberg.compaction.max.file-num |
50 |
Integer |
If number of small Iceberg manifest metadata files exceeds this limit, always trigger manifest metadata compaction regardless of their total size. |
metadata.iceberg.compaction.min.file-num |
10 |
Integer |
Minimum number of Iceberg manifest metadata files to trigger manifest metadata compaction. |
metadata.iceberg.database |
(none) |
String |
Metastore database name for Iceberg Catalog. Set this as an iceberg database alias if using a centralized Catalog. Multiple databases can be specified with semicolons, e.g. 'db1;db2'. The table will be registered in each database. |
metadata.iceberg.delete-after-commit.enabled |
true |
Boolean |
Whether to delete old metadata files after each table commit |
metadata.iceberg.format-version |
2 |
Integer |
The format version of iceberg table, the value can be 2 or 3. Note that only version 3 supports deletion vector. |
metadata.iceberg.glue.skip-archive |
false |
Boolean |
Skip archive for AWS Glue catalog. |
metadata.iceberg.hadoop-conf-dir |
(none) |
String |
hadoop-conf-dir for Iceberg Hive catalog. |
metadata.iceberg.hive-client-class |
"org.apache.hadoop.hive.metastore.HiveMetaStoreClient" |
String |
Hive client class name for Iceberg Hive Catalog. |
metadata.iceberg.hive-conf-dir |
(none) |
String |
hive-conf-dir for Iceberg Hive catalog. |
metadata.iceberg.hive-skip-update-stats |
false |
Boolean |
Skip updating Hive stats. |
metadata.iceberg.manifest-compression |
"snappy" |
String |
Compression for Iceberg manifest files. |
metadata.iceberg.manifest-legacy-version |
false |
Boolean |
Should use the legacy manifest version to generate Iceberg's 1.4 manifest files. |
metadata.iceberg.previous-versions-max |
0 |
Integer |
The number of old metadata files to keep after each table commit. For rest-catalog, it will keep 1 old metadata at least. |
metadata.iceberg.storage |
disabled |
Enum |
When set, produce Iceberg metadata after a snapshot is committed, so that Iceberg readers can read Paimon's raw data files.
Possible values:- "disabled": Disable Iceberg compatibility support.
- "table-location": Store Iceberg metadata in each table's directory.
- "hadoop-catalog": Store Iceberg metadata in a separate directory. This directory can be specified as the warehouse directory of an Iceberg Hadoop catalog.
- "hive-catalog": Not only store Iceberg metadata like hadoop-catalog, but also create Iceberg external table in Hive.
- "rest-catalog": Store Iceberg metadata in a REST catalog. This allows integration with Iceberg REST catalog services.
|
metadata.iceberg.storage-location |
(none) |
Enum |
To store Iceberg metadata in a separate directory or under table location
Possible values:- "table-location": Store Iceberg metadata in each table's directory. Useful for standalone Iceberg tables or Java API access. Can also be used with Hive Catalog
- "catalog-location": Store Iceberg metadata in a separate directory. Allows integration with Hive Catalog or Hadoop Catalog.
|
metadata.iceberg.table |
(none) |
String |
Metastore table name for Iceberg Catalog.Set this as an iceberg table alias if using a centralized Catalog. |
metadata.iceberg.uri |
(none) |
String |
Hive metastore uri for Iceberg Hive catalog. |