Quota Management
Enable Quotas
celeborn.quota.enabled=true
Quota Properties
Property | Description |
celeborn.quota.tenant.diskBytesWritten | Maximum disk bytes written per tenant |
celeborn.quota.tenant.diskFileCount | Maximum disk file count per tenant |
celeborn.quota.tenant.hdfsBytesWritten | Maximum HDFS bytes written per tenant |
celeborn.quota.tenant.hdfsFileCount | Maximum HDFS file count per tenant |
dynamicConfig.yaml Example
# System-level defaults
- level: SYSTEM
config:
celeborn.quota.tenant.diskBytesWritten: 1T
celeborn.quota.tenant.diskFileCount: 10000
celeborn.quota.tenant.hdfsBytesWritten: 500G
# Tenant-level quota override
- tenantId: production
level: TENANT
config:
celeborn.quota.tenant.diskBytesWritten: 5T
celeborn.quota.tenant.diskFileCount: 50000
users:
- name: data_engineer
config:
celeborn.quota.tenant.diskBytesWritten: 2T
celeborn.quota.tenant.diskFileCount: 20000
Identity Provider
# Use Hadoop UGI to derive username for quota enforcement
celeborn.quota.identity.provider=org.apache.celeborn.common.identity.HadoopBasedIdentityProvider