Acceldata
 ODP
Search documentation… ⌘K

Quota Management

Enable Quotas

celeborn.quota.enabled=true

Quota Properties

Property

Description

celeborn.quota.tenant.diskBytesWritten

Maximum disk bytes written per tenant

celeborn.quota.tenant.diskFileCount

Maximum disk file count per tenant

celeborn.quota.tenant.hdfsBytesWritten

Maximum HDFS bytes written per tenant

celeborn.quota.tenant.hdfsFileCount

Maximum HDFS file count per tenant

dynamicConfig.yaml Example

# System-level defaults
 
- level: SYSTEM
 
 config:
 
 celeborn.quota.tenant.diskBytesWritten: 1T
 celeborn.quota.tenant.diskFileCount: 10000
 celeborn.quota.tenant.hdfsBytesWritten: 500G
 
 
 
# Tenant-level quota override
 
- tenantId: production
 
 level: TENANT
 
 config:
 
 celeborn.quota.tenant.diskBytesWritten: 5T
 celeborn.quota.tenant.diskFileCount: 50000
 
 users:
 
 - name: data_engineer
 
 config:
 
 celeborn.quota.tenant.diskBytesWritten: 2T
 celeborn.quota.tenant.diskFileCount: 20000

Identity Provider

# Use Hadoop UGI to derive username for quota enforcement
 
celeborn.quota.identity.provider=org.apache.celeborn.common.identity.HadoopBasedIdentityProvider