Reconfigure Cluster
Create Service Config Files
Use the following command to create a service-specific configuration YAML.
accelo admin makeconfig <service_config_name>
Use accelo info to get the respective service config name, except for the following services:
Service Configuration Name | Pulse Service |
ad-core-connectors | ad-connectors, ad-sparkstats |
ad-core | ad-db, ad-tsdb, ad-streaming, ad-graphql |
Update Pulse Configs
Application Config
All cluster configuration information configured during accelo config cluster is stored under the file config/acceldata<cluster_name>.conf
Host Config
All the host-level configuration details are stored in the vars.yml file, and the host list is stored in the hydra_hosts.yml file.
Accelo Reconfig Cluster
This command regenerates the pulse configuration files with the updated information from the cluster.
The list of files that are generated is:
Files | Description |
default-endpoints.yml | Stores the endpoints generated from the Services which is used by the Alerts Services |
hostRoleMap.yml | Store the Host To Roles Maps of the services |
acceldata_<cluster_name>.conf | Main Configuration file for the pulse which stores the connectors info, monitor groups info |
vars.yml | Store the agent-related configuration |
hydra_hosts.yml | Store the list of the hosts in which the hydra agents are to be installed and running. |
Overriding Default vars.yml
To overwrite existing configurations present in vars.yml create a file named as override.yml under work/<clustername>
Vars.yml
base:
enable_ntp_stats: "false"
ntp_daemon: ntpd
Override.yml
base:
enable_ntp_stats: "true"
ntp_daemon: "chronyd"
The following command overrides vars.yml with the required property values.
#here -a is optional for running across all configured clusters
accelo reconfig cluster -a
$ accelo reconfig cluster -h
Dynamic Reconfiguration of the Cluster
Usage:override.yml
accelo reconfig cluster [flags]
accelo reconfig cluster [command]
Available Commands:
vars Archive the vars.yml and hostRoleMap
Flags:
-a, --all All clusters to be reconfigured
-h, --help help for cluster
-s, --skip Skip the reconfiguring of the cluster and will just update the Epoch time and reload the hydra server
For reconfiguring cluster run the following commands:
accelo reconfig cluster -a
Any changes in the acceldata config file require config push to DB using the following command:
accelo admin database push-config
Agents, Base, Static, Components, and JMX
To override the Agent Section, add the agent section to the override.yml file and specify the fields with the values you wish to change.
For example, Suppose you want the pulsenodeagent to have a 60-second interval. The override.yml file will look like the following:
agent:
interval: 60s
Similarly, you can override the Base, Hydra, Static, Components, and JMX sections.
Extra Plugins
Extra Plugins has three individual subsections:
Section | Description |
Impalaagent | If the Impala Service is installed, running, and enabled during the config cluster, this area is filled. |
hbazer | If Hbase Service is installed, running, and enabled during the config cluster, this area is populated. |
jmxagent | The section contains all of the JMX data components that are installed, operating, and enabled during the config cluster. |
To override a subsection of the Extra Plugins Section, specify the subsection you want to change with the modified values.
For example: In the jmxagent subsection, addhive server interactive appended to the previous values (HBase master, HBase regionserver, data node, namenode, journalnode).
extra_plugins:
jmxagent:
- hive_server_interactive
- hbase_master
- hbase_regionserver
- datanode
- namenode
- journalnode
EnabledJMX
These two sections contain a list of plugins that are enabled during the config cluster process.
- extra_plugins
- enabled_jmx
To uninstall a single or several plugins, add the section to the override.yml file.
For example, If you want to add hive_metastore in the enabled_jmx section do the following:
enabled_jmx:
- hive_metastore
If you want to remove the hive_metastore from the enabled_ jmx section copy the section from the vars.yml file and paste it in the override.yml file and remove the hive_metastore line and run the accelo reconfig cluster.
enabled_jmx:
- hive_server
- zookeeper_server
- zeppelin_master
- kafka_broker
Hosts
The Host to the installed component Role map is found in the Hosts section.
If you wish to add a host that is not already in the Ambari, Cloudera, or standalone cluster, you can do so in the override.yml file.
For example:
hosts:
host1:
- nifi_server
LogLocation
Add the log title to the enabled logs section if you wish to add a LogLocation that was not previously enabled. Add the subsection from the vars.yml file with that change.
For example, If you want to enable the Kafka_err logs then the override.yml file will look like the following block.
enabled_logs:
- kafka_err
- hive_server
- hive_metastore
log_locations:
kafka_err:
- path: /var/log/kafka/kafka.err
type: KAFKA_ERR
On removing the kafka_err logs the override.yml file will look like this:
enabled_logs:
- hive_server
- hive_metastore
Elastic connection timeout from ad-graphql
A new configuration named ELASTIC_CONNECTION_TIMEOUT has been introduced, with a default value set to 60 seconds. This configuration allows control over the connection timeout from ad-graphql to Elasticsearch.
Add the variable ELASTIC_CONNECTION_TIMEOUT to the env section of ad-graphql in the $AcceloHome/config/docker/ad-core.yml file. After adding the variable, your ad-core.yml file should appear as follows:
ad-graphql:
image: ad-graphql
container_name: ""
environment:
- ELASTIC_CONNECTION_TIMEOUT=10000
