Install Impala on Kubernetes using CLI

Sergei Tikhomirov

This article covers how to deploy Impala in Kubernetes using the adc CLI tool. Security configurations for Impala on Kubernetes are covered in separate articles:

Prerequisites

To deploy Impala on Kubernetes via CLI, you need:

  • A Kubernetes cluster (1.32 or later) with access configured through kubectl.

  • The CLI tool that is unpacked from your offline pack.

  • The following images that are unpacked and pushed to your repository:

    • hub.arenadata.io/adc-enterprise/impala-operator:<version>

    • hub.arenadata.io/adh-enterprise/impala-docker:<version>

    These artifacts can be found in the offline packages, which can be requested from the Arenadata support team.

Further prerequisites depend on the storage being used:

  • HDFS

  • S3

A compatible version of an ADH cluster with the following services is up and running:

  • Core configuration

  • ADPG

  • Zookeeper

  • HDFS

  • YARN

  • Hive

  • Hive Metastore (you can optionally use your own catalog, but there’s no compatibility guarantee).

  • S3-compatible storage.

If you use ADH HMS, perform the following steps:

  • In ADCM, configure the Core configuration service by setting the following parameters in the core-site.xml group:

    • fs.defaultFS — s3://demo-s3

    • fs.s3a.endpoint — <s3_host>

    • fs.s3a.access.key — <access_key>

    • fs.s3a.secret.key — <secret_key>

    • fs.s3a.path.style.access — true

  • Configure the Hive service by setting the following parameter in the hive-site.xml group:

    • hive.metastore.warehouse.dir — s3a://demo-s3/apps/hive/warehouse

Step 1. Install the Impala operator

  1. Initiate the Impala operator:

    $ ./adc init --impala-operator -o impala-operator.yaml

    This operation creates the impala-operator.yaml file with a configuration template.

  2. Edit the configuration file to your needs:

    impala-operator.yaml
    apiVersion: adc.arenadata.io/v1alpha1
kind: ImpalaOperator
metadata:
  name: impala-operator
  namespace: impala-operator (1)
spec:
  image: hub.arenadata.io/adc-enterprise/impala-operator:<tag> (2)

  # Number of replicas
  # replicas: 1

  resources:
    limits:
      cpu: 500m
      memory: 256Mi

  # Operator ServiceAccount. create: true (default) also creates the manager and per-payload-namespace Role/RoleBinding bound to it; create: false skips all three - name then refers to a ServiceAccount (and RBAC) managed entirely outside the CLI.
  serviceAccount: (3)
    create: true
    name: "impala-operator"

  # Whether the CLI creates the product namespace.
  # The namespace name is set in metadata.namespace.
  namespace:
    create: true

  # Create namespaces to run the payload.
  createPayloadNamespaces: true

  # List of namespaces to run the payload in.
  payloadNamespaces: (4)
    - impala

  ## Image pull secret for a private registry.
  ## Set 'externalSecretName' to reference an existing Secret,
  ## or set 'credentials' and optionally 'secretName' to let the CLI create one.
  #imagePullSecret:
  #  # Use a Secret managed outside ADC.
  #  externalSecretName: existing-registry-secret
  #
  #  ## Or let ADC create the Secret.
  #  #secretName: custom-registry-secret
  #
  #  #credentials:
  #  #  registry: registry.example.com
  #  #  username: user
  #  #  password: pass

  ## Operator monitoring configuration. Supports product-specific metrics export and optional vmagent delivery to an external ADM.
  #monitoring:
  #  # Renders a namespace-scoped vmagent that sends metrics to an external ADM.
  #  vmagent:
  #    remoteWrite:
  #      url: http://vminsert.example.com/insert/0/prometheus/api/v1/write
  #    scrapeInterval: 30s
  #    image: hub.arenadata.io/adm-enterprise/vmagent:1.136.0-adm-5.0.0-x86_64
  #    resources:
  #      limits:
  #        cpu: 500m
  #        memory: 512Mi
  #      requests:
  #        cpu: 100m
  #        memory: 128Mi
  #
  #    ## HTTPS settings used by vmagent when scraping product metrics.
  #    #tls:
  #    #  ## CA certificate source used to verify the metrics endpoint.
  #    #  #ca:
  #    #  #  # Use a Secret managed outside ADC.
  #    #  #  externalSecretName: existing-product-metrics-ca
  #    #  #
  #    #  #  ## Or let ADC create the Secret.
  #    #  #  #secretName: product-metrics-ca
  #    #  #
  #    #  #  # Key containing the CA certificate in the referenced Secret.
  #    #  #  certificateKey: ca.crt
  #    #  #
  #    #  #  ## Local CA certificate read by ADC to create the configured Secret.
  #    #  #  #files:
  #    #  #  #  certificatePath: /path/to/ca.crt
  #    #
  #    #  ## Server name used to verify the metrics endpoint certificate hostname.
  #    #  #serverName: metrics.example.com
  #    #
  #    #  # Skip verification of the metrics endpoint certificate. Do not use together with ca.
  #    #  insecureSkipVerify: true
    1 Namespace settings.
    2 URL to the Impala operator image in your repository.
    3 Service account settings.
    4 Payload namespace settings. The listed namespaces will be available to the Impala operator instance.

  3. You can check the configuration about to be applied by running the apply command with the --dry-run option:

    $ ./adc apply -f impala-operator.yaml --dry-run > impala-operator-render.yaml
    impala-operator-render.yaml
    ---
apiVersion: v1
kind: Namespace
metadata:
  name: impala-operator
spec: {}
status: {}
---
apiVersion: v1
kind: Namespace
metadata:
  name: impala
spec: {}
status: {}
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: impala-operator
  namespace: impala-operator
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: impala-operator-impala-operator-manager
  namespace: impala-operator
rules:
- apiGroups:
  - events.k8s.io
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - impala.arenadata.io
  resources:
  - clusters
  - executorgroups
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - impala.arenadata.io
  resources:
  - clusters/status
  - executorgroups/status
  verbs:
  - get
  - patch
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: impala-operator-impala-operator-manager
  namespace: impala-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: impala-operator-impala-operator-manager
subjects:
- kind: ServiceAccount
  name: impala-operator
  namespace: impala-operator
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: impala-operator-impala-operator-payload
  namespace: impala
rules:
- apiGroups:
  - events.k8s.io
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - ""
  resources:
  - secrets
  - services
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - deployments
  - statefulsets
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - impala.arenadata.io
  resources:
  - clusters
  - executorgroups
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - impala.arenadata.io
  resources:
  - clusters/status
  - executorgroups/status
  verbs:
  - get
  - patch
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: impala-operator-impala-operator-payload
  namespace: impala
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: impala-operator-impala-operator-payload
subjects:
- kind: ServiceAccount
  name: impala-operator
  namespace: impala-operator
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: operator
    app.kubernetes.io/managed-by: adc-cli
    arenadata.io/operator-type: impala
  name: impala-operator-impala-operator
  namespace: impala-operator
spec:
  selector:
    matchLabels:
      app.kubernetes.io/component: operator
      app.kubernetes.io/managed-by: adc-cli
      app.kubernetes.io/name: impala-operator-impala-operator
  strategy: {}
  template:
    metadata:
      labels:
        app.kubernetes.io/component: operator
        app.kubernetes.io/managed-by: adc-cli
        app.kubernetes.io/name: impala-operator-impala-operator
    spec:
      containers:
      - args:
        - -ns=impala
        image: hub.arenadata.io/adc-enterprise/impala-operator:<tag>
        imagePullPolicy: Always
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8081
          initialDelaySeconds: 5
          periodSeconds: 10
        name: app
        readinessProbe:
          httpGet:
            path: /readyz
            port: 8081
          initialDelaySeconds: 5
          periodSeconds: 10
        resources:
          limits:
            cpu: 500m
            memory: 256Mi
          requests:
            cpu: 500m
            memory: 256Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
      securityContext:
        fsGroup: 10001
        runAsGroup: 10001
        runAsNonRoot: true
        runAsUser: 10001
      serviceAccountName: impala-operator
      terminationGracePeriodSeconds: 10
status: {}

  4. If the manifest is correct, apply the configuration and deploy the Impala operator:

    $ ./adc apply -f impala-operator.yaml

    The expected output contains the confirmation of success:

    time="20260724055338UTC" level="info" msg="operator impala/impala applied to namespace impala-operator"

  5. Verify the Impala operator:

    $ kubectl get pods -n impala-operator

    The expected output should be similar to:

    NAME                               READY   STATUS    RESTARTS      AGE
impala-operator-impala-operator-6bf8788587-7s22r   1/1     Running   0          150m

Step 2. Install Impala cluster

  1. Prepare the hadoop_conf.yaml Hadoop configuration file:

    hadoop_conf.yaml

    • HDFS

    • S3

    sites:
  core:
    fs.defaultFS: hdfs://adh
    hadoop.security.authentication: simple
    dfs.client.failover.proxy.provider.adh: org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider
    dfs.ha.namenodes.adh: nn_tsn-adh-k8s-1,nn_tsn-adh-k8s-3
    dfs.namenode.rpc-address.adh.nn_tsn-adh-k8s-1: tsn-adh-k8s-1.ru-central1.internal:8020
    dfs.namenode.rpc-address.adh.nn_tsn-adh-k8s-3: tsn-adh-k8s-3.ru-central1.internal:8020
    dfs.nameservices: adh
  hdfs:
    dfs.client.read.shortcircuit: false
  ozone:
    ozone.om.address.adh.om_tsn-adh-k8s-1: tsn-adh-k8s-1.ru-central1.internal:9862
    ozone.om.address.adh.om_tsn-adh-k8s-2: tsn-adh-k8s-2.ru-central1.internal:9862
    ozone.om.address.adh.om_tsn-adh-k8s-3: tsn-adh-k8s-3.ru-central1.internal:9862
    ozone.om.nodes.adh: om_tsn-adh-k8s-1,om_tsn-adh-k8s-2,om_tsn-adh-k8s-3
    ozone.om.service.ids: adhom
  hive:
    hive.metastore.sasl.enabled: false
    hive.metastore.uris: thrift://tsn-adh-k8s-1.ru-central1.internal:9083
    metastore.use.SSL: false
    sites:
  core:
    fs.defaultFS: s3a://demo-s3
    fs.s3a.impl: org.apache.hadoop.fs.s3a.S3AFileSystem
    fs.s3a.access.key: p.petrov@RU-CENTRAL1.INTERNAL
    fs.s3a.secret.key: 3ab197f82fac43374519b4ad3015a76c778acf51897cf73f409ecba827e64255
  hive:
    hive.metastore.uris: thrift://hms-adh-nia-01.ru-central1.internal:9083
    metastore.use.SSL: false
    fs.s3a.endpoint: http://adh-ctrl-nia-02.ru-central1.internal:9878
    fs.s3a.path.style.access: true
    hive.metastore.warehouse.dir: s3a://demo-s3/apps/hive/warehouse

  2. Initialize the Impala cluster:

    $ ./adc init --impala-cluster --hadoop-file hadoop_conf.yaml -o impala-cluster.yaml

    This operation creates the impala-cluster.yaml file with a configuration template.

  3. Edit the configuration file to your needs:

    impala-cluster.yaml

    • HDFS

    • S3

    apiVersion: adc.arenadata.io/v1alpha1
kind: ImpalaCluster
metadata:
  name: impala
  namespace: impala (1)
spec:
  image: hub.arenadata.io/adc-enterprise/impala:<tag> (2)

  ## Image pull secret for a private registry.
  ## Set 'externalSecretName' to reference an existing Secret,
  ## or set 'credentials' and optionally 'secretName' to let the CLI create one.
  #imagePullSecret:
  #  # Use a Secret managed outside ADC.
  #  externalSecretName: existing-registry-secret
  #
  #  ## Or let ADC create the Secret.
  #  #secretName: custom-registry-secret
  #
  #  #credentials:
  #  #  registry: registry.example.com
  #  #  username: user
  #  #  password: pass

  hadoop: (3)
    core:
      dfs.client.failover.proxy.provider.adh: org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider
      dfs.ha.namenodes.adh: nn_tsn-adh-k8s-1,nn_tsn-adh-k8s-3
      dfs.namenode.rpc-address.adh.nn_tsn-adh-k8s-1: tsn-adh-k8s-1.ru-central1.internal:8020
      dfs.namenode.rpc-address.adh.nn_tsn-adh-k8s-3: tsn-adh-k8s-3.ru-central1.internal:8020
      dfs.nameservices: adh
      fs.defaultFS: hdfs://adh
      hadoop.security.authentication: simple
    hdfs:
      dfs.client.read.shortcircuit: "false"
    hive:
      hive.metastore.sasl.enabled: "false"
      hive.metastore.uris: thrift://tsn-adh-k8s-1.ru-central1.internal:9083
      metastore.use.SSL: "false"
    ozone:
      ozone.om.address.adh.om_tsn-adh-k8s-1: tsn-adh-k8s-1.ru-central1.internal:9862
      ozone.om.address.adh.om_tsn-adh-k8s-2: tsn-adh-k8s-2.ru-central1.internal:9862
      ozone.om.address.adh.om_tsn-adh-k8s-3: tsn-adh-k8s-3.ru-central1.internal:9862
      ozone.om.nodes.adh: om_tsn-adh-k8s-1,om_tsn-adh-k8s-2,om_tsn-adh-k8s-3
      ozone.om.service.ids: adhom

  ## Kerberos configuration for authentication.
  #kerberos:
  #  realm: EXAMPLE.COM
  #
  #  # Service name in the Kerberos principal. Defaults to the product name.
  #  service: impala
  #
  #  # Hostname in the Kerberos principal.
  #  # Required for a fixed service principal; leave it empty only to derive one principal per pod from the cluster domain.
  #  hostname: kerberos.example.com
  #  keytab:
  #    # true - kerberos-operator creates the keytab Secret.
  #    # false (default) - reference an existing keytab Secret with name keytab.secretName.
  #    create: false
  #
  #    # Name of the keytab Secret.
  #    # Optional when create: true - names the generated Secret (default: <name>-keytab).
  #    # Required when create: false - must reference an existing Secret.
  #    secretName: kerberos-secret
  #
  #    # Label selector for the Pod that generates the keytab.
  #    # Required when create: true; ignored when create: false.
  #    labelSelector:
  #      env: prod
  #    #additionalPrincipals:
  #    #  - HTTP/kerberos.example.com
  #
  #    clusterDomain: cluster.local
  #    rotation:
  #      interval: 24h
  #      checkInterval: 1h

  ## LDAP authentication configuration.
  ## Uncomment and fill url and userBindPattern.
  ## For ldaps:// URLs the ssl: or ca: section must also be configured (depends on product)
  #ldap:
  #  # LDAP service url.
  #  url: ldaps://ldap.example.com:636
  #
  #  # LDAP user Bind pattern.
  #  userBindPattern: uid=#UID,cn=users,dc=example,dc=com

  ## Ranger plugin configuration.
  ## Uncomment and fill the lines below. adc apply derives the rest.
  #ranger:
  #  # fill ranger.plugin.impala.policy.rest.url below with Ranger endpoint, e.g. https://adps-adc.ru-central1.internal:6182
  #  # fill ranger.plugin.impala.service.name below with Ranger service name you want to use for product, e.g. adc_impala_id_1
  #  security:
  #    ranger.plugin.impala.policy.rest.url: ""
  #    ranger.plugin.impala.service.name: ""
  #
  #  # fill xasecure.audit.destination.solr.zookeepers below with Zookeepers endpoints to resolve solr service, e.g. adps-adc.ru-central1.internal:2181/Arenadata.Hadoop-2.solr.server
  #  audit:
  #    xasecure.audit.destination.solr.zookeepers: ""
  #
  #  # Local Ranger files 'adc apply' writes into the configs Secret.
  #  # Relative paths are resolved against the config file.
  #  files:
  #    jceksStorePath: /path/to/ranger.jceks

  ## Java KeyStore/TrustStore certificate configuration.
  ## Set externalSecretName to reference an existing Secret,
  ## or set files and optional secretName to have ADC create it.
  #ssl:
  #  ## Name of the Secret containing Java keystores.
  #  #secretName: custom-ssl-secret
  #  externalSecretName: existing-ssl-secret
  #
  #  # Key in the Secret containing the truststore file.
  #  trustStoreKey: truststore.jks
  #
  #  ## Password for the truststore (optional).
  #  #trustStorePassword: bigdata
  #
  #  ## Key in the Secret containing the keystore file (optional).
  #  #keyStoreKey: keystore.jks
  #
  #  ## Password for the keystore (optional).
  #  #keyStorePassword: bigdata
  #
  #  ## Local files 'adc apply' puts into the Secret named by ssl.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  trustStorePath: /path/to/truststore.jks
  #  #  #keyStorePath: /path/to/keystore.jks

  ## Use an external complete configs Secret instead of the one rendered by ADC.
  #configsSecret:
  #  # Use a Secret managed outside ADC.
  #  externalSecretName: existing-impala-configs
  #
  #  ## Or let ADC create the Secret.
  #  #secretName: custom-impala-configs

  catalog:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata
  coordinator:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata
  executor:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata
  statestore:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata

  ## Monitoring configuration.
  #monitoring:
  #  # Enables Prometheus metrics export from this product's pods.
  #  exportMetrics: true

  ## Admission-control resource pools. Rendered into a coordinator-only Secret (fair-scheduler.xml and optional llama-site.xml) mounted at /opt/impala/resource-pools.
  #resourcePools:
  #  # Fair-scheduler queue tree and placement policy, rendered to fair-scheduler.xml.
  #  allocations:
  #    # Queue tree, rooted at a single queue.
  #    queues:
  #      - aclSubmitApps: ' '
  #        name: root
  #        queues:
  #        - aclSubmitApps: '*'
  #          maxResources:
  #            memory: 50000
  #            vcores: 0
  #          name: default
  #          type: leaf
  #        type: parent
  #
  #    # Rules that route an incoming query to a queue.
  #    queuePlacementPolicy:
  #      rules:
  #        - create: false
  #          name: specified
  #        - name: default
  #
  #  # Raw llama-site.xml properties, rendered verbatim in list order.
  #  llamaProperties:
  #    # List of name/value pairs written to llama-site.xml.
  #    properties:
  #      - name: llama.am.throttling.maximum.placed.reservations.root.default
  #        value: "10"

  ## TLS certificate configuration.
  ## Set externalSecretName to reference an existing Secret,
  ## or set files and optional secretName to have ADC create it.
  #tls:
  #  ## Optional name of the Secret ADC creates from local files.
  #  #secretName: custom-tls-secret
  #  externalSecretName: existing-tls-secret
  #
  #  # Key in the Secret containing the TLS certificate.
  #  certificateKey: tls.crt
  #
  #  # Key in the Secret containing the TLS private key.
  #  privateKey: tls.key
  #
  #  ## Key in the Secret containing the client CA certificate.
  #  #clientCaCertificate: ca.crt
  #
  #  ## Local files 'adc apply' puts into the Secret named by tls.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  certificatePath: /path/to/tls.crt
  #  #  privateKeyPath: /path/to/tls.key
  #  #  #clientCaCertificatePath: /path/to/ca.crt

  ## TLS certificate configuration for web UI and HTTP endpoints.
  ## Set externalSecretName to reference an existing Secret,
  ## or set files and optional secretName to have ADC create it.
  #webTLS:
  #  ## Optional name of the Secret ADC creates from local files.
  #  #secretName: custom-web-tls-secret
  #  externalSecretName: existing-web-tls-secret
  #
  #  # Key in the Secret containing the web TLS certificate.
  #  certificateKey: tls.crt
  #
  #  # Key in the Secret containing the web TLS private key.
  #  privateKey: tls.key
  #
  #  ## Local files 'adc apply' puts into the Secret named by webTLS.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  certificatePath: /path/to/tls.crt
  #  #  privateKeyPath: /path/to/tls.key

  ## CA certificate configuration for Impala.
  ## Set externalSecretName, or let ADC create a Secret from files.
  #ca:
  #  ## Optional name of the Secret ADC creates from a local file.
  #  #secretName: custom-ca-secret
  #  externalSecretName: existing-ca-secret
  #
  #  # Key in the Secret containing the CA certificate.
  #  certificateKey: ca.crt
  #
  #  ## Local files 'adc apply' puts into the Secret named by ca.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  certificatePath: /path/to/ca.pem

  ## Controls whether Secret/ConfigMap changes restart pods.
  ## Set enabled: false to update referenced Secrets without restarting
  ## the workload; pods keep running the previous configuration until
  ## the policy is re-enabled. Defaults to enabled.
  #configurationRollout:
  #  enabled: false
    1 Namespace that the Impala cluster will use.
    2 Settings for pulling the Impala cluster image.
    3 Hadoop settings that were taken from the previously created hadoop_conf.yaml. 
    apiVersion: adc.arenadata.io/v1alpha1
kind: ImpalaCluster
metadata:
  name: impala
  namespace: impala (1)
spec:
  image: hub.arenadata.io/adc-enterprise/impala:<tag> (2)

  ## Image pull secret for a private registry.
  ## Set 'externalSecretName' to reference an existing Secret,
  ## or set 'credentials' and optionally 'secretName' to let the CLI create one.
  #imagePullSecret:
  #  # Use a Secret managed outside ADC.
  #  externalSecretName: existing-registry-secret
  #
  #  ## Or let ADC create the Secret.
  #  #secretName: custom-registry-secret
  #
  #  #credentials:
  #  #  registry: registry.example.com
  #  #  username: user
  #  #  password: pass

  hadoop: (3)
    core:
      fs.defaultFS: s3a://demo-s3
      fs.s3a.access.key: p.petrov@RU-CENTRAL1.INTERNAL
      fs.s3a.impl: org.apache.hadoop.fs.s3a.S3AFileSystem
      fs.s3a.secret.key: 3ab197f82fac43374519b4ad3015a76c778acf51897cf73f409ecba827e64255
    hive:
      fs.s3a.endpoint: http://adh-ctrl-nia-02.ru-central1.internal:9878
      fs.s3a.path.style.access: "true"
      hive.metastore.uris: thrift://hms-adh-nia-01.ru-central1.internal:9083
      hive.metastore.warehouse.dir: s3a://demo-s3/apps/hive/warehouse
      metastore.use.SSL: "false"

  ## Kerberos configuration for authentication.
  #kerberos:
  #  realm: EXAMPLE.COM
  #
  #  # Service name in the Kerberos principal. Defaults to the product name.
  #  service: impala
  #
  #  # Hostname in the Kerberos principal.
  #  # Required for a fixed service principal; leave it empty only to derive one principal per pod from the cluster domain.
  #  hostname: kerberos.example.com
  #  keytab:
  #    # true - kerberos-operator creates the keytab Secret.
  #    # false (default) - reference an existing keytab Secret with name keytab.secretName.
  #    create: false
  #
  #    # Name of the keytab Secret.
  #    # Optional when create: true - names the generated Secret (default: <name>-keytab).
  #    # Required when create: false - must reference an existing Secret.
  #    secretName: kerberos-secret
  #
  #    # Label selector for the Pod that generates the keytab.
  #    # Required when create: true; ignored when create: false.
  #    labelSelector:
  #      env: prod
  #    #additionalPrincipals:
  #    #  - HTTP/kerberos.example.com
  #
  #    clusterDomain: cluster.local
  #    rotation:
  #      interval: 24h
  #      checkInterval: 1h

  ## LDAP authentication configuration.
  ## Uncomment and fill url and userBindPattern.
  ## For ldaps:// URLs the ssl: or ca: section must also be configured (depends on product)
  #ldap:
  #  # LDAP service url.
  #  url: ldaps://ldap.example.com:636
  #
  #  # LDAP user Bind pattern.
  #  userBindPattern: uid=#UID,cn=users,dc=example,dc=com

  ## Ranger plugin configuration.
  ## Uncomment and fill the lines below. adc apply derives the rest.
  #ranger:
  #  # fill ranger.plugin.impala.policy.rest.url below with Ranger endpoint, e.g. https://adps-adc.ru-central1.internal:6182
  #  # fill ranger.plugin.impala.service.name below with Ranger service name you want to use for product, e.g. adc_impala_id_1
  #  security:
  #    ranger.plugin.impala.policy.rest.url: ""
  #    ranger.plugin.impala.service.name: ""
  #
  #  # fill xasecure.audit.destination.solr.zookeepers below with Zookeepers endpoints to resolve solr service, e.g. adps-adc.ru-central1.internal:2181/Arenadata.Hadoop-2.solr.server
  #  audit:
  #    xasecure.audit.destination.solr.zookeepers: ""
  #
  #  # Local Ranger files 'adc apply' writes into the configs Secret.
  #  # Relative paths are resolved against the config file.
  #  files:
  #    jceksStorePath: /path/to/ranger.jceks

  ## Java KeyStore/TrustStore certificate configuration.
  ## Set externalSecretName to reference an existing Secret,
  ## or set files and optional secretName to have ADC create it.
  #ssl:
  #  ## Name of the Secret containing Java keystores.
  #  #secretName: custom-ssl-secret
  #  externalSecretName: existing-ssl-secret
  #
  #  # Key in the Secret containing the truststore file.
  #  trustStoreKey: truststore.jks
  #
  #  ## Password for the truststore (optional).
  #  #trustStorePassword: bigdata
  #
  #  ## Key in the Secret containing the keystore file (optional).
  #  #keyStoreKey: keystore.jks
  #
  #  ## Password for the keystore (optional).
  #  #keyStorePassword: bigdata
  #
  #  ## Local files 'adc apply' puts into the Secret named by ssl.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  trustStorePath: /path/to/truststore.jks
  #  #  #keyStorePath: /path/to/keystore.jks

  ## Use an external complete configs Secret instead of the one rendered by ADC.
  #configsSecret:
  #  # Use a Secret managed outside ADC.
  #  externalSecretName: existing-impala-configs
  #
  #  ## Or let ADC create the Secret.
  #  #secretName: custom-impala-configs

  catalog:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata
  coordinator:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata
  executor:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata
  statestore:
    replicas: 1
    #resources:
    #  limits:
    #    cpu: "2"
    #    memory: 8Gi
    #  requests:
    #    cpu: 300m
    #    memory: 384Mi

    ## Component arguments. Key-value pairs passed to the component configuration.
    #args:
    #  redirect_stdout_stderr: "false"

    ## Environment variables passed to the component container.
    #envs:
    #  - name: JAVA_TOOL_OPTIONS
    #    value: |-
    #      -Djavax.net.ssl.trustStore=/etc/ssl/truststore.jks
    #      -Djavax.net.ssl.trustStorePassword=bigdata

  ## Monitoring configuration.
  #monitoring:
  #  # Enables Prometheus metrics export from this product's pods.
  #  exportMetrics: true

  ## Admission-control resource pools. Rendered into a coordinator-only Secret (fair-scheduler.xml and optional llama-site.xml) mounted at /opt/impala/resource-pools.
  #resourcePools:
  #  # Fair-scheduler queue tree and placement policy, rendered to fair-scheduler.xml.
  #  allocations:
  #    # Queue tree, rooted at a single queue.
  #    queues:
  #      - aclSubmitApps: ' '
  #        name: root
  #        queues:
  #        - aclSubmitApps: '*'
  #          maxResources:
  #            memory: 50000
  #            vcores: 0
  #          name: default
  #          type: leaf
  #        type: parent
  #
  #    # Rules that route an incoming query to a queue.
  #    queuePlacementPolicy:
  #      rules:
  #        - create: false
  #          name: specified
  #        - name: default
  #
  #  # Raw llama-site.xml properties, rendered verbatim in list order.
  #  llamaProperties:
  #    # List of name/value pairs written to llama-site.xml.
  #    properties:
  #      - name: llama.am.throttling.maximum.placed.reservations.root.default
  #        value: "10"

  ## TLS certificate configuration.
  ## Set externalSecretName to reference an existing Secret,
  ## or set files and optional secretName to have ADC create it.
  #tls:
  #  ## Optional name of the Secret ADC creates from local files.
  #  #secretName: custom-tls-secret
  #  externalSecretName: existing-tls-secret
  #
  #  # Key in the Secret containing the TLS certificate.
  #  certificateKey: tls.crt
  #
  #  # Key in the Secret containing the TLS private key.
  #  privateKey: tls.key
  #
  #  ## Key in the Secret containing the client CA certificate.
  #  #clientCaCertificate: ca.crt
  #
  #  ## Local files 'adc apply' puts into the Secret named by tls.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  certificatePath: /path/to/tls.crt
  #  #  privateKeyPath: /path/to/tls.key
  #  #  #clientCaCertificatePath: /path/to/ca.crt

  ## TLS certificate configuration for web UI and HTTP endpoints.
  ## Set externalSecretName to reference an existing Secret,
  ## or set files and optional secretName to have ADC create it.
  #webTLS:
  #  ## Optional name of the Secret ADC creates from local files.
  #  #secretName: custom-web-tls-secret
  #  externalSecretName: existing-web-tls-secret
  #
  #  # Key in the Secret containing the web TLS certificate.
  #  certificateKey: tls.crt
  #
  #  # Key in the Secret containing the web TLS private key.
  #  privateKey: tls.key
  #
  #  ## Local files 'adc apply' puts into the Secret named by webTLS.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  certificatePath: /path/to/tls.crt
  #  #  privateKeyPath: /path/to/tls.key

  ## CA certificate configuration for Impala.
  ## Set externalSecretName, or let ADC create a Secret from files.
  #ca:
  #  ## Optional name of the Secret ADC creates from a local file.
  #  #secretName: custom-ca-secret
  #  externalSecretName: existing-ca-secret
  #
  #  # Key in the Secret containing the CA certificate.
  #  certificateKey: ca.crt
  #
  #  ## Local files 'adc apply' puts into the Secret named by ca.secretName.
  #  ## Relative paths are resolved against the config file.
  #  #files:
  #  #  certificatePath: /path/to/ca.pem

  ## Controls whether Secret/ConfigMap changes restart pods.
  ## Set enabled: false to update referenced Secrets without restarting
  ## the workload; pods keep running the previous configuration until
  ## the policy is re-enabled. Defaults to enabled.
  #configurationRollout:
  #  enabled: false
    1 Namespace that the Impala cluster will use.
    2 Settings for pulling the Impala cluster image.
    3 Hadoop settings that were taken from the previously created hadoop_conf.yaml.

  4. You can check the configuration about to be applied by running the apply command with the --dry-run option:

    $ ./adc apply -f impala-cluster.yaml --dry-run > impala-cluster-render.yaml
    impala-cluster-render.yaml
    ---
apiVersion: v1
kind: Secret
metadata:
  name: impala-configs
  namespace: impala
stringData:
  core-site.xml: |-
    <configuration>
      <property>
        <name>dfs.client.failover.proxy.provider.adh</name>
        <value>org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider</value>
      </property>
      <property>
        <name>dfs.client.read.shortcircuit</name>
        <value>false</value>
      </property>
      <property>
        <name>dfs.ha.namenodes.adh</name>
        <value>nn_tsn-adh-k8s-1,nn_tsn-adh-k8s-3</value>
      </property>
      <property>
        <name>dfs.namenode.rpc-address.adh.nn_tsn-adh-k8s-1</name>
        <value>tsn-adh-k8s-1.ru-central1.internal:8020</value>
      </property>
      <property>
        <name>dfs.namenode.rpc-address.adh.nn_tsn-adh-k8s-3</name>
        <value>tsn-adh-k8s-3.ru-central1.internal:8020</value>
      </property>
      <property>
        <name>dfs.nameservices</name>
        <value>adh</value>
      </property>
      <property>
        <name>fs.defaultFS</name>
        <value>hdfs://adh</value>
      </property>
      <property>
        <name>hadoop.security.authentication</name>
        <value>simple</value>
      </property>
      <property>
        <name>ozone.om.address.adh.om_tsn-adh-k8s-1</name>
        <value>tsn-adh-k8s-1.ru-central1.internal:9862</value>
      </property>
      <property>
        <name>ozone.om.address.adh.om_tsn-adh-k8s-2</name>
        <value>tsn-adh-k8s-2.ru-central1.internal:9862</value>
      </property>
      <property>
        <name>ozone.om.address.adh.om_tsn-adh-k8s-3</name>
        <value>tsn-adh-k8s-3.ru-central1.internal:9862</value>
      </property>
      <property>
        <name>ozone.om.nodes.adh</name>
        <value>om_tsn-adh-k8s-1,om_tsn-adh-k8s-2,om_tsn-adh-k8s-3</value>
      </property>
      <property>
        <name>ozone.om.service.ids</name>
        <value>adhom</value>
      </property>
    </configuration>
  hive-site.xml: |-
    <configuration>
      <property>
        <name>hive.metastore.sasl.enabled</name>
        <value>false</value>
      </property>
      <property>
        <name>hive.metastore.uris</name>
        <value>thrift://tsn-adh-k8s-1.ru-central1.internal:9083</value>
      </property>
      <property>
        <name>metastore.use.SSL</name>
        <value>false</value>
      </property>
    </configuration>
type: Opaque
---
apiVersion: impala.arenadata.io/v1alpha1
kind: Cluster
metadata:
  name: impala
  namespace: impala
spec:
  catalog:
    metadata: {}
    replicas: 1
    spec:
      image: hub.arenadata.io/adc-enterprise/impala:<tag>
      imagePullPolicy: Always
  coordinator:
    metadata: {}
    replicas: 1
    spec:
      image: hub.arenadata.io/adc-enterprise/impala:<tag>
      imagePullPolicy: Always
  executor:
    metadata: {}
    replicas: 1
    spec:
      image: hub.arenadata.io/adc-enterprise/impala:<tag>
      imagePullPolicy: Always
  hadoopConfigsSecretName: impala-configs
  statestore:
    metadata: {}
    replicas: 1
    spec:
      image: hub.arenadata.io/adc-enterprise/impala:<tag>
      imagePullPolicy: Always
status: {}

  5. If the manifest is correct, apply the configuration and deploy the Impala cluster:

    $ ./adc apply -f impala-cluster.yaml

    The expected output contains a confirmation of success:

    time="20260518133858UTC" level="info" msg="cluster impala applied to namespace impala"

  6. Verify the Impala cluster pods:

    $ kubectl get pods -n impala

    The expected output is:

    NAME                   READY   STATUS    RESTARTS   AGE
impala-catalog-0       1/1     Running   0          70m
impala-coordinator-0   1/1     Running   0          70m
impala-executor-0      1/1     Running   0          70m
impala-statestore-0    1/1     Running   0          70m

Step 3. Allow JDBC connections to Impala

For external JDBC access to Impala, you need to expose the service using one of the supported publication methods, for example, through a load balancer or Ingress controller.

All configurations related to exposing a service, including DNS, annotations, Ingress settings, load balancing rules, and other platform-specific settings, should be specified according to your Kubernetes environment.

  1. Get the external IP address of your Ingress controller or load balancer. For example:

    impala-lb                    LoadBalancer   10.96.231.158   10.92.42.144   21050:32154/TCP,26000:30753/TCP,24000:32645/TCP   25h

  2. Add the following entry to your /etc/hosts file:

    <lb_ip> impala-jdbc.ru-central1.internal

    where <lb_ip> is the external IP exposed by your load balancer. In this example, it is 10.92.42.144.

  3. Connect to the Impala cluster over JDBC, for example, using DBeaver. For this, the JDBC connection string looks as follows:

    jdbc:impala://impala-jdbc.ru-central1.internal:21050/default

  4. Once connected, verify the Impala cluster operability:

    SHOW DATABASES;

    The expected output:

    name            |comment                                     |
----------------+--------------------------------------------+
_impala_builtins|System database for Impala builtin functions|
default         |Default Hive database                       |

Step 4. Provide access to Impala web UI

To access Impala web interface, you need to expose the service using one of the supported publication methods, for example, through a load balancer or Ingress controller. All configurations related to exposing a service, including DNS, annotations, Ingress settings, load balancing rules, and other platform-specific settings, should be specified according to your Kubernetes environment.

  1. Get the external IP address of your load balancer or Ingress controller. For example:

    NAME             CLASS   HOSTS                               ADDRESS       PORTS   AGE
impala-ingress   nginx   impala-cloud.ru-central1.internal   10.92.41.95   80      8m45s

  2. Add the following entry to your /etc/hosts file:

    <ingress_ip> impala-cloud.ru-central1.internal

    where <ingress_ip> is the external IP exposed by Ingress. In this example, it is 10.92.41.95.

  3. Open Impala web UI in your browser, using the URL: http://impala-cloud.ru-central1.internal (change the protocol to https if you use Kerberos and SSL).

    Impala web UI
    Impala web UI
    Impala web UI
    Impala web UI

Delete instances

IMPORTANT

Delete the operator only after all the resources it manages have been deleted.

To delete the Impala cluster, run the command below:

$ ./adc delete -f impala-cluster.yaml

To delete the Impala operator, run the command below:

$ ./adc delete -f impala-operator.yaml
Found a mistake? Seleсt text and press Ctrl+Enter to report it