Skip to content

Quine Enterprise Helm Chart Quickstart

Follow this link for the Full Helm chart.

See it as a guided tour

The Kubernetes Deployment Tour walks the same deployment interactively, built from plain manifests instead of a chart: baseline pod, Cassandra persistence, clustering, OIDC auth, metrics, and Kafka streaming, one capability at a time, with every config file shown.

Prerequisites

Kubernetes

A Kubernetes cluster is required to install Quine Enterprise with Helm.

Helm

Helm must be installed to use the charts. Please refer to Helm's documentation to get started.

Once Helm has been set up correctly, add the repo as follows:
helm repo add thatdot https://helm.thatdot.com

If you have previously added this repo, run helm repo update to retrieve the latest versions of the packages. You can then run helm search repo thatdot to see the charts.

Cassandra Compatible Database

Apache Cassandra or a Cassandra-compatible database such as ScyllaDB is required to use Quine Enterprise.

The database used with Quine Enterprise may be installed in the same Kubernetes cluster as Quine Enterprise, but this is not required. However, it is recommended that it be physically located as close to the Quine Enterprise installation as possible to reduce network latency, such as ensuring they are both located in the same AWS region or availability zone.

There are a variety of ways to install Apache Cassandra or ScyllaDB into Kubernetes. For production, use an operator, which manages scaling and upgrades:

Set the persistor's local datacenter to the name of your Cassandra datacenter with cassandra.localDatacenter in your Helm values. A plain Cassandra install uses datacenter1, which is also the default for this setting. With an operator, use the name you gave the datacenter.

License

Quine Enterprise is proprietary software and a license is required to use it.

To request an evaluation, schedule a demo, or purchase a license, contact thatDot or email sales@thatdot.com.

Once you receive your license key, you must store it in a Kubernetes secret and reference that secret in the Helm values using the licenseKeySecret field.

The same key is also your credential for the private image registry that distributes the Quine Enterprise image, registry.license.thatdot.com: the registry username is your client id, and the password is the license key itself. You can verify your credentials from a workstation with docker:

echo <LICENSE_KEY> | \
    docker login registry.license.thatdot.com \
    -u <CLIENT_ID> \
    --password-stdin

docker pull \
    registry.license.thatdot.com/thatdot/quine-enterprise:2.2.1

Installation

Helm Values

First create a values file called quine-enterprise-values.yaml with required helm configuration values (see values.yaml for a reference of available configuration and defaults):

# 3 Quine Enterprise hosts is a good minimum size for a cluster. Feel free to use more, up to the amount of available CPU and memory in your cluster.
hostCount: 3

# For simple installations, ClusterIP is a good choice. LoadBalancer can also be used to create an externally available endpoint for your cluster.
service:
  type: ClusterIP

# Settings for using Cassandra cluster as the persistor. REQUIRED.
cassandra:
  enabled: true
  endpoints: "<CASSANDRA_ADDRESS_1>,<CASSANDRA_ADDRESS_2>"
  plaintextAuth:
    enabled: true
    username: <CASSANDRA_USERNAME>
    password: <CASSANDRA_PASSWORD>
  shouldCreateKeyspace: true
  shouldCreateTables: true

# Evaluation resource settings, should increase to 3-32cpu and at least 15Gi memory for production
resources:
  requests:
    cpu: 2
    memory: 7Gi
  limits:
    cpu: 4
    memory: 7Gi

Create a Kubernetes secret containing your license key:

kubectl create secret generic quine-enterprise-license \
  --from-literal=license-key="YOUR_LICENSE_KEY"

Then reference this secret in your Helm values:

licenseKeySecret:
  name: quine-enterprise-license
  key: license-key

The Quine Enterprise image is pulled from thatDot's private registry, authenticated with the same license key. Create a kubernetes.io/dockerconfigjson secret holding that credential:

kubectl create secret docker-registry thatdot-registry \
  --docker-server=registry.license.thatdot.com \
  --docker-username=<CLIENT_ID> \
  --docker-password=<LICENSE_KEY>

The resulting secret is a docker login stored as data. If you manage manifests directly or use a secrets operator, the equivalent shape is:

apiVersion: v1
kind: Secret
metadata:
  name: thatdot-registry
type: kubernetes.io/dockerconfigjson
stringData:
  .dockerconfigjson: |
    {"auths": {"registry.license.thatdot.com": {
      "username": "<CLIENT_ID>",
      "password": "<LICENSE_KEY>",
      "auth": "<base64 of CLIENT_ID:LICENSE_KEY>"}}}

Reference the secret and the image in your Helm values:

image:
  repository: registry.license.thatdot.com/thatdot/quine-enterprise
  pullPolicy: IfNotPresent
  tag: "2.2.1"

imagePullSecrets:
  - name: thatdot-registry

The chart renders these into the Deployment's pod template, where the secret is referenced by name for image pulls.

If you deploy without Helm, the same wiring in a Deployment manifest looks like:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      imagePullSecrets:
        - name: thatdot-registry
      containers:
        - name: quine-enterprise
          image: registry.license.thatdot.com/thatdot/quine-enterprise:2.2.1

Install the cluster with Helm using the command (Don't forget to first helm repo add the thatdot Helm repo in the prerequisites section):

helm install --values=quine-enterprise-values.yaml \
    my-quine-enterprise thatdot/quine-enterprise

To inspect a Quine Enterprise pod after the install, open a shell with kubectl exec or attach a debug container. The Quine Enterprise container image has bash, the Java runtime and basic tools such as ls, cat and grep, but not curl, ps or a package manager. Work in a running container lists what is there and what to do about the rest.

Uninstall the cluster:
helm uninstall my-quine-enterprise

Cluster Scaling

In order to change the number of active cluster members of a Quine Enterprise, it must be scaled down to zero hosts and then back up. This is because Quine Enterprise uses a static sharding scheme to determine which nodes each member will manage. However, all persisted data and standing queries can remain between different sized clusters, so it is possible to effectively pause a workload, scale up or down, and then resume the workload.

Recommended

Delete all per-member ingests before starting to scale down. Use the Delete Ingest Stream: DELETE /api/v2/graph/{graphName}/ingests/{ingestName} API endpoint.

Note

Per-member ingests are each defined on a single member, so it is important to consider the topology of which member is ingesting from which stream. Deleting and recreating ingests is preferable when each host should be ingesting from a partitioned Kafka cluster or similar source. Cluster-wide ingest streams need no change, because they are defined for the whole cluster and the cluster places their partitions on the hosts it has after the restart.

Use Helm to scale the cluster down to 0 active hosts:

helm upgrade --wait --reuse-values \
    --set hostCount=0 \
    my-quine-enterprise thatdot/quine-enterprise

Once the operation has completed, update the quine-enterprise-values.yaml file with the desired cluster size and use Helm to deploy the changes:

helm upgrade --wait --values=quine-enterprise-values.yaml \
    my-quine-enterprise thatdot/quine-enterprise

Recommended

Re-create all required per-member ingests. Use the Create Ingest Stream: POST /api/v2/graph/{graphName}/ingests API endpoint.

What scaling changes in the deployment itself (member count, hot spares, the seed Service members find each other through) is walked in the Kubernetes Deployment Tour.

Upgrade/Downgrade Version

Some version transitions require a full cluster restart, scaling the cluster down to 0 members and back up, in a similar procedure to the Cluster Scaling section:

  • Upgrading from or downgrading to a version prior to 1.8.3

  • Upgrading from 2.0.x (or earlier) to 2.1.0 or later, due to a change in the cluster's internal message format (Upgrading to 2.1.0)

  • Upgrading from 2.1.0 (or earlier) to 2.1.1 or later, due to a change in how cluster-wide operations are relayed and acknowledged between members; a rolling, member-by-member upgrade is not supported (Upgrading to 2.1.1)

  • Upgrading from 2.1.x (or earlier) to 2.2.0 or later, due to a change in the persistence format that every member checks when it starts; a rolling, member-by-member upgrade is not supported, and the upgrade is one-way, so take a backup first (Upgrading to 2.2.0)

helm upgrade --wait --reuse-values \
    --set hostCount=0 \
    my-quine-enterprise thatdot/quine-enterprise

For other upgrades between versions 1.8.3 or newer, you can run helm upgrade as below without scaling the cluster down to 0 members. Note that all work will be saved, but ingests, standing query outputs, and ad-hoc queries will be paused during the transition period.

Update the quine-enterprise-values.yaml file with the desired Quine Enterprise docker image (example: updating to version 2.2.1):

# Override image repository and tag to change the docker image used to run thatDot Quine Enterprise
image:
  repository: registry.license.thatdot.com/thatdot/quine-enterprise
  # Overrides the image tag whose default is the chart appVersion.
  tag: "2.2.1"

Keep image.tag set. Without it, the chart runs the image tagged with the chart's appVersion, which can be older than the version you run (helm search repo thatdot/quine-enterprise shows it). Run helm repo update to fetch the latest charts before you upgrade.

Then deploy the update with Helm:

helm upgrade --wait --values=quine-enterprise-values.yaml \
    my-quine-enterprise thatdot/quine-enterprise

You may also specify a chart version explicitly with Helm (helm search repo thatdot/quine-enterprise --versions lists each chart version with its app version):

helm upgrade --wait --values=quine-enterprise-values.yaml \
    --version <CHART_VERSION> \
    my-quine-enterprise thatdot/quine-enterprise

It is important to always use the latest helm chart version available for the particular appVersion of Quine Enterprise you want to deploy.

Metrics and Monitoring

Quine Enterprise exposes a large number of JVM and application metrics via the Dropwizard Metrics Library.

The recommended monitoring stack consists of influxDBv1 to record metrics and grafana to graph and visualize them.

See helm-charts/extras/monitoring-stack for an example to quickly bootstrap a thatDot environment with an InfluxDb and a Grafana instance.

# Metrics Configuration:
metrics:
  influx:
    enabled: true
    database: metrics
    scheme: http
    # Set "host" and "port" to the reachable address of the influx server
    host: "<INFLUXDB_HOST_ADDRESS>"
    port: 8086

Note

Quine Enterprise also supports emitting metrics to Prometheus. A guide on how to set this up with Helm is here.