Standard In

Quine Enterprise fully supports reading from Standard In. Together with writing Standing Queries to Standard Out, Quine Enterprise is a powerful tool for any command-line data processing task.

The following is a simple Recipe that ingests each line of input from Standard In as a node in the graph. It also uses a Standing Query to write each node's line to Standard Out:

version: 2
title: Pipe
contributor: https://github.com/landon9720
summary: Ingests from Standard Input and writes to Standard Output
description: |-
  Ingests each line from Standard Input as a graph node with property "line".
  Each ingested line is written to Standard Output.

ingestStreams:
  - name: stdin-ingest
    source:
      type: StdInput
      format:
        type: Line
      characterEncoding: UTF-8
    query: |-
      MATCH (n)
      WHERE id(n) = idFrom($that)
      SET n.line = $that

standingQueries:
  - name: stdout-output
    pattern:
      type: Cypher
      query: |-
        MATCH (n)
        WHERE n.line IS NOT NULL
        RETURN DISTINCT id(n) AS id
      mode: DISTINCT_ID
    outputs:
      - name: output-1
        preEnrichmentTransformation:
          type: InlineData
        resultEnrichment:
          query: |-
            MATCH (n)
            WHERE id(n) = $that.id
            RETURN n.line AS line
          parameter: that
        destinations:
          - type: StandardOut

nodeAppearances: []
quickQueries: []
sampleQueries: []

To run this Recipe, save it as pipe.yaml, then pipe data from another program into Quine. This example uses the Unix find program as a data source:

❯ find /dev | java -jar quine-2.2.1.jar -r pipe.yaml
Graph is ready
Running Standing Query stdout-output
Running Ingest Stream stdin-ingest
{"line":"/dev/oslog_stream"}
{"line":"/dev/console"}
{"line":"/dev/oslog"}
{"line":"/dev/stdin"}
...
stdin-ingest status is completed and ingested 350
Quine web server available at http://127.0.0.1:8080
Default API version: v2

Quine keeps running after find finishes, so you can explore the ingested nodes in the web UI. Press Control-C to stop it.

Cluster-Wide Ingest

Standard input belongs to one process's console. A cluster-wide ingest from StdInput is therefore refused unless you supply pinnedTo naming the host whose console you mean, at which point it runs as a single pinned partition that is never relocated.

This is supported, but it is rarely what you want: standard input is a development and demonstration source, and pinning it gains you a cluster-managed definition rather than any distribution. See Pinning an ingest to one host.