Recipes Docs

What is a Quine Recipe

For Quine, a recipe is a document that contains all of the information necessary for Quine to execute any batch or streaming data process. Quine recipes are written in yaml and built from components including:

  • Ingest Streams to read streaming data from sources and update graph data
  • Standing Queries to transform graph data, and to produce aggregates and other outputs
  • Cypher expressions to implement graph operations such as querying and updating data
  • Exploration UI configuration to customize the web user interface for the use-case that is the subject of the recipe

You can learn how to write a recipe in the recipes tutorial section in the getting started guide.

When to use a Recipe

Recipes enable you to quickly iterate in Quine and to share what you’ve built with others so they can reproduce, explore and expand upon a solution.

Consider writing a recipe:

  • When you are actively developing an event streaming solution.
  • To preserve a solution whenever a development milestone is achieved.
  • To configure the visual aspects of the Quine Exploration UI.
  • To store quick queries and sample queries that aide in graph analysis.
  • To share your solution with collaborators or the open source community.
  • When interacting with Quine support.

Recipes are NOT useful for production workloads. Once you are ready to put the recipe into production, you need to convert it to API calls because:

  • API calls allow for naming of ingest streams and standing queries. In recipes, they are named for you in the format of INGEST-# or STANDING-#.
  • By default, Quine launched with recipes creates a temporary persistent data store (in your tmp dir). Each subsequent launch of Quine w/ a recipe replaces this temporary data store.

Recipe Structure

A recipe is stored in a single YAML text file. The file contains a single object with the following attributes:

version: 1
title: recipe title
contributor: https://github.com/example-user
summary: ""
description: ""
ingestStreams: []
standingQueries: []
nodeAppearances: []
quickQueries: []
sampleQueries: []
statusQuery: null

Each configuration object is defined by its corresponding API entity in the REST API. Follow the links in the table below for details regarding each attribute.

Attribute Type Description
version Integer The recipe schema version, right now the only supported value is 1
title String Identifies the recipe
contributor String URL to social profile of the person or organization responsible for this recipe
summary String Brief information about this recipe
description Text Block Long form description about this recipe
ingestStreams Array of IngestStream objects Define how data is read from data sources, transformed, and loaded into the graph
standingQueries Array of StandingQuery objects Define both sub-graph patterns for Quine to match and subsequent output actions
nodeAppearances Array of NodeAppearance objects Customize node appearance in the exploration UI
quickQueries Array of QuickQuery objects Add queries to node context menus in the exploration UI
sampleQueries Array of SampleQuery objects Customize sample queries listed in the exploration UI
statusQuery A CypherQuery object OPTIONAL Cypher query that is executed and reported to the terminal window during execution

Differences Between Recipes and the REST API

Recipes package together Quine config, graph logic/structure, and exploration UI customizations that are run automatically when Quine starts. A recipe file can define the graph and exploration UI configuration directly however, when a recipe is run, it infers a Quine configuration. There are a couple of operational differences that you need to keep in mind when launching Quine along with a recipe.

  • API calls allow naming - When you create an ingest stream or a standing query using the API, you choose the name for the object in the URL. The corresponding recipe object uses standardized names in the form of INGEST-# and STANDING-#.

  • Persistent storage - Starting the Quine application jar without providing a configuration file, Quine creates a RocksDB-based persistent data store in the local directory. That persistor retains the previous graph state and is appended to each time Quine starts. Alternatively, when Quine launches a recipe, a temporary persistent data store is created in the system tmp directory. Each subsequent launch of the recipe will replace the data store, discarding the graph from the previous run. You can override the default temporary storage behavior with your configuration settings by using the --force-config command line flag.

Running a Recipe

Recipes are interpreted by quine.jar using command line arguments. For help on quine.jar command line arguments, use the following command:

❯ java -jar {{ quine }} -h
Quine universal program
Usage: quine [options]

  -W, --disable-web-service
                           disable Quine web service
  -p, --port <value>       web service port (default is 8080)
  -r, --recipe name, file, URL
                           follow the specified recipe
  -x, --recipe-value key=value
                           recipe parameter substitution
  --force-config           disable recipe configuration defaults
  --no-delete              disable deleting data file when process exits
  -h, --help
  -v, --version            print Quine program version

In order to launch Quine with a recipe, use -r, followed by either the short name of a sample recipe or a local YAML filename. Some recipes can expect input parameters. The parameter values are passed by using command line arguments with -x or --recipe-value.

Recipe Parameters

A recipe may contain parameters that are used to pass information to the recipe. To use a parameter in a recipe file, a value in a recipe must start with the $ character. The following example demonstrates a recipe with a parameter called in-file:

--8<-- "docs/recipes/assets/ingest.yaml"

Running the above example (without specifying the parameter value) causes an error:

❯ java -jar {{ quine }} -r ingest.yaml
Missing required parameter in-file; use --recipe-value in-file=

The error message indicates the command must be run with an additional command line argument that specifies the required parameter:

❯ java -jar {{ quine }} -r ingest.yaml --recipe-value in-file=my-file.txt

The parameter value is substituted into the recipe at runtime. Common examples for recipe parameters include file names, URLs, and host names.

Additional Command Line Arguments

When Quine is started without command line arguments, its default behavior is to start the web service on port 8080. The following options are available to change this behavior:

  • -W, --disable-web-service: Disables the web service
  • -p, --port: Specify the TCP port of the web service

Quine is configurable as described in Configuration. Normally when running a recipe, the configuration for store is overwritten with PersistenceAgentType.RocksDb and is configured to use a temporary file. This configuration is appropriate for most use cases of recipes and can be changed with the following command line arguments:

  • --force-config: Quine will use the store that is configured via Configuration (instead of overwriting it as described above)
  • --no-delete: Quine will not delete the DB file on exit and will print out that path to it

Note: The --force-config and --no-delete options are mutually exclusive (only one of the two is allowed at a time).

RocksDB may not function on some platforms and you need to use MapDB instead by starting Quine using parameters as follows:

java -Dquine.store.type=map-db -jar {{ quine }} --force-config

Recipe Repository

Complete recipes are useful as reference applications for use cases. Quine’s recipe repository highlights recipes shared by our community members. Additionally, you can view all of the shared recipes directory in the Quine Github repository.

Contribute

If you make a recipe and think others might find it useful, please consider contributing it back to the repository on Github so that others can use it. They could use it on their own data, or even just use it as a starting point for customization and remixing for other goals. To share with the community, open a pull request on Github.

1.8.1