Programming and other Random Bits™

Extending Cypher or How I Learnt To Stop Worrying and Love Cypher

Oct 3, 2014
3 minute read

A while back I had asked if I there was an easy way to extend Cypher. Cypher is an excellent declarative way for querying a Neo4j database but the functions in it leave me wanting for me. I had this twitter conversation:

Challenge Accepted

Cypher is built using Scala. As I had only basic knowledge of the language, creating new functionality would have been challenging for me. So I reached for my current favourite language, Clojure. It runs on JVM. So it should have been possible to integrate that with Scala and cypher. Once I was in clojure, I realised that I can really leverage the power of dynamic programming to attach a rocket booster to cypher. By that I mean, I could very easily add really complex functions to cypher easily. And the whole contraption would be stuck together with programming equivalent of duct tape.

So here we go.

A great use case for the extension is adding Date/DateTime support which Neo4j lacks natively. I don’t think its a deal breaker but some people have strong views aroung this, as voiced at the end of Jim Weber’s excellent talk at Skills Matter called Impossible is Nothing. A workaround is to store that data as Unix Epochs, which are just long values. But it means if you want to add a date to a node, you first have to convert it to a long externally and then run your cypher query. So there 2 steps when there should only be 1.

This is my Clojure code which added two date functions to cypher: str-to-date, date-to-str.

(ns cypher-ext.core
  (:require [clj-time.format :as tf]
            [clj-time.coerce :as tc]))


(defmulti func (fn [n args] n))

(defmethod func :str-to-date
  [_ [x fmt]]
  (let [fmt  (or fmt "yyyy-MM-dd")
        ptn  (tf/formatter fmt)
        d    (tf/parse ptn x)]
    (tc/to-long d)))


(defmethod func :date-to-str
  [_ [x fmt]]
  (let [d    (tc/from-long x)
        fmt  (or fmt "yyyy-MM-dd")
        ptn  (tf/formatter fmt)]
    (tf/unparse ptn d)))

(defn call
  [args]
  (func (keyword (first args))
        (rest args)))

The code for cypher-ext is also on github.

With my changes, you can do the following:

CREATE (n { date : specialfunc(["str-to-date", "2013-01-01"]) });
+-------------------+
| No data returned. |
+-------------------+
Nodes created: 1
Properties set: 1
MATCH (n) RETURN n;
+-----------------------------+
| n                           |
+-----------------------------+
| Node[0]{date:1356998400000} |
+-----------------------------+
1 row
MATCH (n) RETURN specialfunc(["date-to-str", n.date]);
+--------------+
| date         |
+--------------+
| "2013-01-01" |
+--------------+
1 rows

So the date is parsed to a long by str-to-date function and a long is converted to date string by date-to-str function.

Obviously now to add more functions to cypher is really easy. All I need to do is add another defmethod in my Clojure code.

The changes I had to make to Cypher in Neo4j codebase are here. The meat of the change is in the compute method of SpecialFunc class:

package org.neo4j.cypher.internal.compiler.v2_1.commands.expressions

import org.neo4j.cypher.internal.compiler.v2_1._
import pipes.QueryState
import symbols._
import org.neo4j.cypher.internal.helpers.CollectionSupport

import scala.collection.JavaConverters._

import clojure.java.api.Clojure

case class SpecialFunc(inner: Expression)
  extends NullInNullOutExpression(inner)
  with CollectionSupport {

  def compute(value: Any, m: ExecutionContext)(implicit state: QueryState) = {
    val require = Clojure.`var`("clojure.core", "require");
    require.invoke(Clojure.read("cypher-ext.core"))
    val fn = Clojure.`var`("cypher-ext.core", "call")
    fn.invoke(makeTraversable(value).toSeq.asJava)
  }

  def rewrite(f: (Expression) => Expression) = f(SpecialFunc(inner.rewrite(f)))

  def arguments = Seq(inner)

  def calculateType(symbols: SymbolTable): CypherType = CTAny

  def symbolTableDependencies = inner.symbolTableDependencies
}

Everything is Awesome!!