What

KiokuJS is a persistence layer for Joose. Its a freestyle port of KiokuDB which does the same thing in the Moose world.

KiokuJS provides transparent storing/retrieving for instances of Joose classes. Native data structures (Object/Array/etc) can be stored as well. “Transparent” means that class can be stored without any modifications to its declaration (see also But).

KiokuJS is cross-platform – it runs equally well in browsers and NodeJS. KiokuJS is non-blocking – all interactions with DB are asynchronous.

Why

Remember the last Web2.0 project you worked on. Quite probably it was looking as:

Note, that you need to define the serialization procedure 4(!) times. Even if you use some kind of client-side helpers of modern frameworks, you still need to (manually) map your client-side JavaScript data structures to the storage layer.

Meanwhile, nowadays its obvious, that JavaScript has evolved from client-side animation libraries into full-stack platform. There are no reasons why your applications can’t be fully written in JavaScript and benefit from the homogeneous environment both on client and server.

So the goals of KiokuJS are:

• eliminate all the serialization boilerplate from the application
• use a single model definition both on client and server
• encapsulate all the storage-layer details in high-level interface

Example

// class declaration

Class('Person', {

    has : {
        self    : null,

        name    : null,

        spouse  : null
    },

    methods : {

        initialize : function () {
            this.self = this // circular ref
        }
    }
})

// arbitrary data structure

var Homer = new Person({
    name    : "Homer"
})

var Marge = new Person({
    name    : "Marge"
})

Homer.spouse = Marge
Marge.spouse = Homer

// handler setup

var handle = new KiokuJS.Backend.CouchDB({
    dbURL   : 'http://local:5984/someDB'
})

// storing

var scope = handle.newScope()

scope.store(Homer, Marge).andThen(function (homerID, margeID) {
    ...
})

// retrieving

var scope = handle.newScope()

scope.lookUp(homerID, margeID).andThen(function (Homer2, Marge2) {

    // Homer2.self   === Homer2
    // Homer2.spouse  === Marge2

    // Marge2.self   === Marge2
    // Marge2.spouse  === Homer2
})

How

From the KiokuJS viewpoint, the data structures represents a cyclic, directed graph:Data graph is then serialized into JSON structures like:

{
    "className": "Person",
    "ID": "2D9E24EE-89BA-A361-B0E6-2D8A6C1226F6",
    "isRoot": true,
    "data": {
        "self": {
            "$ref": "2D9E24EE-89BA-A361-B0E6-2D8A6C1226F6"
        },
        "name": "Homer",
        "spouse": {
            "$ref": "0CFB5A77-8B58-BC7F-9B7D-DBDD78A29AC2"
        }
    },
    "$entry": true
}

{
    "className": "Person",
    "ID": "0CFB5A77-8B58-BC7F-9B7D-DBDD78A29AC2",
    "isRoot": true,
    "data": {
        "self": {
            "$ref": "0CFB5A77-8B58-BC7F-9B7D-DBDD78A29AC2"
        },
        "name": "Marge",
        "spouse": {
            "$ref": "2D9E24EE-89BA-A361-B0E6-2D8A6C1226F6"
        }
    },
    "$entry": true
}

There are a number of options, defining the serialization details (implemented as attribute traits). When required, its also possible to provide a custom serialization format for specific class.

Its worth to note, that all asynchronous interfaces in Kioku are implemented using the continuation-passing style extension for Joose, which provides a precise control over the asynchronous methods. The previous storing example can be elaborated like this:

// storing

UI.maskScreen("Please wait")

scope.store(Homer, Marge).then(function (homerID, margeID) {

    alert('Objects stored successfully')

    this.CONTINUE()

}).CATCH(function (e) {

    alert('There were an exception [' + e + '] during storing operation')

    this.CONTINUE()

}).FINALLY(function () {

    UI.unMaskScreen()

}).now()

But

Persistence is mostly transparent, but there are some limitations. Generally any closures-related magic won’t be recognized, simply because there are no according introspection capabilities in the language. For example, if you have a function, tied to a scope with some closures, as the value of the attribute:

var closure = { foo : 'bar' }

homer.magic = function () { return closure.foo }

then KiokuJS will store the function declaration only, not the scope it was defined in. So after retrieving the instance from the backend and running “homer.magic()” there will be an exception like [“closure” is not defined].

Another limitation is related to searching. Searching is constrained to backends capabilities only.

Backends

Kioku is backend-agnostic, but currently mostly target NoSQL (key/value) backends because of their simplicity, native JSON support and HTTP APIs, which allows to directly access the DB from the browsers.  With some additional efforts can be implemented a relational backend, either static (when classes definitions will be generated from the relational schema) or dynamic (when the relational schema will be adjusted to the classes definition).

For the development and testing there is a “Hash” backend. For the production – CouchDB backend.

Test suite of Kioku is organized that way, that makes it possible to run for each backend.

There is also a special backend (actually a role for backend) – KiokuJS.Backend.Batch, intended for usage in browsers. It allows to combine the individual requests for separate documents into batches, saving a lot of HTTP requests.

Writing new backends is easy – they only need to define 4 methods.

Status

KiokuJS is at 0.01 version. This means its “stable enough”, has the extensive test suite, but is mostly intended for early adopters, which are welcome to Joose IRC channel with any questions. Currently there is no documentation.

To install Kioku, run

> npm install kiokujs

You may also want to install CouchDB backend with:

> npm install kiokujs-backend-couchdb

After that this mini demo app should just work from NodeJS, supposing you have correct value for NODE_PATH environment variable (see items 3.x in Joose installation).

Roadmap

• Add more backends – Riak and MongoDB are 1st candidacies.
• Try to implement backends for graph databases (very promising).
• Syncler (draft name) – real-time backend, which use the optimistic replication to synchronize the state of replicas between clients.