Archive for February, 2011
JooseX.CPS Tutorial – Part II
This is the 2nd part of the JooseX.CPS tutorial. Before reading it, make sure you’ve groked the 1st part. In the 1st part, we were managing the chains of anonymous functions. Such task can quickly become cumbersome, because the functions are, well, anonymous. Lets see how we can bring this concept to the higher level – out goal is to integrate the continuation passing style right into Joose classes.
Phase 1 – Synchronous
As the show case, lets use the following simple task. You have a class – wrapper around the text file, which provides simple convenience methods, like “read”, “write” etc. The wrapper is being used by Manager, which has one method – `processFile`. This method takes the source file name, reads the file and then saves the upper-cased content of the source file to another file, appending `-res` to the filename. Skim the code below:
require('task-joose-nodejs')
var fs = require('fs')
Class('TextFile', {
has : {
fileName : { required : true },
data : null
},
methods : {
asUpperCase : function () {
return this.data.toUpperCase()
},
read : function () {
return this.data = fs.readFileSync(this.fileName, 'utf8')
},
write : function (text) {
this.data = text
this.save()
},
save : function () {
this.saveAs(this.fileName)
},
saveAs : function (fileName) {
fs.writeFileSync(fileName, this.data)
}
}
})
Class('Manager', {
my : {
methods : {
processFile : function (fileName) {
var sourceFile = new TextFile({ fileName : fileName })
sourceFile.read()
var targetFile = new TextFile({
fileName : fileName.replace(/\.(\w\w\w)$/, '-res.$1')
})
targetFile.write(sourceFile.asUpperCase())
}
}
}
})
try {
Manager.processFile('source.txt')
} catch (e) {
console.log('Exception caught: %s', e)
}
Everything operates synchronously, any exceptions can be caught with `try/catch` blocks and all is fine. Now save the code in some js file and run it in node. You should see this:
Exception caught: Error: ENOENT, No such file or directory 'source.txt'
Of course, there is no source file. Create the ‘source.txt’ in the same directory as the code file and run it again. This time everything should work and the resulting file ‘source-res.txt’ , containing the uppercased content of the source file should appear.
Now, remove the write permission on the result file and run the script again. You’ll see
Exception caught: Error: EACCES, Permission denied 'source-res.txt'
Note, how we were able to caught the both exceptions from the “processFile” method (1st one thrown from “read” and 2nd – from “saveAs”) without any additional gymnastic.
Ok, lets re-write the code in asynchronous way, keeping the functionality the same.
Phase 2 – Asynchronous
So here’s how it will looks like (note though that we won’t catch “usual” exceptions – only those reported by NodeJS methods):
require('task-joose-nodejs')
var fs = require('fs')
Class('TextFile', {
has : {
fileName : { required : true },
data : null
},
methods : {
asUpperCase : function () {
return this.data.toUpperCase()
},
read : function (callback, errback) {
var me = this
fs.readFile(this.fileName, 'utf8', function (err, data) {
if (err) {
errback(err)
return
}
me.data = data
callback(data)
})
},
write : function (text, callback, errback) {
this.data = text
this.save(callback, errback)
},
save : function (callback, errback) {
this.saveAs(this.fileName, callback, errback)
},
saveAs : function (fileName, callback, errback) {
fs.writeFile(fileName, this.data, function (err) {
if (err)
errback(err)
else
callback()
})
}
}
})
Class('Manager', {
my : {
methods : {
processFile : function (fileName, callback, errback) {
var sourceFile = new TextFile({ fileName : fileName })
sourceFile.read(function () {
var targetFile = new TextFile({
fileName : fileName.replace(/\.(\w\w\w)$/, '-res.$1')
})
targetFile.write(sourceFile.asUpperCase(), callback, errback)
}, errback)
}
}
}
})
Manager.processFile('source.txt', function () {}, function (e) {
console.log('Exception caught: %s', e)
})
I hear you said “Piece of cake?”. How about such requirements then – “write the content to the target file, and only if the operation completed successfully, remove the source file. If there was an error during `processFile` method, append it a log file. And process the array of filenames in no more than 5 parallel threads”. If you say “thats 2 pieces of cake” on this – then my congratulations, you are a callback’s overlord :). Otherwise, read further.
CPS conversion
So lets convert the classes above to the CPS. First thing to note is – we have “synchronous” (usual) and “asynchronous” methods.
Synchronous methods returns the result immediately, with the `return` statement. In the same way they throw the exceptions immediately, with `throw`.
“Asynchronous” methods returns the result, by passing it to the callback, and they throws the exceptions by passing them to errbacks. The call to callback/errback may happen after some arbitrary delay.
Synchronous and asynchronous methods
Lets split the methods into 2 groups, by their synchronicity:
require('task-joose-nodejs')
var fs = require('fs')
Class('TextFile', {
trait : JooseX.CPS,
...
// synchronous methods goes here
methods : {
asUpperCase : function () {
return this.data.toUpperCase()
}
},
// asynchronous methods goes here
continued : {
// right here
methods : {
read : function (callback, errback) {
...
},
write : function (text, callback, errback) {
...
},
save : function (callback, errback) {
...
},
saveAs : function (fileName, callback, errback) {
...
}
}
}
})
Class('Manager', {
my : {
trait : JooseX.CPS,
continued : {
methods : {
processFile : function (fileName, callback, errback) {
...
}
}
}
}
})
The new class builder `continued` is provided by the trait `JooseX.CPS`. It can contain the following properties: “methods/before/after/override” which has the same semantic as usual builders. All asynchronous methods (or method modifiers) goes into that section.
Magic continuation instance
Next thing to note – every asynchronous method has the “callback” and “errback” arguments. Lets clean the definition and move them to the special `this.CONT` symbol instead, which we’ll made available inside of each asynchronous method. Yes, its the continuation instance from the part 1. This magic symbol is something like `this.SUPER` we are all used to, but its not a function – its an object with own methods and properties.
myAsynchronousMethod : function (p1) {
// this.SUPER calls the implementation of the current method
// from the superclass
this.SUPER
// this.CONT refers to the current continuation instance
this.CONT
}
Important notes: Inside of each asynchronous method there is a special `this.CONT` symbol available, which refers to the “current” instance of JooseX.CPS.Continuation. When the method starts, the continuation is empty – it doesn’t contain any tasks.
Then, when inside of any asynchronous method, you call any other asynchronous method – the call is converted into the task for current continuation (with its TRY method). For example:
// asynchronous methods section
continued : {
methods : {
save : function () {
// call to another asynchronous method
this.saveAs(this.fileName).now()
},
saveAs : function () {
//saveAs definition
...
}
}
}
// the above gets translated to
// asynchronous methods section
continued : {
methods : {
save : function () {
this.CONT.TRY(function () {
//saveAs definition
...
}, this, [ this.fileName ]).now()
},
saveAs : function () {
...
}
}
}
Note, that call to TRY contains the object, calling the method, as 2nd argument and arguments for method – as 3rd.
Asynchronous methods are prohibited to return any values with `return`. Instead they should return values using the call to `this.CONT.CONTINUE()` method (or throw the exceptions with `this.CONT.THROW()`). Or they can instead launch the nested continuation – in this case it will be responsible for returning values. In the “synchronous” meaning, asynchronous methods will return the continuation instance they are attached to.
Also, as you remember, continuation tasks won’t launch immediately – they needs to be activated with `now`.
Phase 3 – CPS
The full version of the example above will looks like:
require('task-joose-nodejs')
var fs = require('fs')
Class('TextFile', {
trait : JooseX.CPS,
has : {
fileName : { required : true },
data : null
},
methods : {
asUpperCase : function () {
return this.data.toUpperCase()
}
},
continued : {
methods : {
read : function () {
var me = this
var callback = this.CONT.getCONTINUE()
var errback = this.CONT.getTHROW()
fs.readFile(this.fileName, 'utf8', function (err, data) {
if (err) {
errback(err)
return
}
me.data = data
callback(data)
})
},
write : function (text) {
this.data = text
this.save().now()
},
save : function () {
this.saveAs(this.fileName).now()
},
saveAs : function (fileName) {
var callback = this.CONT.getCONTINUE()
var errback = this.CONT.getTHROW()
fs.writeFile(fileName, this.data, function (err) {
if (err)
errback(err)
else
callback()
})
}
}
}
})
Class('Manager', {
my : {
trait : JooseX.CPS,
continued : {
methods : {
processFile : function (fileName) {
var sourceFile = new TextFile({ fileName : fileName })
sourceFile.read().THEN(function () {
var targetFile = new TextFile({
fileName : fileName.replace(/\.(\w\w\w)$/, '-res.$1')
})
targetFile.write(sourceFile.asUpperCase()).now()
}).now()
}
}
}
}
})
Manager.processFile('source.txt').CATCH(function (e) {
console.log('Exception caught: %s', e)
}).now()
Lets examine what have changed:
1) In the methods which integrates with “raw” asynchronous methods (“read” and “saveAs”) we’ve just removed the callbacks and errbacks from arguments and get them from the current continuation. Its a small winning as the outer interface of the methods becomes a bit simpler.
2) Methods which calls only other CPS methods (“write” and “save”) are now much cleaner – they don’t contain any callbacks at all.
3) The biggest gain is in the “processFile” method (which also uses only CPS methods). It doesn’t contain any callbacks/errbacks now! And all the errors from it get caught correctly (try to launch the code w/o source file and w/o write permissions to result file).
4) We now catch the exceptions from `Manager.processFile` in much more straightforward syntax.
For consolidation, lets see how the `processFile` method will look like on the JooseX.CPS.Continuation level:
processFile : function (fileName) {
var sourceFile = new TextFile({ fileName : fileName })
this.CONT.TRY(function () {
/* `read` method definition here */,
}, sourceFile, [])
this.CONT.THEN(function () {
var targetFile = new TextFile({
fileName : fileName.replace(/\.(\w\w\w)$/, '-res.$1')
})
this.CONT.TRY(function () {
/* `write` definition here */
}, targetFile, [ sourceFile.asUpperCase() ])
this.CONT.now()
})
this.CONT.now()
}
Some features
The fact that methods don’t launch immediately provides couple of interesting features.
First of all, when you call several “continued” methods in a raw and then activate the continuation – the methods will be executed sequentially. Compare asynchronous and synchronous versions:
asyncMethod : function () {
this.anotherAsyncMethod1()
this.anotherAsyncMethod2()
this.anotherAsyncMethod3()
this.anotherAsyncMethod4().now()
}
syncMethod : function () {
this.anotherSyncMethod1()
this.anotherSyncMethod2()
this.anotherSyncMethod3()
this.anotherSyncMethod4()
}
Such pattern works when methods do not depend on the returning values from the earlier methods.
Another feature is that the call to asynchronous method can be used as some sort of “promise” (though JooseX.CPS was not designed to provide this functionality). You can pass the current continuation to some other method which will add some generic step to it. Using JooseX.CPS this way wasn’t intended but its possible.
Some sugar
A JooseX.CPS trait will add a JooseX.CPS.ControlFlow role to your class, which allow you to omit the `this.CONT` when calling various methods of current continuation. So you can call ‘this.CONTINUE()` instead of `this.CONT.CONTINUE()`, `this.getCONTINUE()` etc.
Conclusion
Hopefully this post illustrates the ideas from this one. Note how the meta-layer absorbed all the low-level details and provided a much cleaner abstractions.
JooseX.CPS significantly lowers the overhead of creation and managing the asynchronous interface of your classes. No excuses to write blocking code anymore :)
P.S.
Curious how the “2 pieces of cake” example will looks like in CPS? Here it is (or here download from here) . For brevity we avoided synchronization problems when writing to log file and just output to console. Try to run the code, already having one result file, w/o write permission.
require('task-joose-nodejs')
var fs = require('fs')
Class('TextFile', {
trait : JooseX.CPS,
has : {
fileName : { required : true },
data : null
},
methods : {
asUpperCase : function () {
return this.data.toUpperCase()
}
},
continued : {
methods : {
read : function () {
var me = this
var callback = this.CONT.getCONTINUE()
var errback = this.CONT.getTHROW()
fs.readFile(this.fileName, 'utf8', function (err, data) {
if (err) {
errback(err)
return
}
me.data = data
callback(data)
})
},
write : function (text) {
this.data = text
this.save().now()
},
save : function () {
this.saveAs(this.fileName).now()
},
saveAs : function (fileName) {
var callback = this.CONT.getCONTINUE()
var errback = this.CONT.getTHROW()
fs.writeFile(fileName, this.data, function (err) {
if (err)
errback(err)
else
callback()
})
},
remove : function () {
var callback = this.CONT.getCONTINUE()
var errback = this.CONT.getTHROW()
fs.unlink(this.fileName, function (err) {
if (err)
errback(err)
else
callback()
})
}
}
}
})
Class('Manager', {
my : {
trait : JooseX.CPS,
continued : {
methods : {
log : function (text) {
console.log(text)
this.CONTINUE()
},
processArray : function (fileNames) {
Joose.A.each(fileNames, function (fileName) {
this.AND(function () {
this.processFile(fileName).except(function (e) {
this.log("Error processing " + fileName + ": " + e).now()
}).now()
})
}, this)
this.ANDMAX(5).now()
},
processFile : function (fileName) {
var sourceFile = new TextFile({ fileName : fileName })
sourceFile.read().andThen(function () {
var targetFile = new TextFile({
fileName : fileName.replace(/\.(\w\w\w)$/, '-res.$1')
})
targetFile.write(sourceFile.asUpperCase()).andThen(function () {
sourceFile.remove().now()
})
})
}
}
// eof methods
}
// eof continued
}
})
var fileNames = [
'source1.txt',
'source2.txt',
'source3.txt',
'source4.txt',
'source5.txt',
'source6.txt',
'source7.txt'
]
Manager.processArray(fileNames, "log.txt").now()
JooseX.CPS Tutorial – Part I
So, after the recent crack of our server, the post about the JooseX.CPS was lost. It was poorly written anyway, so may be its even better that the server was hacked – because I’m writing a new post. JooseX.CPS is quite important extension, and its being extensively used in many other Joose-based projects, like KiokuDB, Syncler and Symbie. The new post is written in form of tutorial and contains a runnable code. At the end of this post you should grok the continuation passing style :)
Setup
To follow this tutorial you will need:
The latter can be installed with `npm install task-joose-nodejs`.
Part I – Meet The Continuation
The tutorial will be split into 2 parts. In this part we’ll examine the low-level JooseX.CPS.Continuation class and its features. The second part will describe a JooseX.CPS trait itself, which provides a CPS sugar layer on top of JooseX.CPS.Continuation for you Joose classes.
Just try it, aka “Hello world”
Before we’ll dig in, lets see how the “hello world” will looks like in CPS world:
require('task-joose-nodejs')
TRY(function () {
console.log('Hello')
this.CONT.CONTINUE()
}).DELAY(1000).THEN(function () {
console.log('world!')
}).now()
Try to launch this in your Node. You should see the following with 1s delay between the words:
nickolay@desktop:~/Documents/JooseBlog/CPS-Tutorial$ node hello_world.js Hello world!
Compare with
console.log('Hello')
console.log('world!')
You see the idea? With some additional boilerplate between them, the synchronous the statements work asynchronously, thats it. And as we’ll move forward in this tutorial, the amount of boilerplate will reduce, really.
Continuation anatomy
In the first approximation, continuation is an array of tasks, also containing the “catch” and “finally” functions:
CONT = {
tasks : [
{
func : function doThis() { ... },
scope : {},
args : [ ... ]
},
{
func : function doThat() { ... },
...
},
...
],
catchFunc : function () { ... },
finallyFunc : function () { ... },
}
The “task” is simply a function (mandatory), plus a scope for its execution and an array of arguments (both optional). Such continuation roughly corresponds to the following “synchronous” code:
try {
doThis()
doThat()
} catch (e) {
cathcFunc(e)
} finally {
finallyFunc()
}
Once more – continuation is a single “try/catch” block, containing an array of tasks. You can add tasks to the continuation with the `TRY(func, scope, args)` method. Adding tasks will not launch them immediately – you need to “activate” the continuation manually with its method `now`.
After activation, continuation will launch the first task. It will embed a new, fresh continuation instance into the scope of task, as the `this.CONT`. To pass the control flow from the 1st task to the next one, you need to call the `CONTINUE` method of the embedded continuation. Any parameters, passed to `CONTINUE` will be available as the arguments of the next task. Lets re-write the “Hello world” example, “by hands”:
require('task-joose-nodejs')
var CONT = new JooseX.CPS.Continuation()
var scope = { toString : function () { return 'My scope' } }
CONT.TRY(function () {
console.log("Current scope 1: %s", this)
console.log('Hello')
// a new, fresh instance of JooseX.CPS.Continuation has been embedded:
// this.CONT != CONT
this.CONT.CONTINUE('world', '!')
}, scope)
CONT.TRY(function (str1, str2) {
console.log("Current scope 2: %s", this)
console.log(str1, str2)
})
CONT.now()
Some sugar: there is `THEN` synonym for the `TRY` method, which sounds more naturally for the 2nd and further tasks. Note, that if you have not provided a scope for the task, it will be taken from the previous task.
The call to `CONTINUE` doesn’t have to be done synchronously, you can delay it. To do that, capture it to the closure with the `getCONTINUE` method. Lets add the 1s pause between the outputs, “manually” (w/o the `DELAY()`):
require('task-joose-nodejs')
var CONT = new JooseX.CPS.Continuation()
var scope = { toString : function () { return 'My scope' } }
CONT.TRY(function () {
console.log("Current scope 1: %s", this)
console.log('Hello')
// this.CONT != CONT - its a new, fresh instance of JooseX.CPS.Continuation
// passing the parameters further
this.CONT.CONTINUE('world', '!')
}, scope)
CONT.THEN(function (str1, str2) {
var CONTINUE = this.CONT.getCONTINUE()
setTimeout(function () {
CONTINUE(str1, str2)
}, 1000)
})
CONT.THEN(function (str1, str2) {
console.log("Current scope 2: %s", this)
console.log(str1, str2)
})
CONT.now()
Hey don’t scan, read the sources :)
Instead of manual instantiation of JooseX.CPS you can use global `TRY` helper which just creates a new instance and delegates to its `TRY` method. And of course, the `TRY/THEN` method return the continuation itself, so the calls can be chained. There is also a `DELAY` method and so we comes to the initial “Hello world”:
require('task-joose-nodejs')
TRY(function () {
console.log('Hello')
this.CONT.CONTINUE()
}).DELAY(1000).THEN(function () {
console.log('world!')
}).now()
Nesting
Instead of calling the `CONTINUE` of the inner continuation, you can add a nested continuation – using `TRY` as usual. Keep in mind, that you need to activate it with `now`. When the last task of the nested continuation will call `CONTINUE` the control flow will continue on the outer level:
require('task-joose-nodejs')
TRY(function () {
console.log('Hello')
this.CONT.DELAY(500).THEN(function () {
console.log(', ')
this.CONT.CONTINUE()
}).DELAY(500).THEN(function () {
this.CONT.CONTINUE('world')
}).now()
}).THEN(function (str) {
console.log(str)
}).now()
Pretty intuitive, don’t you think? Important note: once you’ve nested the continuation (instead of calling the `CONTINUE`) you must not call its `CONTINUE` method to avoid the split of the control flow. From this point, its the nested continuation responsibility to return the control flow.
There is also a `RETURN` method to skip the other tasks from the current nesting level and return the control flow to the outer.
Exceptions
We talked about the correspondence to the `try/catch/finally` block, and here’s their equivalent in the CPS. Surprisingly, they are named `TRY/CATCH/FINALLY` :) . Simplest case:
require('task-joose-nodejs')
TRY(function () {
throw "oops"
}).CATCH(function (e) {
console.log("Exception caught: %s", e)
this.CONT.CONTINUE()
}).FINALLY(function () {
console.log("Cleanup happens")
}).now()
More text in bold: The `CATCH/FINALLY` tasks are also “continued”. That is, to return the control flow from them you need to use `this.CONT.CONTINUE()` as in `TRY/THEN`. This will allow you to write asynchronous exceptions handlers.
Nesting and Exceptions
The exceptions will correctly propagate from the nested continuations:
require('task-joose-nodejs')
TRY(function () {
console.log('Hello')
this.CONT.DELAY(500).THEN(function () {
console.log(',')
throw "world"
}).DELAY(500).THEN(function () {
console.log("YOU SHOULD N'T SEE THIS TEXT")
}).now()
}).CATCH(function (str) {
console.log(str)
}).now()
they could be re-thrown just fine:
require('task-joose-nodejs')
TRY(function () {
console.log('Hello')
this.CONT.DELAY(500).THEN(function () {
console.log(',')
throw "f$$k"
}).CATCH(function (str) {
throw "polite"
}).now()
}).CATCH(function (str) {
console.log(str, "world")
}).now()
And in general, behaves very much like “usual” exceptions.
Show me the parallel
Of course any library for asynchronous control flow should provide the “parallel” mode. In JooseX.CPS you can activate it with `AND`:
require('task-joose-nodejs')
TRY(function () {
console.log('Activated branch 1')
var CONTINUE = this.CONT.getCONTINUE()
setTimeout(function () {
console.log('world')
CONTINUE('!')
}, 1000)
}).AND(function () {
console.log('Activated branch 2')
var CONTINUE = this.CONT.getCONTINUE()
setTimeout(function () {
console.log('Hello')
CONTINUE('!')
}, 500)
}).THEN(function (res1, res2) {
console.log(res1[0], res2[0])
}).now()
If you’ll run this code, you’ll see that branches were activated simultaneously, but the 2 final exclamation characters appears only after the branches gets merged. The very 1st task following the `AND` group receives the `arguments` objects passed to `CONTINUE` in the order of branches declaration.
Of course, you can nest a new continuation into any of the branches:
require('task-joose-nodejs')
TRY(function () {
console.log('Activated branch 1')
this.CONT.CONTINUE('Branch 1 result')
}).AND(function () {
console.log('Activated branch 2')
this.CONT.DELAY(500).THEN(function () {
console.log('The very long')
this.CONT.CONTINUE()
}).DELAY(500).THEN(function () {
console.log('asynchronous task')
this.CONT.CONTINUE()
}).DELAY(500).THEN(function () {
console.log('from several steps')
this.CONT.CONTINUE('Branch 2 result')
}).now()
}).THEN(function (res1, res2) {
console.log(res1[0], res2[0])
}).now()
Some sugar
JooseX.CPS.Continuation also has lower-cased synonyms for the most used methods:
- THEN – then
- AND – and
- CATCH – except
- FINALLY – ensure
Also, the `this.CONT.THEN( … ).now()` can be written as `this.CONT.andThen( … )`
These synonyms simplifies a code a little bit and makes it more readable, but it still contain a lot of boilerplate. This boilerplate can be greatly reduced when using JooseX.CPS trait, which adds the “continued” methods to your class.
It will be described in the part 2 of the tutorial, stay tuned!