A COMMAND PROMPT TALK
Node.js and postgres
An Introductory Glance
With Live Code, Later
2010 AURYNN SHAW, COMMAND PROMPT INC.
Hi!
What is Node.js
Fast
Pure event-driven
server-side
javascript programming environment
Ew, javascript
Built on Google’s V8
Runs on modern unix-likes (Linux, OSX, Freebsd, Cygwin)
Speaks network natively, and easily (Hello world is in http!)
Ends up Similar to Python, Perl, Ruby, ETc.
Did I mention fast?
Single thread can do many thousands of connections
standard library doesn’t block
Did I mention fast?
Single thread can do many thousands of connections
standard library doesn’t block
Keeps the slow tasks from blocking
Keeps the quick tasks quick
blocking?
# Standard synchronous processing
fh = open(“somefile”,”r”)
contents = fh.read()
# what if somefile is huge, 1GB+ ?
print contents
Every step waits
How is node different?
Did I mention fast?
Single thread can do many thousands of connections
standard library doesn’t block
Keeps the slow tasks from blocking
Keeps the quick tasks quick
Really good for anything I/O
While slow tasks run, node
carries on
Non-blocking!
fs.open(“somefile”, function (fd) {
// When the file descriptor gets returned.
var buffer = null;
fs.read(fd, buffer, length, position, function (e, bytes) {
// bytes read are here.
// buffer will be not-null.
});
});
As always, there’s caveats to
this
Some caveats
node is entirely single-threaded
Event-driven programming is *really* different
A maze of callbacks
fs.open(“/tmp/hello”, function (fd) {
var buffer = null;
fs.read(fd, buffer, length, position, function (e, bytes) {
// bytes read are here.
// buffer will be not-null.
});
});
Some caveats
node is entirely single-threaded
Event-driven programming is *really* different
Javascript’s prototype model is also really different
The syntax has warts
Build a bridge out of her?
/*
This allows you to kind of inherit properly.
*/
var o = function () {};
var p = function () {
/* Because ‘this’ changes */
var self = this;
o.call(this);
};
p.prototype = o;
p.prototype.constructor = p;
So why is this important?
Why is it important?
Javascript owns the frontend
Why is it important?
Javascript owns the frontend
More jquery and javascript programmers every day
Why is it important?
Javascript owns the frontend
More jquery and javascript programmers every day
Javascript is fast
Faster every day
And now, we can use
postgres!
When I got here...
postgres.js worked.
When I got here...
postgres.js worked.
Used the simple query protocol
When I got here...
postgres.js worked.
Used the simple query protocol
Used SQL escape to prevent injection attacks.
whyyyy
SOURCE: http://www.flickr.com/photos/striatic/2192192956/
But it did work!
And so I wanted to make it
BETTER
So, I had to ask,
How do I not allow this?
Well...
to prepare a query
postgres.js Didn’t use libpq.
SOURCE: http://www.flickr.com/photos/tanaka/2319325408/
to prepare a query
postgres.js Didn’t use libpq.
Speaks pure protocol
to prepare a query
postgres.js Didn’t use libpq.
Speaks pure protocol
...So I had to learn the protocol.
Wax on, wax off
MONTAGE!
Sadly, reality was a lot less
exciting
Complexity Abounds
Simple protocol sends one message to pg
Easy to queue, internally.
REALLY SIMPLE MESSAGE QUEUE
QUERY
QUERY
QUERY
QUERY
QUERY
QUERY
RESULTS
Really, though
Simple protocol sends one message to PG
Easy to queue, internally.
Prepared statements require at least 5
Parse, describe, execute, bind, flush
THIS TYPE OF QUEUE DOESN’T WORK
SYNC
EXECUTE
BIND
DESCRIBE
PARSE
QUERY
QUERY
And while all this is happening
You cannot block the main thread
And postgres expects a synchronous request/response
But it does give responses...
And while all this is happening
You cannot block the main thread
And while all this is happening
You cannot block the main thread
And postgres expects a synchronous request/response
And while all this is happening
You cannot block the main thread
And postgres expects a synchronous request/response
But it does give responses...
So, we buffer message sets
KIND OF LIKE THIS
SIMPLE
PREPARED
GROUP
SIMPLE
SIMPLE
PREPARED
GROUP
SIMPLE
SIMPLE
So we never block waiting for
the DB
the application keeps
responding
But that’s all a bit on the
technobabbly side
SO
Let’s look codE!
Using postgres.js
/* Node setup */
var pg = require(“lib/postgres-pure.js”),
sys = require(“sys”);
var db = new
pg.connect("pgsql://test:12345@localhost:5432/template1");
/* We now have a working DB handle.
Or, we have an error. Either way.
*/
Using postgres.js
/* We now have a working DB handle. */
/* From here, we do: */
db.query(“select 1::int;”, function (err, rs) {
// Wait, a function?
console.log(sys.inspect(rs));
});
Why a function?
What’s going on here?
/* We now have a working DB handle. */
/* From here, we do: */
db.query(“select 1::int;”, function (err, rs) {
// Wait, a function?
console.log(sys.inspect(rs));
});
What you get
1
3
// A little while later
2
It’s happening asynchronously!
The function argument is what’s going to happen when the DB comes
back
db.query returns immediately
and your app keeps on going!
this isn’t the cool bit
Prepared queries
/* Prepared queries work the same way */
db.prepare(“select ?::int as foobar;”, function (sth) {
sth.execute(1, function (err, rs) {
console.log(sys.inspect(rs));
});
sth.execute(2, function (err, rs) {
console.log(sys.inspect(rs));
});
});
Prepared queries
/* which will return */
[ { foobar: 1 } ]
[ { foobar: 2 } ]
Or this!
/*
This becomes a prepared statement behind the scenes
*/
db.query(“select ?::int as foobar;”,
1, // A single argument
function (err, rs) {
console.log(sys.inspect(rs));
});
It would return
[ { foobar: 1 } ]
Sub-ideal Aspects
Bad, bad, bad.
postgres.js is still very alpha.
Node API is in flux
Learning Node takes a while
Cool bits!
Things that are pleasing
clean, perl DBI-based API.
Official support from Command prompt
Performance!
Vroom
[ananke:~/src/postgres-js] aurynn> time node benchmark.js
40000
[ { count: 40001 } ]
node benchmark.js 40000 18.85s user 2.75s system 87% cpu
24.602 total
Just over 2122 queries/second.
Some Stuff we can’t wait to
add
Shiny things!
Notify support
PG notifications
/* Event-driven frameworks are ideal for this, and
the notification setup can be handled behind the
scenes.
*/
db.on(“<NOTIFYNAME>”, function (payload) {
// Do something useful as a result of
// the notification from PG
});
Payload is also arbitrary data
Shiny things!
Notify support
Stability
Shiny things!
Notify support
Stability
Better error handling
Shiny things!
Notify support
Stability
Better error handling
More test cases
Shiny things!
Notify support
Stability
Better error handling
More test cases
Better type coercion
But the most important thing
we’d like?
Users.
You guys. People who care.
So that’s postgres.js.
Any questions?
Okay. LIVE CODE!
Shout out ideas!
FORK IT, AND HELP!
UNSTABLE
https://github.com/aurynn/postgres-js/
NOT QUITE AS UNSTABLE
https://github.com/commandprompt/postg
res-js/