Virtual World Framework – Addt’l Notes to Add
VWF Introductory Notes (from intro session)










Dynamic Plant Model (DPM) – complex model, ex. Nuclear plant system – don’t want to write
javascript to handle model, proxy it in as a client
Levels:
o Kernal
o Model: Script
o Model: Scene
o Model: Properties – Switch Lamp
 Click goes into a model, filters down to kernel
 All external events bounce off the reflection server (with timestamp)
 A DPM would be an additional web service to bounce off reflector
Transform index.vwf into an application – transform browser into a vwf application running in a
browser
Drivers
o One expresses data from 3D models (collada files)
o Web service driver – does proxy into remote server
Kernal brings in each driver and asks “can you handle this event?” ex. Mouse click
o Changes state of switch, sends to kernel and changes are propagated through scene
manager to be changed
o Application sees a method was called and properties changed
Views – accepts canvas mouse click – method call to kernel, pointerClick on this node, sent to
reflector and sent to clients to internally execute
Kernal
Models – Switch, Lamp
Code –
o Lib/ server code is here – starts in vwf.rb, reflector in vwf server
o Public/ application files – finds index.vwf.yaml (encoded in json or yaml)
o Support/client/lib – loads index.html
 Vwf.js is kernel
 View-.js are drivers
 Vwf/model.js, model/glge.js, model/object.js – Models
 Async.js – managing callbacks
Radio example:
o Application side – public directory – index.vwf.yaml – indentation important
o Dashes mean it’s an array – script array example
o References are local to file, relative path
o XML glge scene manager file
o Children – can be defined in yaml and bind to a model defined in XML





Children can have properties, and can be a shortcut notation or define set
function and value
 Arbitrary javascript for child or application overall, depending on indentation
 Initialize – timing issue work around – ensure the positions, etc properties are
initially set, controls order things happen
Naming convention – need to be able to determine whether it’s a component or not
o http://vwf.example.com/types/node3 (without extension) local cache on server
o appscene.vwf relative to path
Load order: treats each vwf file as a directory, puts all client directory files automatically
(index.html)
o Index.html – loads jquery and other libraries
o Kernel file – vwf.js
o Require.js library – module loader
o Loads drivers
o Call vwf.initialize – configuration of drivers (here you could attach drivers to certain
parts of the graph – on initialize configuration)
o Stylesheets for admin view
o Custom HTML style, jquery content – vwf.html file – describes page, imported when
application loaded, dropped into body
Vwf.js
o Keeps track of models and views
o Initialize function – takes in models and views to initialize
o Parses any URL arguments
o Constructs each model and view
o Jumps to ready function
o Establishes socket connection
o Sets up callback function to handle messages
o Handles incoming and outgoing messages
o Tick – how it handles time
o Kernal functions – createNode, addChild, removeChild, setProperty, createProperty, etc
o 2 APIs
 Kernal API – coming into the kernel, createNode, createProperty, callMethod
 Kernal goes to each model, and says “setting property – do you want to
handle it?”
 Model API – settingProperty, actively doing, “gettingProperty”
 View – past tense of property name, “gotProperty”
Model Drivers
o Modules – call global function define
 Extending model object with object


Initialize – in kernel, define model api functions that the kernel will invoke –
active form “creatingNode”, goes to get object based on nodeId, remember ID
and source, create new hash for properties
 If something is returned, kernel will quit asking, otherwise will keep bubbling it
up
o Object driver – object.js
 Default driver
o Javascript driver – javascript.js
 CreatingProperty – needs ability to invoke getters and setters
o Glge.js – current state, view and model code mixed up together
Eventually get to table based scripting – using text implementations – 80% of scripting done that
way, better for people who don’t know programming
Email Notes
model/view api assistance
- configurations
- maintains ids
- has own node ids and requires translation
- can orphan
- can maintain a reference to a node not (yet) attached to a parent
- can prototype
- can mirror the prototype graph and cow properties
- cares about
- sub-tree rooted at source type, eg, node[@type=’model/x-collada’]//*
- nodes with scripts, eg, node[script]
- partial api on a node, eg,
- property gets, but not sets (exposes read-only property)
- methods, but no properties or events
Notes on components & types
- Will simplify component specifications.
- Typical minimum currently: { extends: “.../scene”, source: “something.unity3d”, type:
“application/vnd.unity” }
- Becomes: { source: “something.unity3d” }
- Or even, “something.unity3d”
- Allow component URIs to resources other than .vwf files. Assign base (“extends”) using mime type
mapping.
- “abc.vwf” => { extends: “abc.vwf” }
- “…/scene.vwf” =? { extends: “…/scene.vwf” }
- “something.unity3d” => { source: “something.unity3d” } => { extends: “…/scene”, source:
“something.unity3d” }
- source => type => extends
TODO
Reflector
Timeline
- Play/pause/stop. Playback rates (inherited, modified by hierarchy?)
Sync
- Client sync, resync. Maintain state on server. Pass to client when joining.
- Client resync. Track client latency reduce to minimum. If a client lags behind, purge and resync from
current state.
- Merge-based state updates. Update system state based on locality of changes, not strictly ordered.
Like a version control system, time-clocks, Google Wave.
Admin
- Admin interface. Show applications, instances, users. Invite, kick, ban users. Show time and message
stream. Play/pause/stop. Authentication is separate from application. Is a separate Rack app?
Component API
- Component APIs: node, node2, node3 (visible, transforms, materials, animations), camera
- “Initialize” script call.
- Behaviors (implements) in addition to prototype (extends).
- Bubble up events
Scripting
- JavaScript scripting. Getters, setters, events, on method, etc.
- Table-based scripting. Same language as for describing lesson progress.
- JavaScript prototype delegation properties, methods.
- JavaScript event listeners.
Get Evented Programming right
- Handle events, callbacks properly throughout vwf. For VWF API (createNode with delayed loads, etc.)
and for interactions with models and views (or only for models since views are not synchronous?).
Sequences of calls into vwf should return only when complete (but possibly via a callback) and should
execute in order.
- Calls into the view API deliver return values via callback.
Model, View API (driver API)
- Provide helper functionality in models and views to provide standard implementation for id mapping,
resolving prototypes, and other common functions.
- Remove any reference to vwf in models and views. Always route through the base model or view
instance.
- Separate GLGE into proper model and view pieces
socket.io library
- Support socket.io reconnections. Client should attempt to reconnect after the server connection is
dropped.
- Upgrade to current socket.io. Requires reworking ruby socket.io server.
- Reduce socket.io client files to remove developer content. Locate proper source tracking updates.
Map media types into component types
- Auto-assign node types with help from drivers. Avoids all of the “extends node3”, etc. references for
common cases. Drivers may indicate a node’s default type when the driver is binding to the node (eg, for
a scene manager, an untyped child of a bound parent may be indicated as node3 if the parent in the
scene has a child matching by name).
- Drivers (or the server?) should provide a list of supported content types that can be treated as
applications.
- Determine the content type from the asset. Use OS’s assignment, configuration in server, or defaults
in client, in that order.
- Standard camera implementation. Provide a default camera implementation that doesn’t require any
special application configuration. A raw asset should be usable as an application with reasonable default
behavior. Provide orbit, walk, fly (which ones?). Allow application to configure or to provide specialized
camera(s) that replace default. Should work the same for WebGL and Unity content. May also want to
do something relevant for 2-D content (zoom, scroll, etc.—useful for svg, large images).
- Allow component scripts to refer to files. Load dynamically.
- Develop a succinct language for describing lesson progression. Should be a declarative version of the
API that JavaScript has access to. Automatically handles delays & callbacks so that event sequences can
be specified linearly. Support branching and merging for parallel actions. Use as a component scripting
language too for simple actions (may handle majority of behavior scripting).
- CoffeeScript
Kernel
- Proper node ids. IDs must be map between the same node and id on each client, but each client must
assign IDs independently and in the face of arbitrary load order for delayed content.
- Content binding. By path and/or other attributes. XPath-like queries. root.a.b, root.”a a”.b, root//b,
root/*[@name=’a’]/b, root/*[@some-other-property-identifying-a=’whatever’]/b. As with Xpath, may
select more than one at a time.
- Resolve address, bind on reference
- Develop a scheme for wrapping a UI on a component. Consider something like panels or blocks from
web content management systems. Allow drivers to specify default UI wrappers with editors for the
data they manage (somewhat like what vwf-view-html does).
- Compose UI blocks to form a full scenario UI for the application (like FAPVplayer).
- Lesson display and tracking.
- Decide on names for: kernel (vwf), drivers (vwf-model-xxx, vwf-view-xxx), content (public/…)
- All relative paths referenced by content files should be relative to the referencing item. Asset
references from components should be relative to the component, and references within those should
be relative to the asset, etc.
- Verify URI escaping in server. Using encoded/decoded versions properly?
- Authentication, authorization. Provide a login mechanism. Allow applications and instances to specify
access controls. Move application instance tracking to a session cookie, remove instance id from URL,
and provide invitation URLs to compensate.
- Handle text encodings properly. Specifically, let the JSON decoder work with text containing a BOM.
- Allow any accepted asset type as content index: index.vwf.template, index.dae, index.unity3d, etc.
Search for any accepted asset type by extension if named resource is not found
(http://vwf.example.com/path/to/content tries index.vwf[.template], index.asset if content is a
directory then .../content.vwf[.template], content.asset)?
- Break out “resource” field in reflection API? Most calls have one. Leaves parameters array exclusively
for parameters.
- Something with 3d css for lightweight 3d
- Debugger for VWF scripts
- Logging. Unify server and all clients. Record by level. Turn up/down by area (particular subsystem,
component, etc.) Disable for release. Remove from JavaScript to remove string calculations even if not
displayed and to reduce file size.
- SCRIPT_NAME, REQUEST_PATH usage correct in server?
- Server access to shared state must be thread aware?
- Module loader. Be RequireJS (?) compliant. Use RequireJS.
- Use script loader. Load driver-dependent scripts only when needed (eg, jQuery UI).
- Missing API functions. removeChild(), deleteProperty(), etc.
- JavaScript security settings for component scripts. Hide window, document, vwf. Set non-writable,
etc. appropriately.
- JavaScript “use warnings”.
- Optimize calls to drivers: by content type (only 3-D nodes), node IDs (descendants of scene root),
script type (only nodes with JavaScript), …. Driver specifies and updates dynamically.
- vwf-model-object (vwf-model-node?) backstop driver to record node graph and property values if no
other driver handles.
- Driver priority of influence (vwf-model-glge has current answer for animated transform even though
vwf-model-object has also recorded it)
- Make a GLGE forks that is well-behaved for VWF.
- Define specifications for good behavior for a VWF model (no state change other than directed by VWF
stimulus, math, library standards), VWF view (no direct access manipulation of model).
- Validate API calls across reflector, both client and server side. Delegate client-side by function
invoked.
- Application source order. Possible prioritization: from vwf.initialize(), in URI (?application=uri), from
reflector, same location as index.html, selection dialog. Allow overriding what reflector would say if
index.html was loaded from an application page (eg, which to choose from:
http://vwf.example.com/path/to/application.vwf?application=http://other.host.com/other/application.
vwf)?
- Don’t reload a component when a load has already started but not yet completed.
- Handle component and asset load failures.
- Order of construction. Construct top-down, then initialize bottom-up? Or both bottom-up. Want to
prevent reliance on parent during construction so that any child node may exist independently as a
component. For a well-defined graph though, drivers would prefer to construct top down to avoid
duplicate storage before node is attached.
- Mock filesystem for tests.
- Read me (github, Sinata-style introduction and overview).
- Handle clients joining running sessions. Initialize with current node graph and property values instead
of application default.
- For animations, maintain playing/paused state, start time, pause time in properties for properly
connecting new clients.
- Transcendental consistency.
- kernel.setProperty/getProperty (and other calls?) from a model should (optionally be able to?) begin
the evaluation starting with the driver below the driver making the call. This is similar to the way that
gets/sets originating in a javascript accessor function don’t start at the top. The motivation is vwf-viewglge (and presumably a future vwf/model/glge) wanting to getProperty() to get the initial values
(originally sent with the createProperty) for playing/looping/speed to initialize the internals after a
delayed load callback. (With proper callback handling, this specific case wouldn’t be necessary, but
similar cases probably are.)
Get/Set use cases:
- Notes:
- need to call on derived/deriving objects when property is changed on deriving/derived object?
- look at get/set in JS defineProperty() for reference
- move to property definition to keep property code with property initial value and main definition
- also makes clear that the property must be declared there when only attaching get/set
- allow property assignments to accept value and/or value+get+set hash
- rename getter/setter to get/set since that’s familiar to JS defineProperty() users
- Cases
- none
- set updates stored value, get returns stored value
- notification
- get and set are called but don’t affect the stored value
- operates like none, except for side effects from get/set handlers
- would like to not require handlers to return a value or explicitly set the internal value
- transformation
- set modifies the incoming value, to quantize, range limit, etc.
- filtering
- set rejects invalid values
- the value isn’t set and no other setters or drivers are notified
- aggregation
- set redirects the value into one or more other properties, color into r/g/b, for example
- get gathers the value from one or more other properties
- no stored value in this case
- no drivers are notified for the aggregated property since there is not value change
- Considering prototypes
- for js
- assigning to derived searches the prototype chain for a get/set definition as for a search when
getting a value
- with a setter on base and no setter on derived, assigning to derived still invokes setter declared on
base
- with a setter on derived and a setter on base, assigning to derived invokes the setter on derived
- assigning to base invokes the setter on base but not the setter on derived
- without a setter on derived, assigning to derived without base as the prototype sets the value
- after assigning base as derived’s prototype, derived’s value is suppressed and base’s setter surfaces
at derived
- anonymous (transform=) extends derived extends node3 (transform)
Unity demoability
- Prepackage Firefox portable with the required version of Unity player?
- Find public Unity player content that will prompt to load the correct Unity player.
- with proxy enabled: visit Unity content site; install or verify player
- disable proxy; visit VWF Unity demo
Component mockup (rough), shows:
- child references as jQuery selectors and/or XPath queries
- property initializers plain or with value, get, set
- event declaration and handler manipulation
- table-based scripting as a hash indexed by incoming event with an array of steps
- changed event per property
- nested declarative script with fork and join
--extends: scene
source: radio.xml
type: model/x-glge
properties:
power: false
otherpower:
set: function( value ) { children["#indicator].properties.on = value }
type?
get: ... get script ...
value: false
events:
power: ~
children:
#switch:
properties:
on: false
#indicator:
properties:
on: false
scripts:
- text: >
function initialize() {
properties.power = false;
# script
properties.power.set = function( value ) {
children["#indicator].properties.on = value } # just one setter; only available to
component
properties.power.add( function( value ) { … } ) # every property has a
changed event for use outside the component
events.power.add( function( value ) { … } ) # explicit power changed event
# events.power.remove( handler ), events.power.flush( context )
methods.something.add( function( … ) { … } ) # every method has an invoked
event? probably not.
}
type: application/javascript
- text
switch.click:
- children.["#switch"].rotateTo( "on" ) # plays animated transform
- children.["#indicator"].on = true
# executes when transform complete
- events.power.fire( properties.power )
switchwithmorecomplicatedhandler["rotating-position-two"]: # non-alphanum event
id
- left:
# fork into “left” and “right” paths running in parallel
- step1
# “left” steps execute in sequence
- step2
- step3
right:
- stepA
# “right” steps execute in sequence
- stepB
- sounds.complete.play() # join, then execute next step of main sequence
type: application/vwfscript
Technical overview notes:
Block diagram of json/yaml path
What server looks like block diagram
Server variants: apache, ruby, node.js
Overview of which & why
- Node.js: lightweight, going quickly?
- Ruby: middle weight
- Apache: full infrastructure
Three pieces to server: file, jsonp, reflector, application root magic
All three running to same API.
VMware, amazon images, boot disks
Pretend to have all the stuff local even if it isn’t. Write that down. (Server application URL magic.)
Email a live working world.
How Unity fits in. How marshalling in and out works.
Unity detailed implementation. View side only OK in VWF. Harder to expose too much Unity
functionality down into VWF APIs.
Fundamentally: graph of nodes with p, m, e.
Import Collada into Unity running in parallel with GLGE. Unity in IE, GLGE in Chrome.
Check Google plug-in to IE for WebGL support
How to automatically install Unity or Google plug-in when first launching on IE?
HTML snippet to paste into page to embed world
That whole thing that I just did there with the duck and plane and timeline start/stop fast/slow and log.
Do the application UI (sliders on demo pages).
Source .dae from SketchUp pages directly from URL.
Recursive embedding of worlds on faces of nodes in scenes.
Separation of timelines when embedding instances of worlds.
/path/to/component.vwf: { property1: value, p2: value }
/path/to/hello.vwf: alert( “hello” )
<script src=”/path/to/component.vwf“></script>
<script>{ property1: value, p2: value }</script>
<script>alert( “hello” )</script>
<script src=”/path/to/component.vwf?name=func“></script>
<script>func() { return { property1: value, p2: value } }()</script>
Diagram notes:
- angles:
- single-user: state machine, external inputs, internal state sequence, state is quantized, time is an
external event
- model/view:
- state machine is programmed as objects (components) arranged in a graph, communicating
through properties, messages, events
- model is state machine, implemented as an amalgamation of drivers
- view presents model state, injects external events, also implemented as an amalgamation of drivers
- state machine
- applications
- engine is kernel + federation of drivers
- state machine
- view vs model
- reflector
Property accessor usage patterns:
- notify: receive notification of set/get
- set: ... do something ... ; this.property = value
- limit: range limit or other validation
- set: this.property = Math.max( 0, Math.min( value, 1 ) )
- proxy: act as a proxy for other properties
- set: this.length = value.length ; this.unit = value / this.length
- get: return this.unit * this.length
- reject: prevent invalid assignments
- set: if ( typeof value == "string" ) this.property = value
Specifying set or get as null prevents writing and/or reading, respectively:
- property
- set: null
- get: null
Component specifications may be any of:
- Object literal
- URI to .vwf or other identifiable type: .dae, ...
- JSON-encoded object (primarily for use in the single-user mode application= URI parameter)
They may appear in:
- extends, implements, child within a component specification
- application=... parameter for single-user mode (URI or JSON-encoded only)
- createNode() call
Nodes inherit from their prototype:
- source/type
- specifying in a derived node replaces the value in the prototype
- properties, methods, events
- derived node's definitions augment those in the prototype
- new name is a new definition
- existing name overrides prototype's definition
- how to delegate to overridden definition in prototype?
- children
- derived node's definitions augment
- new name is a new definition
- existing name configures definition from prototype
- scripts
- derived node's scripts augment the prototype