>> Lev Nachmanson: I'm very glad to introduce Michael Wybrow. He's a research
fellow in the Clayton School of Information Technology at Monash University of
Australia.
And Michael has attracted several awards, including the CORE Australasian
Distinguished Doctoral Dissertation award for the best computer science thesis of 2008.
And currently he is working on tools which provide edge routing and placement for
diagramming.
So let's -- I hope it will be a very interesting talk.
>> Michael Wybrow: Thank you, Lev, and thanks for coming.
So this is work that I've been doing as part of my Ph.D. research and directly following
that over the last sort of five or six years.
It's collaborative work that was done with a few people. Tim Dwyer, who's now at
Microsoft but while he was at Monash University, as well as Kim Marriott, Linda McIver
and Peter Stuckey. So I just wanted to mention that.
This talk today I'm going to talk specifically about placement tools for diagramming, and
these are things like geometric tools for layout.
I'm going to talk about the way that we are advocating implementing these, which is why
the use of multiway constraints.
And I'm going to present to you some results of some usability evaluations that we did
comparing multiway tools with other solutions.
I'm also going to talk to you a little bit about the ways that we approach connector routing
in diagrams as well.
So all right. Let me start out and give you some motivation. This is an image that came
from just Googling for phases of the moon. So if you type this into Google, you'll get
probably a couple hundred images that show effectively the same thing, the different
phases of the moon, dependable on the position of the earth or the sun.
One of the things you'll notice looking at a diagram like this is that you can immediately
see that there is intended layout or geometric relationships within it.
Things are mostly in line with each other. Labels are mostly centered with the
accompanying objects that they're labeling.
But you can also see that this is something that has either been arranged by hand or with a
piece of software that probably didn't provide great layout tools. And I can say this
because you can see the intention in the diagram, but you can also see where it fell down
and the intention has not be fulfilled, and that is that certain objects are not correctly in
line and certain labels are not quite aligned or centered with objects.
So it's probably better to create diagrams where you fulfill all of these relationships,
because, one, they just look neater, but this is more important because where you have
these relationships that aren't quite satisfied, it's a little jarring. And these are things that
users will notice and there will be this unintended implicature, okay, where people look
and possibly see some information that is not relevant at all and shouldn't be there.
So if possible, ideally we would create tools that allow the users to fulfill these
relationships as easily as possible and maintain them throughout their editing.
So a few points of note. Producing well-laid-out diagrams is going to be difficult and
tedious for users if they do it manually.
You're often going to want to be doing things like aligning and distributing objects,
spacing them equally.
Now, many existing tools have automatic features for things like grouping objects
together, connector tools that will automatically reroute connectors as you move objects
around your diagram.
But these are not a complete solution and don't satisfy the problem completely.
Also some existing software or a lot of existing software just provide layout tools where
you select some objects and you click a button and you say I want to align these objects.
And literally what will happen is you'll get alignment to a new position and there's no
persistent relationship at that point.
When you use these and you change parts of your diagram, you're going to find that you
need to repeatedly make use of these tools to keep satisfying such relationships.
So you have this problem of repeated use where you're requiring users to do the same
task over and over again, which is very annoying.
So let's talk about some of the existing tools. The first one that many pieces of
diagramming software offer, what are called -- or what we're going to call once-off
placement tools, and these are the ones where you just have some shapes, you just collect
them in whatever way is necessary, and you apply a tool that moves into a new position.
And I'm sure you're all familiar with this, because basically any piece of diagramming
software has this ability.
The key thing here is there's no lasting relationship.
The next way, and this is something I'm sure you'll be familiar with as well because it's
something that is offered by Microsoft Visio as part of its layout tools, is the ability to
create one-way constraints or what is intended to be a persistent form of alignment and
distribution tools.
And how these actually implemented is that you have a guideline object that's added to
the page that represents your alignment. And you have a set of one-way constraints
analogous to formulas in spreadsheets where you set the position of a shape, like shape A
in this example, the exposition of it, to be equal to the exposition of the guideline.
In this case the user can use the guideline as a handle or an object to move the entire
alignment relationship. So they can click it with the mouse and drag it and obviously like
you'd expect in a spreadsheet, the formulas flow through and the values update and your
shapes move with it.
Now, there are some issues with this or at least we conjectured at the beginning of this
research that there are some issues with this. If you really want to use these as your
method for lying out objects within a diagram, you can have situations like this.
So here's one of the basic situations where this causes problems. We have some shapes,
A, D, and B, the user selects shape D and A and they create one of these persistent
alignments. So they have a guideline, they have the positions of these two shapes have
been constrained to that guideline.
Next they select shape D and they select shape B and they create an alignment with the
bottom of these two shapes. What actually happens at this point is, okay, the Y position
of B is constrained to the guideline. The Y position of D is also constrained to the
guideline. At this point it can be constrained to the Y position of both these guidelines at
the same time.
All right. So at this point it's just constrained to the bottom one. And the user will not
know this. They can't tell looking at the diagram at this point. It's only when they
interact with it that they find out that this is the case. So when they drag this guideline,
they find out that the relationship they set up is no longer there.
So a few issues that we felt there were with doing this via the approach of one-way
constraints. Firstly, it's not flexible ->> Lev Nachmanson: Sorry.
>> Michael Wybrow: Yeah.
>> Lev Nachmanson: [inaudible] mic.
>> Michael Wybrow: Oh, I'm sorry.
So the first problem is, yeah, it's not flexible in the sense that the user can't interact with
just any of the objects in the diagram. They must interact with the guideline rather than
the shapes. The ordering of the way -- the order they apply the tools matters. So if they
create one guideline or alignment before the other, then different things happen.
And finally they can't evaluate the state of the diagram at a particular point by looking at
it. Let's say they would open the diagram at the third stage of that example I showed.
They can't tell what's going to happen next unless they either look at the formulas that are
actually enforcing these relationships or they actually move objects and see what -- how
the attachments are connected.
So what we decided to do was look at implementing these placement tools via multiway
constraints.
So the idea here is that you have a constraint solver, a piece of mathematical software
that sits behind the diagram editor and handles these relationships. And instead you have
relationships that work in both directions. And what the behavior of this actually means
to the user who's interacting with it is that they drag the guideline and the shapes move
with it. They drag one of the shapes, the guideline and the other shapes move with it.
They resize one of the shapes, the guideline and the shapes move with it as a result.
So that's probably fine. I'll talk about the study, and then I'll show you what it actually
looks like.
So the first user study we classically did in this area was we looked at using Microsoft
Visio, and we did this because, one, it was sort of a leading tool in its class of tools. Also
because it had a plug-in interface to it, which meant we could do some of this stuff. And
finally because it already provided once-off placement tools as well as one-way
constraint-based placement tools.
And what we did was we implemented the multiway constraints and gave them an
interface and toolbar buttons and everything. So they were very, very similar to the ones
for the existing tools.
And then we gave users a set of tasks that required them to create these relationships and
then modify the diagrams and interact with them. And these were sort of real-word
layout tasks. I'm not going to go into too much detail about this because in some sense I
feel this studied was flawed, so I'm not going to tell you too much about it.
But basically the only thing that we noticed that was statistically significant in the results
was that the one-way tools performed worse than the others. So they were just bad. And
this sort of reencouraged initial feelings that there were severe usability problems with
them.
And what I'm saying by performing worse in this case, I'm saying they performed worse
in terms of taking a lot longer for people to produce the final diagram we asked them to
produce, and also the number of errors of things they could not get correct in this diagram
they were asked to produce.
Now, actually we feel that -- oh, the interesting thing is multiway didn't appear to
perform any faster than the other techniques, and also the once-off tools are very basic
ones, were just faster and less error prone than the one-way tools in these results.
Really we didn't have a high-level confidence in the results of this. And a lot of it we
actually felt were due to the ways that these tools actually implemented in Visio, more
than because of the constraints of doing it with one-way constraints as opposed to with
multiway constraints.
And what I mean by this is some -- an example of one of the issues that Visio does is if
you align an object horizontally and then place it into a vertical alignment relationship, it
will actually break it from the horizontal relationship even though the shape has not
moved vertically. So there's no need to break it from that relationship.
And these were things that we couldn't deal with easily in the plug-in interface, so we just
decided that we would reimplement a prototype editor with similar functionality so we
could do a true comparison.
The other thing that I will note is that some of the other issues are that users pointed out
in Visio when you create a distribution relationship and it turns out this is still the case
with at least Visio 2007 because we're having a look at it yesterday, that when you have a
set of aligned objects and then you distribute them and space them equally, there is no
on-screen indicator that represents this relationship has occurred other than the guidelines
change from blue to purple in color. And at this point they behave differently to how
they used to, and if you drag one of the outer ones, the distribution resizes and you can no
longer drag the internal ones, and often the users were trying to drag these not
understanding why they couldn't work out, what the relationships were on the screen.
So the next stage was I wrote a prototype diagram editor, and you'll notice it looks very
similar to Visio in terms of its interface. And the reason for this was that really we were
trying to do a fair comparison against the existing tools that were in Visio against the sort
of multiway constraint-based approaches.
So it offered the sort of standard diagram editing functionality. You've got copy and
paste and undo and redo. And then it had both multiway and one-way versions of these
tools that behaved in what we felt was the best way possible given the constraints of the
underlying techniques.
So I'll actually just show you what this looks like. So this is done and obviously we can
just move objects about. You can select some objects and, say, align them. And we get a
guideline here that represents the alignment and we can move any of these shapes.
There's also some connector routing going on in the background, which is why things
move around a bit.
If we resize any of these objects, you get the relationship is still maintained.
Also, there's distribution in here and it looks a little bit like this. You can distribute some
objects. You get an on-screen indicator that represents the distribution. So this is -- I
guess we're trying to have a visual representation of all of the underlying constraints so
you can tell what's going to happen before you even interact with it.
Also this is an object that you can use to alter or to remove the distribution relationships.
So I can hit the delete and remove the distribution relationship, the guidelines will
remain.
I can resize the distribution. In fact, I can choose to resize it around any particular
guideline in here if I choose to.
I can also add and remove objects to these relationships and I can, say, delete a guideline
or remove it from a distribution by dragging it out. And then this one is now free. And
the rest of the distribution remains.
So this next study we did we looked at specifically the usefulness of multiway tools
versus the one-way equivalent. And as a secondary thing as part of this study, we also
considered what the value of having immediate feedback in such constraint-based tools
was.
And what I'm really referring to there is you'll notice in Dunnart here everything that you
do updates live all of the connector relationships, all of the alignment distribution
relationships. As the user interacts with them, they get immediate feedback as to what's
changing.
This is in contrast to what you would see in some other software where instead you see -click an object and you drag and you see ghosted-out lines of that object until it's placed
at a position, and then any updates that happen as a result like maybe some connector
routing happen.
So we implemented both of these versions so you can also drag these objects and then the
constraints update once the user has finished their movement action.
So some of the exercises were things like beautifying a diagram which is in the sense the
standard task someone might do if they were just given a diagram that's being very
quickly arranged by hand and they just want to align up objects and space them quite
neatly. And this doesn't usually require much actual interaction with the constraints
themselves. Like it should be -- if you do things in, say, an optimal order, you could do
this with the same number of clicks or actions regardless of whether you were using sort
of one-way tools or the once-off tools or the multiway.
But then we got them to do some other exercises which were things like saying, okay,
you had this diagram arranged, you now flip it to a landscape view and we want you to
rearrange it to make use of the space on the page. And make it look something like this.
So what we were doing was we were giving them tasks that had real-world motivation so
they could see why they were doing it. They also -- in some sense a realistic diagram.
This is one about just making a cup of coffee.
And we also said that these are the relationships we'd like to be present in the final
diagram. And we gave them a few pages of printouts that showed the initial diagram
they started with, the target diagram both with and without these indicators for the
relationships shown on them.
So and then we looked at timings and error rates and things. So this is an interaction plot
that shows mean times taken to finish the tasks. And also with error bars on them.
So the initial things to take away from here are firstly that the blue and green line, which
represent the multiway tools, are overall faster to complete the tasks and significantly -statistically significantly faster than the one-way tools in general. So that's just the blue
and green lines and their own error bars are completely below the red and black lines.
The other thing to note is the crossing here, basically when you see a crossing like this, it
says to you that the tools behave differently for different tasks. So in this case this first
task was this beautify one where it didn't really require any interaction with the
constraints, like actually working with them once they were created. So we ended up
getting a different result that the immediate feedback tools performed better than the
delayed feedback tools in this result.
What is perhaps particularly interesting here is that the delayed feedback versions were
faster -- and this is a trend, but it's not statistically significant -- than the immediate
feedback.
And this is sort of against what we'd expect because we conjectured that this immediate
feedback was beneficial to the user. It should make them be able to perform the tasks
faster because they see exactly what's happening immediately, and they can do things
without having to weight and then ultimately go, oh, that's wrong on error reset and
choose to do something different.
So -- yeah, yeah.
>>: Do you have easy undo?
>> Michael Wybrow: Yes. So in the sense that you could -- you can just drop something
and hit undo, you can also do escape. In one of the early ones, we didn't have like hitting
escape during the action, but we did later on.
So yeah. The first thing to take away from this is one-way versus multiway constraints.
Okay. It's very obvious that the multiway constraint-based tools are faster to accomplish
this task and they also result in less errors.
What is interesting is that for immediate feedback we know that there's some value for
comprehension involved in this. And especially one of the interesting things that were
noticed during watching replays of this interaction is that participants will instinctively
shape objects or sets of connected constrained objects.
And they sort of do this as a way of going, okay, I'm just going to say how is this object
related to other things. And they just sort of move it about a little and go, okay, that's
what's going to move and then I'll drag it somewhere else.
And this is particularly interesting because this wasn't part of the training we provided to
users. We didn't say, okay, these are constraint-based tools, they update live, this is a
way you might like to use to understand how they interact with each other. This is
something that because the behavior of the tool is quite a real-world approach or sort of
metaphor for them, they just make use of it and they do that automatically.
The second thing that's really interesting is that, okay, the immediate feedback did not
lead to significantly faster time. And this sort of comes to your question. And we went
back and we looked at why this was.
And it turned out that the reason it's slower is that people have this system of real-world
interaction where they drag something and they go, okay, no, that's wrong, rather than
realizing they're doing something wrong and taking the easy undo approach, which is
very easy, you hit escape or you drop it and you hit undo and everything jumps back to
how it was.
They don't do that. Instead they manually and slowly drag objects back to where they're
originally positioned, which is sort of the -- I guess the real-world approach of how if you
were doing this with connected objects in the real world, this is what you'd do.
So we found this quite interesting. We suspect, but we haven't tested it, that if you were
to do training on these users and tell them that this is a much faster and easier way to do
this, you'd get by without it. Or, sorry, you'd negate the effects of this, I should say.
>>: Another question.
>> Michael Wybrow: Yeah, yeah.
>>: So when they jiggle the objects, did it end up slightly changing their diagram by
pixel layer, pixel layer, and then they -- what I'm wondering is if you were to add a
feature where if you mouse-hover over something it just automatically jiggles everything
connected to it, if that would have helped or not.
>> Michael Wybrow: So I think that quite possibly it would. And I'll just talk about this
stuff in a second.
So, yeah, exactly. One of the follow-up -- we did a follow-up study to this. And this was
also just looking at in a little more detail ways that we could possibly avoid this problem
of clutter, because one of the things that we noticed from the very first study we did and
the later one I just talked about, one of the common criticisms that we got from users are
ways that they felt that the software could be improved was that there were too many
things shown on the screen.
And this especially becomes the case when you have lots of these constraint indicators
that represent guidelines or alignments and distributions.
The other thing we did as part of this study was looked at ways that we could possibly
help the user understand the system of constraints that were actually on the screen and
particularly how objects were connected.
Because one of the problems that I guess some of these users ran into would be I set up
these relationships but then sometimes I still don't understand exactly how they interact.
And they move something. And an object they don't expect to move on the screen moves
with it, and they sort of go, okay, why does that happen. And it's hard for them to figure
out often the chain of constraints that may cause this to occur.
So some of this stuff we did here was we -- there were a fair amount of suggestions, and
these also come from some suggestions that have been made in literature on constraints in
diagramming tools as well. Because this is a well-known problem, this issue of clutter on
the screen.
We had some options where we could try doing different things, and some of these were
have the indicators glow when they're active. So if you move a shape and another shape
is attached to it through a guideline, you have that guideline glow orange and it sort of
highlights the effect of the movement.
And the immediate feedback we think should be helpful for this because it does motion
highlighting in a sense that you see the movement of all the objects that are connected,
which is I guess why the jiggling itself is useful.
We also tried having just a constant reduced visibility to these objects so they're just
shown in some level of transparency. And also the other thing we tried was looking at
having them fade out when they're inactive. So basically rather than glowing, I guess,
they would be at full visibility when they're in use, and then over, say, a couple of
seconds they would just fade down to reduce level of visibility.
And what we found -- oh, and the second thing we did as part of this tool was something
similar to the jiggling you were talking about but note quite, and this was to have a sort of
query tool -- and sorry about this looking terrible, but it was an old version that showed
connector points at all times and stuff. But you select an object and then you can use
brushing to basically move your mouse cursor over other objects in the diagram and see
if and how these objects are connected.
So in this case, this particular shape here is connected through this guideline, which is
attached to the selected shape, through this distribution to this shape which is both center
lined and left aligned and through here and to here.
So from the study, our findings were basically that the participants preferred -- they
preferred I guess visualization for this was for the indicators to fade when inactive but
remain at full visibility when they're in use, when they were active. And to fade down to
a just-visible level.
So really obviously you don't want to hide something on the screen and have it disappear
completely. That's going to be terrible. Same problem we had initially where the user
just can't evaluate the state and how the diagram's going to interact before they make
changes to it.
Also, interestingly, this took away the complaints that we had about clutter. And we use
the same actual tasks, so you would expect to have got similar need back where we got,
say, feedback on this clutter issue from 60 to 70 percent of our initial participants. We
didn't get it here. Instead, they complained about other issues instead that they felt were
more problematic or of the ways that tools can be improved by automatically inferring
alignment or distribution relationships for them. They sort jump onto the next level of
what's possible.
The other thing that came out of this study was that this query tool of doing the
highlighting as you move around was useful. It helped some participants. It still did not
get rid of this issue completely. People were still left going, okay, sometimes I want to
sum feature or total, they weren't able to describe how they wanted it to actually react,
that gave them information about how these connections operated.
And, no, so in answer to your initial question, we did not do something where they could,
say, select an object and have the accompanied ones jiggle. But obviously it may not be
required if you have immediate feedback I guess because the user can very quickly do
that. Though, I guess it allows them to do it without -- while knowing that they don't
have to change the state of their diagram in any way. So it would be very easy to
implement and possibly quite useful.
Oh, that slide wasn't supposed to be in there.
So in addition to this, we've later moved to doing some automatic layout stuff, which I'm
not going to talk about today because Tim has probably already talked about quite a lot of
it here. But you can approach me or him separately I guess if you're interested in more of
it.
But at the same time we switched to using the different solver to represent these
placement tools and placement constraints, and as a result we implemented some further
additions to the placement tools we had available.
And some of these things were automatic nonoverlap of objects, so this is basically just
that concern I'll just get rid of these. You can just turn on nonoverlap of objects. And as
I drag them around they just run into each other and can't cross in any way. They just
bump into each other and move.
Also we created a tool for allowing minimum separations to be specified between
guidelines. So this is sort of similar but not exactly like the distribution. And, again, I'll
show you how this looks. You basically select some objects and say keep them
separated. And, again, we get an on-screen indicator that represents this separation.
You'll notice it basically glows red when one of these constraints reaches its minimum
size and starts interacting. And you can see as they bump into each other this kind of
occurs. Also you can click on the relationship, obviously remove it, but also resize this
minimum separation if you desire. And delete it as well.
Also, we did page containment which can be done very easily with these same separation
constraints. So objects now are placed within the page. But because you still want to
allow the user to have what feels like full control over the application, you want an object
that is dragged with the mouse to still follow the mouse as you move things. So you don't
want to make this constraint on the page stronger than the one that holds the dragged
object to the mouse cursor. So as a result it can sort of still balloon out sometimes. So if
I push an object with the one I'm holding with the mouse, it will go outside the page. But
then if I let go of it, it will spring back. Or if I just move it out of the way you see the
page spring back.
And, again, all of these tools we're working on, our aim with them is to take tasks that the
user would usually fulfill manually or with a repeated process and instead make the
diagram editor to maintain them automatically while still giving the control to the user to
be able to alter these relationships and I guess direct how they behave.
The other thing that we've really been trying to do as part of this is make sure it's very
clear to the user the behavior of them based on the on screen representation. So I guess
we've worked hard on what it should look like when things like these page layout
constraints are not quite satisfied or when you actually -- I guess what different constraint
relationships should look like.
So I'm going to stop talking about the placement tools at this point and talk for maybe the
next ten or so minutes about connector routing that we're doing.
And, again, you've seen -- it's just going on in the background of Dunnart as I was doing
this. We're using an approach for this poly-line corrector routing, and also we're doing
this for orthogonal routing, which I'll show you in a moment, where we construct a
visibility graph of which objects can see which other objects. And then we do a search
through this graph and find paths. And then we do like a final stage that improves these
routes when necessary.
And we particularly choose this approach because it gives us a lot of flexibility, which I
think is really important or beneficial in the context of a diagram editor where if you're in
a diagram editor often the user is going to have certain diagrammatic conventions that
they're following. Obviously a lot of the time aesthetics of the diagram are important for
readability, or all of the time.
So it's fine I think to take, say, just optimal approaches from literature for a certain
problem. But then quite often I think there's a need to adapt these and give them some
flexibility to make them more useful within this particular context.
So the key reason we chose these is really this flexibility to capture certain diagram and
drawing conventions and aesthetics.
So like as an example of this, well, I'll show you first of all what is actually happening
here is this is the visibility graph in this case. It's basically a graph that connects all of
the corners of all of the shapes as well as the connection points.
And it's not a complete graph in the sense that there's no need to have certain edges in it.
Like there's no need to take the bottom right corner of this shape and the top left of this
one, because nothing would ever go directly from those corners. It would always, you
know, come around the shape or come from a connection point that was inside it or
similar.
But then when you're actually doing the determination of the paths through here, and I
realize I don't have exactly the right demo, so I'll just start it up manually -- we're using
an A star approach or search though here, and we can change the cost function to add
penalties to it. And as a result we can change the properties of the routes that are
produced.
So one of the things is you might go, okay, I want to have penalize sharp angles within
these connectors. And as you move up this penalty you start getting better routes for it.
You could also just penalize strictly the number of segments in the connector. And you'll
see as you raise this penalty, you start getting routes that are possibly more aesthetically
pleasing to the user.
One of the improvement techniques we use in these kind of cases is where there's these
shared paths or routes that come together is cases like this where things go around the
corners of the same shapes. You obviously want to avoid this because it's hard for the
user to tell exactly which path is which, other than obviously clicking on them and seeing
one selected by not the other.
So we do a stage where we determine ordering of these things, and we classically nudge
them apart so that there's space between them. I can't actually show you in this one, but
I'll show you in the next one what this looks like.
There's some issue with it right now.
>>: Are you trying to do it for [inaudible]?
>> Michael Wybrow: Yeah, yeah. Well, because -- and I guess I would show you some
of the produced diagrams for it. Maybe after the talk come and see me. Because, yeah,
like it's quite common that you might have a couple of connectors run from here, and
then they also run together like down here at a part later. And it's just really hard to flow
them. So if you can do this -- there's a couple of things you can do, right? And this is
really the benefit of the approach is you can look for these crossings once you've
generated all these paths or even do this on the fly when you're first generating the paths.
And as part of your cost function you can actually look for these crossings or shared
paths and penalize them depending on their actual properties. So if they run in parallel
but then diverge and you know that they don't cross during the course of their shared
path, it might be fine to leave the routing like that and just nudge them apart. But if they
do cross, it might be that for one of them you might want to try searching for a different
route for it, which might be slightly longer but may have no crossings to it.
And you can do this for like do crossing detection for all of these things while you're
routing, but you take a penalty, like a speed hit for it. So the way I've actually
implemented it in here and the then I'd probably advocate doing it is to do all the initial
routings and then look for problem cases, so like do crossing detection there once for
everything. And if there's particular things that you want to fix, you do it at that stage.
And ideally in an interactive context like this, the user physically just selects something
they don't like and says, hey, I want to reroute this connector and I don't want the
crossings and it fixes it.
So this approach also works for the orthogonal case. And this is I guess important here
for you guys as well. I know you're working on this a bit as well. Like this is the
orthogonal connector routing in Visio. And I'd like to point out a couple of things that
are really I feel problems with it. First one is you get issues like these where you can't
tell exactly where the paths go. So the overlap is a problem. Second is you can see some
artifacts in the routes or the layout of the technique that is used to actually determine the
paths here.
So and this is basically where does this line segment end up, does it end up against shape
F or does it end up against shape E. And this will be dependent upon probably the
direction the search is performed when looking for these routes.
And the interesting thing is as you move things about, these change. So it's not at all
predictable for --
>>: [inaudible]
>> Michael Wybrow: Yeah. And I think I agree, that's what it looks like here.
But really the issue with this is that a user who is using it shouldn't have -- shouldn't need
to have any idea of what the underlying algorithm is that's being used, right? The tool
should behave predictably and in the same way regardless of, say, the direction that the
routing is performed.
So ideally like this would be centered in this space and likewise for the line up here. And
we do something about this.
So this is all stuff that we can do by using these kind of approaches that I talk about. So
the -- this is -- and this is with the rounded corners, but it doesn't have to have rounded
corners. Looks a little strange.
So this is what the same diagram looks like in Dunnart. Here's the visibility graph that
it's routing through. And I can show you what the sort of interaction looks like, because
you move things around. And you can see this segment here is centered between these
objects, and you can see the sort of behaviors as you move things around. The top order
here is a bug. I don't know what's going on there. I was working on an optimization for
this that may have broken some things.
Also, the case I haven't implemented is the one where you just block the visibility
because -- let me see if I can do this somewhere like here. This is just where it doesn't
have a line of visibility anywhere because the shapes are actually overlapping.
Something smarter needs to be done here. And this is actually because there's a port on
the top of the shape. So it's not that the shape has visibility in all directions, which
ideally, if it does, like these sort of cases don't really occur. I mean, we could actually do
that here I guess if you connect to the center of this shape.
And maybe I should point out, the other part about this is how we handle connections to
like the center shapes here is you actually just have a point of visibility and special edges
within the visibility graph that just give you connection to the ports on the side of a
shape. And it will automatically just determine the closest one based on that search any
way. So there's no extra sort of logic that's required in your algorithm as part of this.
So there's a few tricky things that are done as part of the improvement step of this. The
first one is you need to do this sort of detection of things like crossings and ordering.
>>: [inaudible]
>> Michael Wybrow: Can you -- sorry?
>>: Can you [inaudible] like any word on the top [inaudible]?
>> Michael Wybrow: Yeah. So -- oh, of a shape? Yeah. The ones in Dunnart right now
have fixed ports on the middle of each side of the shape. But there's no reason you can't
place a port anymore. If you come and see me after the talk, this actual research is being
funded by an industry collaborator who's actually interested in this for circuit diagram
layout. And quite obviously they have -- their objects are like integrated circuits with
pins all along the sides of them. And one object might have, you know, 20 connections
or whatever. So, yes, it certainly works. Effectively all you're doing is you're just
placing a point that you're giving visibility in a certain direction. And sometimes it has
visibility through the edge of a shape if it's contained within that shape.
The other nice thing about it is we do this -- this nudging step, right, that nudges apart
these shared paths and, in fact, does it when they flow through multiple segments. And
what that actually is doing is it's firstly determining an ordering for these things that
prevents them from crossing or makes them just cross once if there is a crossing that
cannot be avoided. And then it's determining the area of free space it has in the
dimension that it's doing the nudging.
And if you would like your ports -- so in this case, we have two connectors that are going
to the same port on shape. But there's absolutely nothing from stopping you from doing
the nudging for these final segments.
So in this case, because our channel of free space is here, we would actually nudge just in
the line of visibility and up to the edge of the shape there. And the nudging would just
space them out equally along the -- that shape from the center. So effectively you get sort
of a nice space layout in that case for free without doing any extra work.
Okay. So I'm not going to go into any more detail for that because I guess my hour here
is about up. So I will conclude. But you're welcome to come talk to me and I can show
you further demos of these things or explain to you exactly what's being done in the
background.
So the key thing I guess I really want you to take away from the talk is that we can
certainly provide both layout tools and connectors that automatically maintain, say,
drawing styles and features, also that allow the user to customize the layout and that also
saves the user a lot of effort. Okay. So a lot of these things could be done automatically,
but they can be done in a way that is done well and that is predictable for the user but also
gives them flexibility.
I think it's really important to make these things behave not truly optimal but in a way
that the user can understand and work with easily. And I also think that the interface for
a lot of these tools is really important. And I think that these are points that are often
forgotten.
So at that point I'll say if you're interested in further information on like these user studies
or the research, there's my Web site which is here at Monash at the top. All of the
underlying methods we're using for doing both the connector routing and for doing the
constraint solving and things are part of a set of C++ class libraries in the Adaptagrams
project. And my prototype editor is also available as some binary downloads at the
moment, but the source will be available sometime shortly as well.
And I will say thank you for you attention.
[applause]
>> Lev Nachmanson: Any questions?
>>: What I find very interesting is [inaudible] a company that is -- subsidizes, so they
don't keep the result of the [inaudible].
>> Michael Wybrow: So, yeah, the interesting thing is they basically came to me and
they said we've seen you've done this poly-line connector routing, and I said yes, and they
said we also see that you say that you're planning to do this orthogonal version; have you
done it yet? And I said no. And they said could you do it quickly; we have a release
coming up in four months or something, we're interested in this. And I said, yeah, it will
probably take a couple of months of work to do an initial version. It won't be great, but
we can investigate how to optimize it later. And they said, okay. So how much will it
cost, this kind of thing,
And we said can -- this is what we'd like, we'd like for the university to own the
intellectual property. And the weird thing is the library was already open source. It was
address [inaudible] they could already have used it, but they preferred to pay us an
additional licensing fee and get the rights to basically use it without telling anyone they
were using it and to not flow the license restrictions, which is not providing source code
for the library and not providing basically dynamic linking of the library.
>>: So you [inaudible] license?
>> Michael Wybrow: Yes. And that was, okay, because the only authors of it are myself
and Monash. So yeah. So it's Joe Larson's. But they are completely happy for the work
to be published, which is why I'm actually here, because I was in Chicago last week
presenting the orthogonal connector work. And also the implementation is all open
source.
And in fact the implementation of the orthogonal connector routing has just been put into
Inkscape as well, the open source vector graphics editor that sort of competes with Adobe
Illustrator. So that's going to have some orthogonal diagramming capabilities. So I had
a -- I was mentoring a summer of code student over the last couple of months doing this.
>>: I think it's good where [inaudible].
>> Michael Wybrow: Yeah, yeah.
>>: That shouldn't be avoided.
>> Michael Wybrow: Yeah, that's right.
>>: So you mentioned for the ports, provided port, which is only the top ->> Michael Wybrow: Yeah, yeah.
>>: -- only provide [inaudible] that do not provide the entire surface.
>> Michael Wybrow: No. So in fact all I'm doing -- so you could -- okay. If you're
worried that you won't have like visibility to the center initially, you could certainly do
this.
At the moment all I'm doing is I'm just providing a single port, and if I want to space
things out along an edge of a shape or I just do it at the improvement step when I do this
nudging, which is I've got a path that I know goes through there, I've got some free space
that I can easily determine. And how I do this is I sweep through the diagram and look
for these channels, so I know where the obstacles are and how much space I have to the
edge of the shape. And then I know also the ordering that is required to prevent
crossings. And then at that point I just space them out.
But if you were worried that you may not have visibility to the center, you could certainly
provide like, I don't know, three of these, but you don't need very many like equally
spaced along there or something. I don't think there's a need to have 20 of them at close
intervals and do it at the routing stage. It's going to be a lot more costly at that point to
try and work out where the crossings are and where the ordering needs to be to avoid the
crossings.
So the code that's used for doing this -- that's used to do the orders is basically similar to
the approaches that are used for determining like metro-line crossings.
>> Lev Nachmanson: I think we need to close to talk and then we can discuss.
>> Michael Wybrow: Sure.
>> Lev Nachmanson: Let's thank Michael.
[applause]