PaaSt the Desktop: Implementing Cloud-Based
Productivity Solutions with the AutoCAD®
ObjectARX® API
Ravi Krishnaswamy – Autodesk Inc.
CP2568
Learning Objectives
At the end of this class, you will be able to:




Create an AutoCAD Arx app with a live data connection to Google Docs
Create an AutoCAD Arx app that syncs data from Google’s App Engine
Identify data and schema mapping considerations in using a PaaS provider data store.
Identify cost and scale issues to consider in connecting AutoCAD and its data to a PaaS
provider
About the Speaker
Ravi Krishnaswamy is a Senior Software architect with the AutoCAD group. He is a long time
developer and contributor to AutoCAD, starting with Release 12, with almost 19 years at
Autodesk.
He was part of the small team that introduced Arx to AutoCAD, and implemented among the
first AcDb based entities in AutoCAD – including the Ellipse and Region entities. Over the years
his contributions to the product have included – drawing compression, xref enhancements
including notification and clipping, spatial indexing, CER, dynamic block evaluation, point
cloud integration and oversight of several core architectural projects. He holds several patents
in various areas of technology implemented in AutoCAD including spatial indexing , dynamic
blocks and CER. His focus includes both internal product architecture and development, as well
as sharing knowledge of the product and platform with end users and developers,
communicating the huge potential of the AutoCAD platform in the context of computing
paradigm shifts. Ravi has a PhD in computer science from Washington State University, and a
BTech in computer science from the Indian Institute of Technology, Mumbai.
Email: ravi.krishnaswamy@autodesk.com
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
0. Contents:
1.
2.
3.
4.
5.
6.
7.
Introduction
Useful terms
Tools, Libraries
Learning objective 1: Arx application for Google Docs
Learning objective 2: Googles Data store – and differences from traditional data stores
Learning objective 3: Arx app for GAE
Learning objective 4: Metering and costs of running a GAE application
1. Introduction
A primary goal of this class is to show how to leverage powerful web service based platforms
and applications in AutoCAD. At the end of this class, one should have the basic tools and
knowledge to implement Arx applications with the cloud platform web services, such as the
Google APIs or any other cloud ‘Platforms as a Service’.
Note: This class includes advanced programming topics.
The AutoCAD programming environment
The C++ Arx api is the foundational platform for AutoCAD application development. It is a very
broad api, and includes the AutoCAD Object Model, as well as apis for editor interaction.
Since key built in objects such as Layers, Linetypes, TextStyles, and entities such as Text,
Tables, Lines, and so on are fully exposed – with access to their properties, this gives the
developer the opportunity to drive data definition from external sources.
Ability to enforce standards and import external information such as tabular data are two
examples of taking advantage of this level of programmatic access to the data.
PaaS
Several companies have made their web application, compute and server infrastructures
accessible programmatically.
These Platforms as a Service (PaaS) takes care of infrastructure. As a result global and highly
scalable compute and storage resources accessible to a developer with very little coding effort.
The developer only has to write the application, the platform takes care of the rest. Azure, GAE,
Force.com are examples of PaaS.
Course contents
As an example of developing applications for a platform as a service, we will implement
examples using Google’s doc apis and the App Engine apis. To address cross platform
considerations, and since the App Engine apis are not .net based, we will be using open source
2
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
tools and libraries with the Java apis. For some of the client side apis, we will offer links and
pointers to implementing the equivalent in .net.
Since the sample applications involve significant amounts of code and different technologies,
we will provide sample code that developers can reuse in their solutions.
If you want to try out the samples, make sure to install the libraries and tools as described in this
handout.
2. Useful terms
These are a list of important terms and concepts that which will be mentioned during
implementation of our sample applications.
Term
Definition
AppEngine
Google’s Platform for server side applications. It provides Java and
Python apis which allows a developer to access the Google
infrastructure where one can easily write and deploy their server side
solution running on Google’s infrastructure.
ApplicationID
An ID you will register, once logged in to your google account, that
identifies your application with the google platform. For google doc apis
access – this is what is used to request permissions to access a users
data.
For the App Engine, the application id will be used for deployment and
monitoring.
ATOM/ Feed
A protocol, originally started with RSS, that is a foundation for how an
entity and a collection of entities are dealt with in cloud/web APIs
Credentials
A snippet of information generated by registering your application that
will be used to Authenticate and Authorize your application to access a
users data on Google Docs
Data Link
AutoCAD mechanism to link contents of a Table entity to sources of
data. We will use this to hook our AutoCAD Table entity to a Google
Docs spreadsheet.
Eclipse
Open source IDE we will use for Java development.
Google API/GData
Apis to google’s applications, including google spreadsheets
3
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
High Replication
Datastore
Googles NoSQL database solution that is one of the features
accessible to developers who use the AppEngine
OAuth2
Authorization protocol - used by an application to access resources
from another application
REST
Representational State Transfer. A standard and framework to
implement web service apis over http. We will be defining our REST
apis for our Google App Engine application.
Scopes
Method to indicate when registering your application with Google what
Google applications you will need access to (like Docs, Gmail,
Google+)
Table Entity
The AutoCAD Table entity. We will use this to store google spreadsheet
data in AutoCAD.
Jersey
A Java library we will use to make implementing REST apis in Java
easy.
Jetty
A Java library we will use to implement a small embedded server for
OAuth2
JNI
Java Native Interface. We will use this to bridge from the C++ Arx world
to Java
Xref Graph
A programmatically accessible structure that reflects which xref dwgs
are referenced from the main drawing. We will use this to represent file
dependencies in Googles Datastore.
3. Tools, Libraries
In this section we will go through the list of tools and libraries for this class.
Java
For Java, we will use the Java Standard Edition that comes with the Java Development Kit (jdk)
1.7. This is downloadable at:
http://www.oracle.com/technetwork/java/javase/downloads/jdk7u7-downloads-1836413.html
Make sure to accept the license agreement, and pick the appropriate platform version you want
to install. I use the 64 bit versions of all applications in this class.
4
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
NOTE: If you are developing the Google App Engine sample, and wish to deploy it to Google,
you should also install the 1.6 (Java6) sdk.
Eclipse
Eclipse is the IDE we will use to develop our java applications. This is downloadable from:
http://www.eclipse.org/downloads/
I used the Eclipse Classic version in this class.
Visual Studio
For visual studio, I use VS 2010 which is the recommended IDE for C++ development for
AutoCAD 2013.
5
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
Setup for JNI
Note that for tying Java into the C++ world of Arx, we will use JNI (java native interface).
In order to make sure that the java runtime, the include files, jni libraries are accessible while
building and running the applications, you need to make sure:
1. The Java Runtime and tools are on the system path:
E.g. I added
C:\Java\jre7\bin;C:\Java\jre7\bin\server
To the PATH variable so that it is accessible by your application. The jre7\bin\server location is
where jvm.dll lives, that will run the Java Virtual Machine inside your arx app.
(you set environment variables in Control Panel\System and Security\System -> Advanced
System Settings -> Environment Variables on win7).
2. In order to builds in Visual Studio, you need to set the include path to your JDK include
files:
3. Set the linker path to the location of jvm.lib
6
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
4. And specific jvm.lib in your .lib dependency list.
Java Libraries
We will use two java libraries in our applications to make development easy.
1. Jersey. This is to make it very easy to add REST apis to our Java application that will
run on the Google App Engine (server).
2. Jetty. This is a very lightweight Java server that we will use for implementing OAuth2 on
our client.
Jersey: The homepage is at http://jersey.java.net
You can download the .jar files in a zip at
http://jersey.java.net/nonav/documentation/latest/chapter_deps.html
I downloaded the zip file.
Jetty: You can download this from the jetty home page
7
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
http://www.eclipse.org/jetty/downloads.php
And, on clicking on the download link, you will get to this page,
Further, click on the download link, to get to the following, where you can download the jar files
for jetty
8
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
Google Libraries
For ther Google Java libraries we have two categories of libraries
1. Google client apis. This is to access the Google Spreadsheet apis.
a. These are links to download the OAuth2 and client libraries.
http://code.google.com/p/google-api-java-client/wiki/OAuth2
http://code.google.com/p/google-api-java-client/wiki/Setup
from which you should download the jar google-api-java-client-1.12.0-beta.zip or whichever
one is current.
http://code.google.com/p/google-api-java-client/wiki/APIs#Google_OAuth2_API
If you click on the latter, pick the following to download the client apis.
9
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
b. You will also need the google GData libraries. These are located at
http://code.google.com/p/gdata-java-client/downloads/list
You will download the jars from the indicated link below (curiously denoted ‘gdata-src’ – but it
actually contains the jars).
2. Google app engine libraries. You will need both the Java app engine SDK as well as the
plugin for Eclipse.
a. You can download the App Engine SDK for java at
10
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
https://developers.google.com/appengine/downloads
b. To install the eclipse plugin for Google’s App Engine, – follow the instructions at:
https://developers.google.com/eclipse/docs/download
(I used the following link for my install in Eclipse.
https://developers.google.com/eclipse/docs/install-eclipse-4.2
At the time you implement, there could be a newer version (this applies to all the
libraries mentioned in this document).)
It should take a few hours to download and install these components.
Most are zip/jar files – i.e. you just download them and place them in a directory.
Once you’re done, you should have all the components you need to implement powerful
AutoCAD Arx apps using the Google APIs!
ARX Development: Classes
The ObjectARX sdk is what you will need if you are implementing your app as an arx extension
to AutoCAD.
http://usa.autodesk.com/adsk/servlet/index?id=773204&siteID=123112
11
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
4. Learning objective 1: Google Docs
In this section, we will use the client apis to access Google spreadsheets. The AutoCAD entity
we will use is a table entity. We could chose to map spreadsheet data to any other AutoCAD
entity/object as well. A Table entity is the most natural mapping to a spreadsheet table, of
course.
Google Account
You will need to create a google account if you don’t already have one.
http://accounts.google.com
Registering an Application Id
In order to implement a client side app accessing the apis, you will need to register the app.
Once logged in to your account, if you go to
https://code.google.com/apis/console
you will be able to create an app id. You can create a project, i.e.
Where you will get to select services you want access for.
12
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
We select the Drive api.
Next you click on the
ApiAccess
menu item, and when prompted at the next screen, pick ‘Create an OAuth2 client Id’.
Click Next, and, at the ‘Create Client ID’ dialog, select installed application and ‘other’ for
installed application type.
13
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
Creating Client ID and Secret
The following page, when you click Create Client Id is important, since this will give you the
authentication/authorization information:
If you click on the ‘Download Json’ link, you will get:
{"installed":
{"auth_uri":"https://accounts.google.com/o/oauth2/auth",
"client_secret":"IKG_U6NwJ9-VeKlAywRR1Bqm",
14
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
"token_uri":"https://accounts.google.com/o/oauth2/token",
"client_email":"",
"redirect_uris":["urn:ietf:wg:oauth:2.0:oob","oob"],
"client_x509_cert_url":"","client_id":
"750928103016.apps.googleusercontent.com","auth_provider_x509_cert_url":
"https://www.googleapis.com/oauth2/v1/certs"}
}
This JSON packet will be what is in the code, as part of what we send to authenticate the client
Scope and Google App Apis
There are two parts to OAuth2 validation, passing the credentials, and requesting scope of
access. There is a decent overview of scopes and api access at
https://developers.google.com/academy/apis/drive/drive-apps/auth/scopes
For spreadsheet specific access, we get the scopes for spreadsheets at
https://developers.google.com/google-apps/spreadsheets/
So – our Java code has the document and spreadsheet access scope as below.
private static final String[] scopes =
{
"https://www.googleapis.com/auth/userinfo.profile",
"https://www.googleapis.com/auth/userinfo.email",
"https://docs.google.com/feeds",
"https://spreadsheets.google.com/feeds"
};
At this point, you’ve registered the application, obtained the Authorization credentials, defined
the scopes you want your application to access.
You have all the pieces, finally, to implement your app!
15
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
Getting Started:
Let’s get started with implementing the Arx app. We will break the app into 4 distinct pieces:
Authorizer, SpreadSheets, JNI and arx.
App: Authorizer
The main purpose of Authorization is to obtain an ‘AccessToken’ – which is special string, that
needs to be sent with every Google API request when getting/setting resources from the user’s
google assets (in our case spreadsheets).
Authentication/Authorization is a fairly detailed topic in itself, and merits a separate class to get
into details. The reader is referred to the google link at
https://developers.google.com/accounts/docs/OAuth2
for an in depth discussion. See the section on Installed Apps for the logic our app uses.
We have implemented an OAuth2 app, based on Google samples. The app is posted as a
runnable jar googleoauth2.jar in the samples Zip file PaaSApps.zip posted as part of the class
materials.
This is implemented as a stand alone java app, with the class com.adsk.oauth2.Authorizer with
the following main public apis:
public static int createOAuth2Authorizer()
public static int setJsonCredentials(int authIndex, String jsonCredentials)
public static int setScopes(int authIndex, String[] scopes)
public static int authorize(int authIndex)
public static String getAccessToken(int authIndex)
createOAuth2Authorizer: Creates an instance of the Authorizer class, returning the id that is
used as the authIndex for the subsequent Api calls.
setJsonCredentials: Sets the JSON credential string that we obtained when we registered our
app with Google as an installable app.
setScopes: Sets the scope strings of what the app is requesting access to.
authorize: Method that performs the authorization. This will invoke the browser instance, and the
user will enter authorization information. The method performs the OAuth2 authorization, and
returns token information to the internal class. The access token is used with the Spreadsheet
connection object, and internally, is passed with every api call.
16
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
To view source of this project, this eclipse project is as a zip archive googleoauth2.zip - you can
import into Eclipse jar. It is in the samples Zip file PaaSApps.zip posted as part of the class
materials.
If you run the main() of this app, it will prompt the user to log in, and once logged in, you will see
a request for access to user information based on your app setup:
This is where your app credentials and scopes are used by the google authorization process to
gain access to a users data with their permission.
App: SpreadSheet
The Spreadsheet app builds on googleoauth2.jar to read data from the spreadsheet once
access is granted by the user. It is implemented as a runnable jar googledocs.jar, with the main
class com.adsk.googledocs.Spreadsheet, that exposes the following main apis:
17
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
public static int createSpreadsheetConnection()
public static String[] getSpreadsheetNames(int conn)
public static String[] getWorksheetNames(int conn, String ssName)
public static String[][] getContents(int conn, String ssName, String wksName)
public static int setContents(int conn, String ssName, String wksName, int rows[],
int columns[], String values[])
createSpreadsheetConnection: establishes a connection to a user’s google docs account,
returning the id used to query/set spreadsheet and worksheet information. It uses the
Authorizer’s authorize() method from the previous app to do this.
getSpreadsheetNames: returns the list of spreadsheets for that user
getWorksheetNames: returns the list of worksheets for a given spreadsheet
getContents: gets all the worksheet contents as a 2 dimensional array.
setContents: sets the contents for the specified row, column and value lists
All of the apis below use the Authorizer to get the access token and set it to the Spreadsheet
connection.
String accessToken = Authorizer.getAccessToken(mAuthorizer);
mSpreadsheetService.setHeader("Authorization", "Bearer " + accessToken);
The following classes are used to access and modify spreadsheets:
import
import
import
import
import
import
import
import
import
import
import
import
com.google.gdata.data.Link;
com.google.gdata.client.spreadsheet.SpreadsheetService;
com.google.gdata.data.spreadsheet.SpreadsheetFeed;
com.google.gdata.data.spreadsheet.SpreadsheetEntry;
com.google.gdata.data.spreadsheet.WorksheetEntry;
com.google.gdata.data.spreadsheet.ListFeed;
com.google.gdata.data.spreadsheet.ListEntry;
com.google.gdata.data.spreadsheet.CellFeed;
com.google.gdata.data.spreadsheet.CellEntry;
com.google.gdata.data.batch.BatchOperationType;
com.google.gdata.data.batch.BatchStatus;
com.google.gdata.data.batch.BatchUtils;
The class names are self-explanatory. It is best to browse the sample code to view the methods
used on these classes.
18
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
To view source of this project, this eclipse project is as a zip archive googledocs.zip - you can
import into Eclipse using the Menu selection: File  Import  Existing Project  From Archive
– and select the .zip file. The file archive jar is in the samples Zip file PaaSApps.zip posted as
part of the class materials.
The api protocol is documented at
https://developers.google.com/google-apps/spreadsheets/.
Where the reader is encouraged to look at the samples for the various classes used. It is
instructive to note that – both the Java and .Net apis are built on top of the same basic
atom/feed protocol:
If you execute the runnable jar googledocs.jar, its main will does a few basic tests – including
connecting to the users account, displaying the spreadsheets and modifying a couple of cells.
For more details, look at the com.adsk.googledocs.SpreadSheet main() method in the sample
application.
You need to export the projects as runnable jars for the next step of being able to load these
applications in your C++ application.
19
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
For this choose File Export  and pick runnable jar.
And on the next button, pick the ‘copy the required libraries into a sub folder next to the
generated Jar’ option.
This will create a .jar file with the libraries it uses available.
App: JNI
To connect the Java apps to C++, which we need to do in order to implement our Arx
application, we use JNI (Java Native Interface).
20
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
The command jnip prints out the signatures of the exported functions you will call from C++, e.g.
C:\juno_workspaces\googledocs>c:\Java\jdk1.7.0_07\bin\javap -s classpath ./bin
com/adsk/googledocs/SpreadSheet
Compiled from "SpreadSheet.java"
public class com.adsk.googledocs.SpreadSheet {
…
public int registerCredentialsAndScopes(java.lang.String,
java.lang.String[]);
Signature: (Ljava/lang/String;[Ljava/lang/String;)I
…
;
If you look at the posted sample in the samples Zip file PaaSApps.zip posted as part of the
class materials the file gdocsjni.cpp has the following apis using namespace std;
int gdocCreateSpreadsheetConnection();
int gdocGetWorksheets(int conn, const string& spreadsheet, vector<string>& names);
int gdocGetSpreadsheets(int conn, vector<string>& names);
int setWorksheetContents(int conn, const string& ss, const string& wks,
const vector<int>& rows, const vector<int>& cols, const vector<string>&
values);
int getWorksheetContents (int conn, const string& ss, const string& wks,
vector<vector<string>>& contents);
which we will use to list spreadsheets and worksheets, get a worksheet’s contents, and set it.
To run the JVM in the same process, we use the
21
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
AcDbTable apis.
The AcDbTable entity is what we will use to represent the spreadsheet data. The AcDbTable
entity has a rich set of apis.
The reader is referred to the AcDbTable::setSize(), AcDbTable::setValue() and
AcDbTable::value() methods as a method to get/set the Spreadsheet data to an
AcDbTable entity.
The following are code fragments for two Arx Commands that gets and sets a AcDbTable
entity from a Google Docs spreadsheet. The full code sample is posted <here>.
Getting the spreadsheet size and contents from the Google Spreadsheet:
static void getTableInfo()
{
AcDbTable* pTable = getTable();
. . .
int stat = gdocGetWorksheetContents(gConn, cstringToUTF8(css),
cstringToUTF8(wks), contents);
int numRows = (int) contents.size();
int numCols = (int) contents[0].size();
err = pTable->setSize(numRows, numCols);
for (int r = 0; r < numRows; r++) {
for (int c = 0; c < numCols; c++) {
wchar_t* cellValue = utf8ToWchar(contents[r].at(c).c_str());
pTable->setValue(r, c, cellValue);
}
}
. . .
}
Setting the spreadsheet size and content from the AcDbTable entity:
static void setTableInfo()
{
AcDbTable* pTable = getTable();
. . .
int numRows = pTable->numRows();
int numCols = pTable->numColumns();
if (numRows > 0 && numCols > 0) {
vector<int> rows;
vector<int> cols;
vector<string> values;
22
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
for (int i=0; i < numRows; i++) {
for (int j=0; j < numCols; j++) {
rows.push_back(i + 1); // Indexing starts at 1
cols.push_back(j + 1); // for google spreadsheets
AcValue val = pTable->value(i, j);
val.convertTo(AcValue::kString,
AcValue::kUnitless, true);
const ACHAR* pStrVal;
val.get(pStrVal);
AcString acStr(pStrVal);
string sVal = acStr;
values.push_back(sVal);
}
}
. . .
int status = gdocSetWorksheetContents(gConn, cstringToUTF8(css),
cstringToUTF8(wks), rows, cols, values);
}
pTable->close();
}
Putting it all together
For this class’s demo – I’ve implemented two ARX commands GDOCGETTABLE and
GDOCSETTABLE that call the functions getTableInfo() and setTableInfo() illustrated above that
get and set data from {user, spreadsheet, worksheet} combination. To determine the user and
default spreadsheet/worksheet – the command GDOCSHEETINFO sets up the connection
(gConn in the above example) by connecting to the users google account through OAuth2, and
listing the users spreadsheets and worksheets. It sets the default spreadsheet/worksheet as the
first one it finds. The JNI functions gdocCreateSpreadsheetConnection(),
gdocGetSpreadsheets() and gdocGetWorksheets() are used to set the defaults.
To Recap the steps:
1. We downloaded the JDK, downloaded and installed the libraries and SDKs,
installed Eclipse and the App Engine plugin for eclipse.
2. We registered our app id in our google account, generating the client secret json.
3. We imported or created two eclipse Java projects:
a. The OAuth2 project
b. The google docs project
And exported two runnable jars, selecting the option to copy referenced libraries
to a subdirectory.
4. We created a VS2010 JNI project – with the C++ access to the java APIs, making
sure the .jars created in step 3 are loaded.
23
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
5. We created an ARX app, incorporating the JNI apis, and commands to get/set the
spreadsheet data to the AcDbTable entity.
6. We set NEXTFIBERWORLD to 0 in AutoCAD, quit, and noted in the next session,
FIBERWORLD was 0. We then ran the app.
5. Learning objective 2: Googles Data store
Setting up a Google App Engine account
The App Engine is where we run the server part of our application on the Google platform.
https://developers.google.com/web-toolkit/doc/latest/tutorial/appengine
This gives a step by step procedure of creating an account and the console.
Anatomy of an App Engine app
We will use eclipse for creating the Java based App Engine app. If you’ve installed the GAE
plugin, under File  New  Project – if you pick ‘Google’ under that you will see ‘Web
Application Project’. The details on the Java app engine project are in the link above. We use a
non GWT based application for our example.
High Replication Data Store
In this exercise, we will be using Google’s so-called ‘High Replication Data Store’ NoSQL data
base to store information. See
https://developers.google.com/appengine/docs/java/datastore/
for details and an overview.
Key concepts for the store are the notion of
Entities – these are the referenceable objects that hold properties
Properties – these are named properties and entity has
Kind – this qualifies the type of entity being created
Key - this is an object that lets you reference an entity.
Details on the High Replication data store is a large topic in itself, beyond the scope of this
class. It’s worth pointing out a few aspects, however.
24
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
Data Storage and retrieval , De Normalization, cost issues
In traditional databases, one of the principles is to normalize data, i.e. avoid data copy or
duplication. With stores such as the high replication store, we need to be concerned about
costs, and retrieval and query costs are more than storage costs especially if done frequently so sometimes, storing a copy of data is not a bad thing. It’s also faster to retrieve a single Entity
with many properties than breaking up that data across several entities, retrieving all of them.
Spatial locality and efficiencies
There are many articles on how to optimize for scalable applications. The following link is a
source for such information.
https://developers.google.com/appengine/articles/scaling/overview
One of the points worth noting from the above discussion is the notion of an “Entity Group”.
When you instantiate an Entity in the Store, if you supply a parent key – that creates the Entity
in a ‘group’ under the parent. This means that any operation on one Entity will prevent
operations on another Entity in the same group until the operation on the first one completes.
This enforces transactional consistency, though does cause contention if simultaneous updates
are common.
Query
The following link explains the query mechanisms for the Store.
https://developers.google.com/appengine/docs/java/datastore/queries
In our application, we use very simple queries based on common ancestor.
The getAllOwnees() method of com.adsk.logger.store.AcQuery in the sample code in the
project archived as googleappeng.zip implements the query. Included is the code below,
illustrating how easy it is to query the data store for all Entities of a certain type owned by a
specified entity .
import com.google.appengine.api.datastore.*;
public
{
Iterable<Entity> getAllOwnees(Key ownerKey, String kind)
Query entsQuery;
if (kind != null)
entsQuery = new Query(kind);
else
entsQuery = new Query();
if (ownerKey != null)
entsQuery.setAncestor(ownerKey);
25
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
PreparedQuery pq
= gStore.prepare(entsQuery);
return pq.asList(FetchOptions.Builder.withDefaults());
}
6. Learning objective 2: GAE
In this exercise we will implement an Arx application that uses Googles Data Store to represent
references between drawing files. While the item in the example is a dwg file – this app can be
modified to represent references between any types of file or non file data, as we will see.
Defining the reference problem
If we define an item with two strings:
ContentId. This would be a signature of the content of the data. In the case of a file – it
could be a digital hash, or, as in our case, for a dwg – we will use the version GUID. For a dwg,
if it is saved through RealDWG or AutoCAD - it is guaranteed that he Guid is changed. So two dwgs with the same Version GUID are guaranteed to have the same content.
LocalPath. This is the file name.
We can represent an item and its references as
{ c, l, {{c1, l1, c2, l2 …, cn, ln}}} where c, l are the content id and local path of the item, and the
list c1,l1, c2, l2 etc. are the items referenced.
If we enter this item and its references into the data store, where we use the <contentId,
localPath> as the key for the datastore Entity, with children as its references, and at the same
time, for each reference, store an Entity that corresponds to the reference, adding a child that is
the entity referencing it. i.e. if we get two items and their references as
{c1,l1, {{cref1, lref1}, {cref2, lref2}} and {c2, l2, {{cref2, lref2}, {cref3, lref3}}}
In the Datastore, for the first item we create 3 entities with lines representing ownership (i.e. the
children are created in the Datastore with the entity {c1, l1} as the ancestor). At the same time
we add the Entities with the referenced items as root and the original item as the child. In the
diagram below, Entities are represented by circles, and ancestor relationship by arrows.
26
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
cref1,
lref1
cref2,
lref2
cref1,
lref1
c1, l1
cref2,
lref2
c1, l1
c1, l1
Now, when the item {c2, l2, {{cref2, lref2}, {cref3, lref3}}} is added, the since Entity {cref2, lref2}
is referenced both by {c1, l1} and {c2, l2}, {cref2, lref2} gets an additional child {c2, l2}. I.e.
cref2,
lref2
c1, l1
c2, l2
What this now gives us is the ability to find out, given an item (or file) – what are the
referencers of the item.
In our example, we store the additional information of timestamp and region (country, state,
city) from which the item is added to the data store. With this information stored in the entity it is
possible to perform the following queries:
1. List all the items that have different local paths (or names) but identical content. I.e. list
duplicates
27
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
2.
3.
4.
5.
List all items referenced by a particular item.
List all items referenced from a particular item.
Qualify queries 1-3 by Date/Time. I.e. between specific dates.
Qualify queries 1-3 by region. I.e. city/state/country.
So this application can be used as an automatic auditing/tracking mechanism of content and
references.
Note we haven’t talked about deletion of data. To keep things simple, one approach is to purge
Entities from the database for a given <contentId, localPath> combination that are older than a
given date, keeping the latest version of the <contentId, localPath> items – both as the
reference root and referencee root. If you look at the AcQuery methods:
public Iterable<Entity> getObjectsByProperties(
Key ownerKey,
String kind,
Map<String, Object> props)
and
public void deleteObjects(Iterable<Key> objKeys)
With these two methods, delete operation in a REST api that, given a timestamp interval
deletes all the root entities can be implemented.
Classes to store reference
The Eclipse project googleappeng contains the class LoggerApi that implements the REST api
to create, store and retrieve the references trees. The Datastore Entities are created using the
AcQuery class apis.
Jersey and REST annotation
In order to implement the REST apis very easily, we use the Jersey library.
If you look at the LoggerApi class, you will notice that the rest Apis are implemented simply as
methods:
e.g.
@Path("/adlogger/")
public class LoggerApi {
...
@POST
@Consumes("application/xml, application/json")
@Produces("application/xml, application/json")
@Path("/userslog")
public String putItem(@Context HttpServletRequest request, FileItem logInfo)
28
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
This method adds an item to the store. The annotations @POST, @Consumes, @Produces
and @path define the url and the http verb.
If you look at the web.ml file in the project under war\WEB-INF, you will see:
<servlet>
<servlet-name>Info Logger Servlet</servlet-name>
<servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer
</servlet-class>
<init-param>
<param-name>com.sun.jersey.config.property.packages</param-name>
<param-value>com.adsk.logger.binding</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>Info Logger Servlet</servlet-name>
<url-pattern>/resources/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>Info Logger Servlet</servlet-name>
<url-pattern>/rest/*</url-pattern>
</servlet-mapping>
Which tells the app engine that the servlet for the Jersey is the handler at the
http://acadgae.appspot.com/rest
and – the rest of the path i.e.
http://acadgae.appspot.com/rest/adlogger/userslog
is defined by the annotation on the LoggerApi class and the putItem method.
Note that the following in web.xml:
<security-constraint>
<web-resource-collection>
<url-pattern>/rest/*</url-pattern>
</web-resource-collection>
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</security-constraint>
Also permits secure access to our REST api via https – which is what the client uses.
The annotation element @POST indicates that when a POST is issued at the above URL,
putItem is called.
29
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
Serialization/Deserialization
In order to pass the file item you might have noticed the @Consumes annotation. This says that
the method putItem() takes as the http body of the post either xml or json. The Jersey
framework parses that into the FileItem argument to the putItem method.
In order to be able to do this, the FileItem class is annotated as:
@XmlRootElement(name = "FileItem")
public class FileItem {
Where the annotation @XmlRootElement makes this Java object serialized to xml via the javax
xml binding.
By just annotating the object, the Jersey libraries takes care of parsing the XML from http – and
all we have to do is program in POJO (Plain Old Java Objects!).
Client side
On the client side, we use Jersey as well. If you look at the googleappengclient eclipse project,
the methods
createLogItem
getDuplicatesOf
getReferences
getReferencesTo
Implement REST calls using Jersey WebResource and Client objects. These classes make it
very easy to build and issue Http GET, POST, DELETE and PUT requests.
JNI
Similar to the google docs api interfaces, interface to the logging methods is through jni calls in
gdocsjni.cpp in the gdocs project.
The methods
string
string
string
string
string
gaeCreateLogItem(const string& xml);
gaeGetReferences(const string& contentId, const string& localPath);
gaeGetReferencesTo(const string& contentId, const string& localPath);
gaeGetDuplicatesOf(const string& contentId);
gaeDeleteLog();
in gdocs.cpp are invoked to implement the arx command that interface to AutoCAD.
30
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
AutoCAD Xref graph
The AcDbXrefGraph represents all the xrefs referenced from a host drawing.
We use this to build the FileItem object that is sent to the GAE service. The AcDbXrefGraph is
obtained by acedGetCurDwgXrefGraph() – and for just top level references, the root nodes out
going references will give the desired list of references.
Putting it together
In the gdocs.arx app, we implement commands
GAELOGREFS, GAEGETREFS, GAEGETREFSTO.
Command GAELOGREFS logs a FileItem to GAE – with the host drawing as the root, and all
the immediate references as the FileItem references. GAEGETREFS and GAEGETREFSTO
when invoked, use the current drawing to determine which are its references, and which other
files reference it respectively, based on the data posted to GAE.
So to summarize, the steps were:
1. Create and setup an Eclipse project for GAE, including installing the GAE sdk and
the GAE plugin.
2. Implement, using Jersey, REST apis that log a FileItem and its references in the
High Replication Data Store.
3. Deploy that app to GAE from within eclipse.
4. Implement a Java client app using Jersey to invoke the REST apis to put a
FileItem, and get references and references to a file item.
5. Implement a JNI layer to invoke the Java client apis
6. Integrate the JNI calls into the ARX app that implements the GAELOGREFS,
GAEGETREFS and GAEGETREFSTO commands.
7. Run the arx app, making sure FIBERWORLD is 0.
At this point – you have now created and ARX app that logs reference information to Googles
High Replication Data Store!
The samples, including prebuilt jars, for this application are in the samples Zip file
PaaSApps.zip posted as part of the class materials.
31
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
7. Learning objective 4: Metering and costs
While in this class we run with in the ‘free’ limits for both the google docs apis and the Google
App Engine apis, there are resource limits for both.
Google APIs and Limits
For the google apis – under the https://code.google.com/apis/console, you can navigate the
resources available for each project you have:
Some api services actually require google checkout/billing. Others require specific requests for
increasing quota.
Google App Engine
Google App engine has many categories of resources including:




Instance Costs
Bandwidth Costs
Database Operation costs
Storage Costs
32
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
The following page https://cloud.google.com/pricing/ discusses cost issues.
Once you create your App Engine account, you can view the resources consumed by your
application, as well as detailed runtime information about your application. Your console
dashboard at https://appengine.google.com/dashboard should look like:
33
PaaSt the Desktop: Implementing Cloud-Based Productivity Solutions with the AutoCAD® ObjectARX® API
Where you can navigate detailed usage and billing information.
Once again, the samples are posted in PaaSApps.zip.
If you have further questions – I’d be happy to answer – my email is
ravi.krishnaswamy@autodesk.com.
34