Technical Strategy and Testing
— X Projects: Kuali Student
Services Implementation
Experiment Initial Findings
Prepared by
Next-Generation Student Services System Team
February 17, 2016
Student and Administrative Information Systems
Information Services and Technology
Document Change Control
Version
0.1
0.2
0.3
0.4
0.5
0.6
0;.7
Date
3/30/2009
3/31/2009
3/31/09
4/5/09
4/9/09
4/9/09
5/5/09
Authors
Catherine Iannuzzo
John Harmon
Catherine Iannuzzo
Catherine Iannuzzo
Catherine Iannuzzo
John Harmon
Catherine Iannuzzo
Summary of Changes
Initial Draft
Updated document formatting
Minor additions
Rearranged outline
Clarified assumptions & conclusions
Minor revisions
Clarified programming issues
Document Management
Role
Owners
Editors
Contributors
Reviewers
Approvers
Names
Catherine Iannuzzo
Catherine Iannuzzo
Catherine Iannuzzo, Jeffrey Walker
Suresh Damodaran, John Harmon, Catherine
Iannuzzo, Jeffrey Walker, Kuali Student services
team, NGS3 Program team
Dean Briggs, JoAnne Stevenson
Notes
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
Contents
DOCUMENT OVERVIEW ........................................................................................... 2
Document Purpose ..................................................................................................... 2
Document Scope ........................................................................................................ 2
INITIAL FINDINGS ..................................................................................................... 3
Introduction ................................................................................................................. 3
Assumptions ............................................................................................................... 4
Summary of Initial Findings ........................................................................................ 4
Release Process......................................................................................................... 5
Interoperability ............................................................................................................ 5
Security ....................................................................................................................... 6
Performance ............................................................................................................... 7
Programming Practices .............................................................................................. 7
Details of the Person Service ..................................................................................... 8
The Functions ............................................................................................................. 9
Setup .......................................................................................................................... 9
Read ......................................................................................................................... 11
Search ...................................................................................................................... 11
Maintenance ............................................................................................................. 12
The Data Structures ................................................................................................. 12
Conclusion ................................................................................................................ 13
Going Forward .......................................................................................................... 14
Appendix 1: Name/Value Pairs in an RPC ............................................................... 14
Modification Date: 2/17/2016
Page 1
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
DOCUMENT OVERVIEW
Document Purpose
The purpose of this document is to summarize the findings of the X1 release, a 3-month
project that kicked off the Technical Strategy and Testing — X Projects. One of the goals of
the X Projects has been to test a possible migration strategy for the MIT Student System
which would involve wrapping the existing MITSIS database with Kuali services. The project
wanted to test the feasibility of this approach, as well as provide technical feedback to the
Kuali Student team about the services and the experiences using them.
Document Scope
This document:


Provides an evaluation of the Kuali Student “Person” service evaluated in the X1
release
Includes feedback on the components available from November, 2008 - February,
2009.
This document does not:


Cover updates made after February, 2009
Evaluate Kuali Student service components other than the Person service used for X1.
Modification Date: 2/17/2016
Page 2
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
INITIAL FINDINGS
Introduction
The MIT Student System VISION project anticipated that new architecture and technologies
would be needed to support the Next-Generation Student Services System (NGS3), and this
future may include Kuali, a community source student system. The purpose of the set of “X”
(for “experimental”) projects is to begin identifying the technical risks, issues and challenges
that will be involved with implementing a new enterprise student system. The projects are
intended to provide input into the NGS3 architectural, technical, business and implementation
strategy and planning process. There will be a series of X Projects over several years as
various components, tools, technologies and methods are tested in our development
environments. While the X Projects do not necessarily lead to direct deployments of
production system components, what is learned from these projects will reduce risk, improve
estimation efforts, and smooth the course of future production development and
implementation projects.
One of the goals of the X Projects has been to test a possible migration strategy for the MIT
Student System which would involve wrapping the existing MITSIS database with Kuali
services. The project wanted to test the feasibility of this approach, as well as provide
technical feedback to the Kuali Student team about the services and our experiences using
them. This report summarizes the findings of X1, a 3 month project that kicked off the “X
Projects” strategy.
For X1, we selected a simple, read-only use case to test the use of a Kuali service wrapping
MITSIS: implementing a portlet that performed a simple Person lookup. The request returns a
user's complete name, email address, telephone, department, job title (if an employee) or
class year (if a student) in response to an input of a name or partial name string. This is
similar to the functionality provided by the person lookup feature on the current MIT home
page. We chose to implement a part of the Kuali Person service to back this test application,
and wrote the service implementation against the existing MIT legacy student data base.
Because our goals for X1 were modest, our findings will mostly address the developers’
experiences in attempting to do two things: write a service that met the Kuali contract for the
methods we needed to fulfill the use case, and write a simple portlet application that
consumed the service. Our analysis will not address such issues as whether the service is
complete, whether the Kuali services in general are sufficient to map MIT’s domain, or
whether the service will support all the business processes which might want to use it. This
requires a detailed analysis which we have not undertaken. Our focus is primarily technical
and focused on the software developer.
Modification Date: 2/17/2016
Page 3
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
Assumptions
To clarify the context of our findings, the MIT team has approached the Kuali services with
the following assumptions:







All Kuali services are intended to be implemented as SOAP web services, and
accessed by RPC over a network connection.
Each Kuali service will have a unique URL, and all the URLs cannot be assumed to
reside within the same firewall or VPN.
Kuali will deliver client modules that implement business functionality as well as
service implementations to support the client module.
One service implementation could be replaced with another one when a participating
institution configures a Kuali client module for deployment. This configuration would
not require code modification or rebuilding of the module.
MIT has existing authentication, authorization, and group management services and
these will not be replaced in the foreseeable future. Any new product must be able to
integrate with these existing services.
MIT’s migration strategy from its legacy student system to a new student system will
involve wrapping legacy components with new interfaces until such time as the legacy
components can be completely replaced.
Kuali is providing a set of standards to which member institutions must conform in
order to obtain interoperability between components, as well as implemented code
artifacts that will be developed, tested, supported, deployed, and put into production by
member institutions.
Summary of Initial Findings
We received a great deal of help from the MIT Kuali team, who were very open with
information. They assisted us with filling in missing parts of the specification and applied their
domain experience with MIT’s legacy system to help us integrate with it. The work would not
have been possible without their help, and particularly we owe a great deal to the
contributions of Norman Wright on the MIT Kuali team.
Our experience with Kuali Student services so far indicates that there are areas where Kuali
as a code base and a program still need to mature. The following are some key areas where
Kuali has not yet defined a clear process or approach:





Release Process
Interoperability
Security
Performance
Programming Practices
Modification Date: 2/17/2016
Page 4
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
Release Process
“Contract first” is the principle of the Kuali service architecture. This implies the corollary that
SOAP development will be “WSDL first.” However, Kuali has not yet created a process for
publishing industry standard WSDL so that potential implementers can use it. The current
process is one that might be used by a small, informal team in research mode: there is a
program that scrapes the Kuali Student wiki pages which contain the service specifications
and generates Java classes from them. This program was run for the team by the MIT Kuali
team and the Java classes provided to us. The WSDL was generated from the Java classes.
The way that we obtained the WSDL highlights that at that time there was no clear definition
of what constitutes a “release.” Besides clarifying the artifacts of a release, the release
process must have a clear articulation of dependencies upon the artifacts of previous
releases, with a method of versioning, name space definition, backward and forward
compatibility requirements, etc.
The WSDL is insufficient to fully express the service contract so that it can be implemented or
used. The contract has many areas where Type Keys are expected to define the actual data
and functions, and actually specifying these Type Keys is not part of the contract. (A Type
Key is a string identifier which is used in many operations to qualify the operation in some
way.) This information has been delegated to a document called a Service Implementation
Specification (SIS), which describes the Type Keys and attributes (dynamic data fields).
Since these details were less clear when the services were being designed, this was
expected to take place in a later phase. Very few of these have been finalized, so in order to
implement the Person service the team relied upon Norm Wright of the MIT Kuali team to
suggest Type Keys for our implementation of the Person service. Norm also provided an
invaluable service in mapping the attributes to the MITSIS database, without which no work
could have been done. It needs to be clear to others what a key role the SIS plays, and this
needs to be governed as carefully as the WSDL in order to make a complete specification.
Kuali’s view of the role of a reference implementation in a production-ready application is not
obvious to us. What is the Kuali definition of “standard” – is it the service definitions, or the
application(s) that implement these definitions developed by Kuali? If the standard is the
service definition, then the application becomes a reference implementation that best shows
how to use the service definitions. If the application is the standard, then the service
definitions become “current interfaces for this release of the application,” which may be
subject to change without notice. Which path is taken will have ramifications for all member
organizations and for MIT’s dependency on Kuali.
Interoperability
Service implementations are viewed as the “field replaceable units” of the Kuali system. It
should be possible for a client application to plug in another service implementation besides
the Kuali reference one. For example, MIT has its own authentication and authorization
services already and would like to continue to use them with new products. However, Kuali
has not specified exactly how this interoperability is to take place.
Modification Date: 2/17/2016
Page 5
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
As previously described, WSDL alone is insufficient to fully specify a service; to interchange
two services both must meet the same Service Implementation Specification. Beyond this, it
is not clear how an application performs the substitution. For example, a client application
using a remote service usually has stub classes of some sort, generated by a SOAP toolkit.
Will these stub classes be the same regardless of service implementation? If they are, then
how does the stub class find and use a remote service which it does not know about at build
time? The client must be written in such a manner that it has no dependency upon a fixed
URL and has a well-understood way of configuring its dependencies when it is deployed.
How does a client application validate that it is connecting to a version of a service that is
compatible with the version for which the client was written? Even if WSDL versions match,
Service Implementation Specifications must also match. Similarly, how does a service
validate that client is compatible? In most remote protocols, this is a negotiation process.
What are the requirements for backwards and forwards compatibility on the part of services?
Some service implementations will be wrapping legacy systems or databases, so
assumptions cannot be made about a common database. In order to be interoperable,
service implementations must have some kind of certification test suite that validates that
they meet a specified version of the standard. These tests should be available to all members
to certify their own work or the work of third parties, and the tests should be part of the
release process.
Security
By “security,” MIT is referring to n-tier connections between client applications and service
applications which are required to secure potentially sensitive data passing back and forth
and provide identity management information from the client application to the service
application in a stateless manner.
The service contracts are implementation independent and do not contain any specifications
for a remote service protocol. Although it is assumed that the services will be implemented
using SOAP (and specifically the Apache CXF toolkit), there are no Kuali standards for how
the services will be secured. For example, the WS-Security specification is broad enough that
it is necessary to spell out what kinds of security tokens are being exchanged as part of the
service protocol in order for a client application to be able to interoperate with multiple service
implementations.
Our general policy at MIT is that the application is less trusted than the service. The service
must guarantee that unauthorized data does not leak and that the database is not corrupted.
The expectations in this area need to be made very clear in order to have truly interoperable
as well as secure services.
For X1, we avoided addressing the security issues by only accessing data that was already
public. A proposed security architecture for NGS3 will be part of the next phase of the project
(X2). We cannot advance to doing any updates to the data without a security model.
Modification Date: 2/17/2016
Page 6
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
Performance
As yet, the Kuali service specifications have not addressed any potential performance issues
which are typical concerns related to a remote API. Performance can be an aspect of Denial
of Service and can represent a security concern, as well as a user satisfaction concern.
One area of concern is the handling of large datasets, MIT has about 20,000 students and
staff. A person lookup that retrieved everyone would not work well in a user interface. Very
large result sets can cause network latency and possible Denial of Service, so page caching
cannot be delegated solely to the client application layer. As yet, there is no approach
defined on the service side. One of the standard ways of addressing this problem is for
services to provide page caching. To do this, the API must allow for the service to return a
token to the client which can be presented with future requests from the client for more
pages. Even though this approach involves a series of exchanges, services must use these
tokens in stateless manner – it cannot be expected that the client application has an HTTP
session with the service that lasts longer than one request/response cycle. Services would
have to be responsible for managing, timing out, and cleaning up their results caches based
on configuration parameters. Regardless of the details, the specifications for this behavior
need to be part of the service contract and should conform to a standard pattern.
There is a common Kuali service pattern of returning object IDs in response to function calls
like “SearchForPerson.” In order to use the resulting list of IDs in most applications – for
example, to display a list of person names – the programmer would have to make individual
calls to fetch the actual information for each person. We did not implement any of these
functions and so cannot provide concrete data on it, but it would seem to be a potential
performance problem.
This raises a potential concern about the granularity of the service API. Too fine-grained an
API causes an excess of network traffic and slow application behavior. Too coarse an API
can result in “god services” that handle too much functionality and are not easily replaceable.
We have no definitive feedback on this area as yet apart from the points made above, merely
a concern that we expect to test further in the future.
Programming Practices
From the software developer point of view, using and implementing the Person service in its
current state involved daunting programming awkwardness. The key issues are:

The generic nature of the interfaces (specifically wide use of lists of name/value pairs)
requires an extra layer of marshalling over a remote procedure call, and moreover,
essential information about the data (like data types) is lost in the process of going
through this extra marshalling. The client has no way of guaranteeing that the
reconstructed object is correct. This extra marshalling phase is not supported by any
common toolkit and must be custom built by the programmer on both sides of the wire.
Modification Date: 2/17/2016
Page 7
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings

There is no object inheritance that allows sub-classes of common classes to be easily
created and used. The Kuali approach of using “type keys” to distinguish different subtypes of a generic object, instead of using sub-classes (or sub-interfaces) as would be
done in a typical object-oriented environment, forces the programmer to act like a
compiler, performing type-checking and similar operations at run-time through
manually written custom code. The extra burden placed on the programmer by this
means that these interfaces are unlikely to be willingly adopted by working
programmers creating applications on a deadline.

The objects do not leverage the capabilities of relational databases, and require
programmers to manually perform joins by going through a result list a single object at
a time. This is both slow and a programming burden that will discourage developers
from adoption.

We have concerns about the granularity of the services, but as yet no final
recommendations. The services are rather fine-grained, representing typical CRUD
(Create/Read/Update/Delete) operations on data elements, plus Search and
dictionary-type functions (Setup). They are much more like data entities than business
services, and therefore represent the types of objects normally used in a persistence
layer of an application. Business services, as typically used in an enterprise, represent
larger-grained operations between systems. MIT’s production student system currently
has interfaces with about 40 applications: in a SOA enterprise, these connection
points would likely be clues to business-level service points. Programmers are unlikely
to see much benefit to manipulating data entities remotely over a SOAP RPC. The
performance issues would be a concern, and the massive infrastructure (each single
fine-grained service running as an independent application distributed across a load
balanced server farm) would be daunting for our operational staff. It is not clear yet at
our level of exploration how the services will be put together or layered to build larger
business level components. Testing this would require a much broader business
application than we are capable of doing at this time.
MIT looks forward to seeing artifacts from the Kuali technical team that will illustrate their
programming application framework and Best Practices.
More technical details about the programming issues are provided in the next section.
Details of the Person Service
It was known at the time that we selected it that although the Person service had been
through several drafts, it was not yet finalized. This was a risk that we were willing to take, as
no Kuali services were truly “final” at the time that we began this project. We chose this
service because we could make a simple application using only the one service with no other
dependencies.
Modification Date: 2/17/2016
Page 8
Technical Strategy and Testing — X Projects: Initial Kuali Findings
NGS3, SAIS
The Person service is currently described in the Kuali Student specification 1(available on the
Kuali Student wiki) as “The Person services supports the management of people related to
Kuali Student. This includes managing person to person relationships, but does not include
relationships of a person to a non-person entity (like an organization). Although generalized
information such as academic or contact information may be available through this service,
manipulation of this information is handled through the appropriate service of record. The
concept of attribute sets allows for the grouping of person attributes. Once defined, attribute
sets can be associated with the person types to identify the relevant information for a given
person. A person will always have one or more person types assigned.”
The Functions
All Kuali services have the same general pattern. They have groups of functions for Setup,
Read, Search, and Maintenance operations that operate on flat data structures. The data
structures can be expressed as XML and in a Java environment they can be expressed as
simple Java beans.
Setup
The Setup functions are used to discover what a service has for Type Keys. A Type Key is a
string identifier which is used in many operations to qualify the operation in some way. These
type keys are vital to the actual functioning of the service.
Each service has three areas of "setup" functions:
 Dictionary Operations (FindObjectTypes, FetchObjectStructure, fetchEnumeration),
 Search Operations (findSearchTypes, fetchSearchType, findSearchTypesByResult,
findSearchTypesByCriteria, findSearchResultTypes, fetchSearchResultType,
findSearchCriteriaTypes, fetchSearchCriteriaType), and
 Domain Operations.
The functions in the first two categories are the same for all services, and the last one is
specific to the service. In the case of the Person service, the Domain operations are: 2
 findPersonTypes,
 fetchPersonType,
 findPersonTypesByKeyList,
 findCreatablePersonTypes,
 findPersonUsageTypes,
 fetchPersonUsageType,
 findPersonUsageTypesByKeyList,
 findPersonUsageTypesByPersonType,
 findPersonPersonRelationTypes.
1
Kuali wiki: https://test.kuali.org/confluence/display/KULSTU/Person+Service
Note that this represents the state of the service at the time that we derived the WSDL; it is not the
current state of the service as of the writing of this document.
Modification Date: 2/17/2016
Page 9
2
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
“Person” is a difficult concept in an educational institution, since the odds are high that the
different kinds of “person” are managed by different systems of record and depending on the
context, different information may be needed or available. For example, “student” is usually
part of a student system, but “instructor” or “advisor” are employees and may be part of an
administrative system handling Human Resources and payroll. The use of Person Type Keys
is intended to help with this problem, allowing for the definition of various types of Person.
The Usage Type Key is intended to address the problem of contexts: different information
may be wanted based on whether the query is about a student being assigned a dormitory, or
an advisor getting student grades.
Although the use of Type Keys allows a great deal of flexibility, the approach exhibited by the
Setup functions raised questions for the programmers working on the application that would
consume Kuali services, because it implies that at the time the client program is being
written, the programmer will not know such basic things as:
 what types of objects the service has,
 what types of searches it can do,
 what criteria are needed for the searches,
 what results the searches return,
 and in the case of the Person service,
 what kind of Persons there are,
 how they can be used and
 what joins are possible between Persons.
All this information would have to be dynamically discovered at runtime. No one on the team
could conceive of how to create such a dynamic UI without some daunting heuristic analysis
and a lot of presentation metadata. If it were possible obtaining presentation metadata from
the service would violate the deeply ingrained Model-View-Controller software design pattern.
What programmers expect instead of runtime information about types is definitions of
interfaces that make use of normal object inheritance, as it is done in commonly accepted
object-oriented programmer practice. For example, a “LearningUnit” interface might have the
sub-interfaces of “Course”, “Subject”, “Program”, etc. These sub-interfaces would inherit from
the parent interface whatever is common to all, and would subclass and extend to create
more specific objects. Objects of this type are what programmers expect and find easy to
use. Kuali essentially moves the sub-classing process to runtime and turns the human
programmer into a compiler; the application code itself becomes burdened with doing the
work that should be done outside of the application by the IDE when it is built. Few
programmers will want to be deprived of the productivity enhancement of using their normal
tools to perform these kinds of processes.
The Kuali Setup approach represents a style dictionary-driven programming, which contrasts
with the visual, user-oriented style of programming currently practiced by most Java
enterprise programmers. Most UI design involves a great deal of study with end users to
understand their tasks, how they react to the placement of certain information in a display,
and so on, and these designs are worked out well in advance of actually programming them.
The setup functions required by this architecture may be present to allow for future flexibility,
but unless client applications are written so as to create their UIs at runtime, they will be
Modification Date: 2/17/2016
Page 10
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
unable to take advantage of that flexibility. The MIT team is very interested in seeing how the
Kuali technical team approaches this.
Read
We did not exercise any Read functions. The Read functions are Fetch operations that are
essentially special cases of Search, where the criteria and results are very well defined.
However, none of these functions met our needs for the use case we defined.
Search
The X1 team spent most of its time on Search. The functionality of Search is very much
dependent upon the definition of Search Type Keys, Criteria Type Keys, and Result Type
Keys. Without these, the functions cannot be used. Since the Kuali Student Person service is
not yet complete, Norm Wright of the MIT Kuali Team compiled some suggested Type Keys,
based on our needs and his knowledge of MIT's legacy system.
Search is a tough problem in general. There are three search methods defined:
SearchForResult,
SearchForPeople,
SearchForPersonPersonRelations.
We used only SearchForResult. The other two searches are more structured in terms of
criteria and results. The results returned by the SearchForPeople and
SearchForPersonPersonRelations methods are all object Ids (either Person or
PersonPersonRelation), and therefore to obtain the actual data that these IDs represent
requires the programmer to go through the list of IDs and fetch the Info objects for each one
individually. This is a case of a programmer doing a job that generally is done more efficiently
by a relational database.
SearchForResult is a generic querying mechanism available from all services that can return
name/value pairs of any kind. They do not have to represent fields in any object that the
programmer already knows, and in fact most of the data we wanted to see is not specified as
being explicitly part of the PersonInfo, which does not contain any "contact" information (like
address, telephone, etc). This sort of information is intended to be managed by a separate
Contact service. However, looking up a person's contact information is one of the most
common cases of interacting with Person information, and because SearchForResult is so
loosely defined, it is possible to return contact fields as a result of a Search operation.
If the application programmer abides strictly by the dictionary-oriented approach, then the
Search Type Keys, Criteria Type Keys, and Result Type Keys must be examined at runtime
to find out what criteria to collect from the end user and what kind of data is being returned.
This makes it difficult to develop a UI, as is further described below, and in our case we did
not use this approach. Instead, we agreed ahead of time what Search Type Keys we had,
Modification Date: 2/17/2016
Page 11
Technical Strategy and Testing — X Projects: Initial Kuali Findings
NGS3, SAIS
and the application directly asked for a specific kind of search, knowing at design time what
the criteria and results would be.
Maintenance
Maintenance refers to Create, Update, and Delete functions. We did not use any
Maintenance functions as our use case was Read Only.
The Data Structures
The data structures in Kuali services are entities that can be expressed as XML schemas and
be converted into Java beans for use in a Java environment. The Person service primarily
handles the management of two such data entities: PersonInfo and PersonRelationInfo.
PersonInfo is defined as a set of core fields about a Person.3 Key elements are a list of
person Type Keys, names (a list), a information related to citizenship (list), visas (list),
residency (list), ethnicity (list), primary and secondary language, birthplace, a gender, marital
status and photo (which for now is described as a string with a .jpg filename; this indicates
the currently unfinished state of the service, as this field would have to be a URI in a webbased service). All data structures have some maintenance metadata, including create
timestamps, last update, etc.
The rest of the information that a student system might find interesting about a person (what
department, major, class year, job title, etc.) are assumed to be part of the dynamic attributes
or a relation with another object. Dynamic attributes are a list of name/value pairs of strings.
What list of name/value pairs is returned with any particular PersonInfo is a matter of what
PersonType and UsageType is provided as context to the functions.. The name/value pairs
have no data typing information associated with them -- they are simply strings.
One of the first reactions of application programmers (who are consuming the services) to
this heavy reliance on lists of string name/value pairs (expressable as a Java Map object) is
that the benefits that programmers expect from a strongly typed programming language are
not available to them. For example, the programmer must know and make sure that a
particular string represents a number in decimal characters, or a true/false Boolean, or a
date, etc. when normally this would be left to the compiler. An list of strings is not the easiest
data structure in which to quickly obtain a value: rather than referencing
“myObject.getMyProperty()” as one would do with a strongly typed structure, the programmer
must iterate over the list for the name desired or retrieve it from a hash map in order to obtain
the value.
There are validation APIs provided for all the data structures, but these require passing a
complete object to the remote function to get a response about its validation state. For
3
The detailed spec can be found at
https://test.kuali.org/confluence/display/KULSTU/personInfo+Structure,
Modification Date: 2/17/2016
Page 12
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
modern web-based UIs, feedback is often expected to be provided to the end user the
moment a field is filled in; this is typically done with JavaScript that is executed within the web
page in the client’s browser. Delegating the validation to a remote service is likely to have
performance issues. Other issues with using name/value pairs in an RPC are described
further below, and in the diagram in the Appendix.
The PersonRelationInfo data structure has no relationships between persons that are predefined by the service, such as advisor/advisee, supervisor/employee, etc. These are all
Type Keys that can be discovered at runtime and are to be specified in the SIS. The
“Relation” type of structure is a common Kuali pattern. It has the benefit of allowing future
extension of the types of relationships various entities can have, but at the cost of not making
it easy for the programmer to obtain efficiently. The application must perform the functions of
a relational database, by essentially performing joins between two PersonInfo objects
manually, and most programmers would rather delegate that task to a database tool.
As an example, if a user wanted to obtain a list of the names of all students who were being
advised by a particular Advisor the calls to the service (written in a Java-like pseudo-code)
would look like this:
relationInfoList = personService.findPersonPersonRelations(advisorId, "advisor");
listIterator = relationInfoList.getIterator();
while (listIterator.hasNext() ) {
new relationInfo = listIterator.getNext();.
if (relationInfo.getPersonId() == advisorId) {
new studentInfo = personService.getPerson ( relationInfo.getRelatedPersonId());
<iterate thru studentInfo.NameInfoList to find a NameInfo object of the appropriate
type>
<do something with NameInfo.fullName to build the display list>
}
}
Note that this specific example could be made more efficient by having the
“findPersonPersonRelations” method include a parameter that limits the scope and looks only
for children or parent nodes. Even if that improvement is made, however, the point is that
when the relationship mapping is embedded in the database via join tables or if it is handled
by an ORM mapping tool, then the programmer has less work to do. It will be hard to
convince programmers to give up the tools that they are used to using.
Conclusion
In summary, the MIT experience was based on the assumptions stated above: that MIT
would have a strategy of writing our own implementations of some services to integrate Kuali
modules with our legacy systems. Based on this, we have feedback in the following areas:

Release process – the top concern of MIT is that the current processes used by the
Kuali organization are not consistent with those used by organizations managing to a
standard or a “contract first” approach and need some maturity.
Modification Date: 2/17/2016
Page 13
NGS3, SAIS




Technical Strategy and Testing — X Projects: Initial Kuali Findings
Interoperability -- the strategy for building interoperable components is not yet clear to
us.
Security – as yet, there is no specification for web services security.
Performance – service specifications do not take into account the problems of large
datasets. We are also concerned about the granularity of the services when used over
a network connection to a stateless remote service.
Programmability – our programmers had a difficult time with the generic and dynamic
nature of the API, which suggested that a lot of effort would be required to build a
dynamic UI for little discernable benefit to our customers. The use of name/value pair
lists and dynamic attributes creates extra overhead for programmers, which affects
their willingness to adopt this style, and it adds inefficiency to a remote procedure
handling. We also have concerns about the granularity of the services, but no specific
findings to report in that area.
Going Forward
MIT’s hopes and expectations going forward are as follows:



MIT is very interested in getting some code artifacts, to see how the Kuali technical
team is approaching implementation and to learn what the expectations and best
practices will be.
More than code, however, MIT needs a well defined and formal release process,
where the nature of the artifacts, mechanics, and dependencies of a “release” are very
clear. This includes versioning and name space definitions for Kuali services.
MIT welcomes the opportunity to provide feedback into the Kuali process and looks
forward to the development of more formal processes for the free flow of information.
Appendix 1: Name/Value Pairs in an RPC
For this first project, we limited the scope of what we were trying to achieve with our first Kuali
service implementation. There is no security model, so we used a read-only use case with
publicly available data. There is not yet a released toolkit, so we used an informal method of
scraping the WSDL from the wiki. There was no Service Implementation Specification, so we
worked with Kuali team members to come up with a plausible set of Type Keys to support our
use case. We did not attempt to interoperate with any other service or client. Having
eliminated all these challenges, where we experienced the most difficulty was in the
name/value pair string-oriented nature of the service API.
Using lists of name/value pairs to represent data is difficult for programmers, and this caused
stress to the developers. It required an extra level of custom code transformation on both the
Modification Date: 2/17/2016
Page 14
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
client-side stub and service-side skeleton in order to transform data out of its known state into
this generic format, and then back from strings into a data object that the programmer can
easily manipulate. SOAP RPC is designed to favor a more data-structure like approach, so
accommodating the attribute list approach adds some complications to the programmer’s job.
We received comments like “unintuitive in an object-oriented language” and “you don’t know
what you have to work with.” Our main concern is that overcoming programmers’ resistance
to this approach will slow adoption, and therefore these kinds of experiences need to be
taken into consideration.
The diagram on the following page illustrates schematically the process of completing a
round-trip request to a remote SOAP service. These steps are generally universal for any
kind of RPC that must transform from a native programming language to a wire protocol and
back to a native programming language. However, the diagram highlights some of the extra
processing required when the data is specified by the service as being lists of name/value
pairs, rather than data structure-like objects. Although name/value pairs can certainly be
represented as Java objects, their use is not efficient for most programming tasks.
In the use case shown, the user has made a request through the client application. The
diagram tracks the return of data from the database through the various software
components back to the user.
1. The data begins as a result set from a database.
2. In most modern JEE/Spring programming, this result set is managed by an ObjectRelational Mapping tool such as Hibernate which transforms it into a Java bean, which
is a simple object having a number of get/set properties that represent the fields of the
data.
3. At this point, the Java bean bears a very close resemblance to the data as it came
from the database, adhering to its original data types as closely as Java can match the
database.
4. Because the Kuali specification wants much of the data returned as name/value pairs
(all of it in the case of SearchForResult), this strongly typed Java bean must be
transformed by custom code (e.g. not boilerplate code provided by a standard SOAP
toolkit) into a less efficient List object containing pairs of strings. In the process of this
transform, the original data typing information is lost and will not be transmitted to the
client.
5. The Java list object(s) are then transformed by JAXB or similar XML tools used by a
SOAP toolkit into an XML representation.
6. The XML document is wrapped in the appropriate SOAP headers and packaged in an
HTTP Response object.
7. The HTTP response containing the XML payload is sent over the network to the client
application.
8. The client-side SOAP toolkit transforms the XML into Java objects again.
9. The Java objects received by the client at this point resemble the generic Java List
objects sent by the service in step 5.
10. Generally speaking, lists of strings are not the most useful way to manipulate data in a
program, and the client side program will want some code that undoes the list format
and turns the generic object back into a data structure object. This particular code was
written by the service provider and given to the client as part of a proxy toolkit, as the
service provider knows what the data actually looks like and the client does not.
Modification Date: 2/17/2016
Page 15
NGS3, SAIS
Technical Strategy and Testing — X Projects: Initial Kuali Findings
11. The result is Java bean(s) that look the ones that first emerged from the ORM tool in
step 3.
12. The client program manipulates these Java beans and displays them to the user.
Modification Date: 2/17/2016
Page 16
Service Implementation Program
ID:
Fixed value 1:
Fixed value 2:
Etc.
Dynamic Attributes:
Key1
Key2
Key3
Key4
Key5
3
4
5
Custom Code to
transform Java
Beans into name/
value lists
Value1
Value2
Value3
Value4
Value5
Java Object
(field datatypes lost
in translation)
ORM tool
(e.g.
Hibernate)
Java Object
Java Beans
get/set
properties
XML example SOAP
payload
SOAP Toolkit to
transform Java into
XML/SOAP
(marshall)
New, or
Legacy DB
Java Object
6
1
2
ORM tool maps
database to Java
Beans
Database
Schema
8
7
<person>
<perstonTypeKeyList>
<personTypeKey>persontype.student
</personTypeKey>
</personTypeKeyList>
…
</person>
<person>
<perstonTypeKeyList>
<personTypeKey>persontype.student
</personTypeKey>
</personTypeKeyList>
…
</person>
XML over HTTP
Java Object
Java Object
Custom Code to
transform name/
value lists into Java
Beans
(field datatypes lost
in translation)
Java Beans
get/set
properties
User
9
Client Application
Java Object
11
SOAP Toolkit to
transform XML/
SOAP into Java
(unmarshall)
10
ID:
Fixed value 1:
Fixed value 2:
Etc.
Dynamic Attributes:
Key1
Key2
Key3
Key4
Key5
Value1
Value2
Value3
Value4
Value5
Client Application
Key
Custom
Java
Existing
toolkits
Transformation
Network
Page 9
Modification Date: 2/17/2016
Page 9