Reviewer 1.
The paper is well written but needs some editing and proof reading. Minor
errors in grammar and punctuation are found throughout the paper.
>>Reviewed and fixed
Page 3: Overcoming *the barriers created by* this diversity ...
>>Fixed
Page 8: What motivated the decision to develop an AURIN workflow
environment instead of re-using Kepler, Taverna, or any other existing workflow
system? These systems have their shortcomings. It would be very interesting to
learn something about the choices made by the authors. This section would also
benefit from references to the relevant literature.
>>Fixed - new section added to section 4 and references given.
Page 20: Conclusions are suddenly numbered in Roman numerals.
>>Fixed
The authors chose to mention other workflow engines only in the conclusions.
This is much too late. As mentioned above, readers would benefit from a
discussion on workflow engine alternatives in section 4.
>>Fixed with new section added to section 4.
The authors note that the success of AURIN will be measured by the uptake and
adoption of the system. If this is a key metric, how are new users won and what
is being done to keep existing users?
>>Statements added to address this point.
Page 21: Numbering of references section should be 7. REFERENCES.
>>Fixed
Reviewer 2.
I thus recommend accepting the paper if the following improvements are
adequately addressed:
The authors should take care to avoid subjective phrases like “The AURIN
platform has demonstrated the vision of what can be achieved for large-scale
federated data access and analysis”.
>>Fixed
Please avoid (or else define) rather vague terms like “rich online research
environment” and “extensive portfolio of tools for data analytics”.
>>Fixed
Please give more detail on how the chosen approach deals with data
heterogeneity besides integrating different spatial aggregations, i.e. how flexible
the approach is with respect to overcoming other aspects of syntactic and
semantic data heterogeneity.
>>The AURIN platform does not support semantic data integration – noting that
there is no ontology in the urban domain that is widely accepted. As discussed in
section 3, various components are available to support data access, re-use and
integration however these do not define/enforce semantic checks. Thus it is
possible to identify meaningless relationships between data, e.g. number of
premises licensed to sell alcohol and the number of trees in a given area.
Syntactic checks are based upon the data/metadata providers by the
organisations involved. A statement has been added to this affect.
It is not evident if the described approach actually deals with ‘big data’ as
the authors state that the data addressed is not especially large in volume. The
authors should thus give information on the actual overall size of the data made
accessible, as well as on the size of selected data sets made available.
>>Fixed and statement added. Note that big data does not always mean big in
terms of volume but includes veracity, variety, velocity and a range of other
demands in accessing and composing large-scale, heterogeneous and distributed
data sets as discussed in section 2.
Similarly, information on the actual data update frequency, as claimed by the
authors to be a relevant challenge, would be of interest.
>>Different data sets, e.g. real time data on road use change every few minutes.
Other data sets, e.g. the Census, change every few years. A statement has been
added on this.
It is not always clear which parts of the overall system re-use existing code /
APIs and which are actually own developments. Please be more specific on this.
>>All components are developed by the AURIN team. The only elements that are
contributed are the tools that are wrapped as workflows by the external
developers. A statement is added to this affect.
The authors state: “Service-oriented architectures (SoA) comprised of
distributed web services offer one model for federated data access and
delivery”. Please briefly characterize also the other available approaches.
>>Fixed with statement on other approaches.
It would be interesting (although not crucial) to learn on the actual percentage
of people projected to live in cities in Australia (“vast majority” is quite vague)
>>These statistics are available from the ABS. A URL has been added.
From sec 4.1 it is not clear to me whether all components to be made available
need to be actually turned into Java objects – please explain.
>>Fixed.
In Fig 4, the comment for @out seems to be incomplete.
>>Fixed.
The case study on voting results does not seem to be a typical example for urban
research. Please shortly list the other examples that have been realized so far.
>>Voting is a key research area in urban settings since it relates to socioeconomics and urban policies (and many other things that impact on the urban
environment). The platform has over 1800 data sets and over 100 tools that can
be used to support a multitude of urban research needs. A reference [19] has
been added to show case studies in the use of the platform.
The acceptance of 3rd party products for data processing tasks that are already
addressed by researchers in their daily work is often confronted with the issue
of trust – please describe briefly whether this issue was a challenge in the
given project and how it is addressed.
>>The trust has primarily been around getting access to the definitive data from
the key agencies. With such data sets available (and often the only way to get
access to them is through AURIN), many of the issues of trust have been
overcome directly.
For the described example WFs it would be interesting to learn about the actual
degrees of freedom for the end user beyond selecting a spatial reference.
Please describe briefly.
>>All workflows can be parameterised with user-driven options and data sets. A
statement has been added on this.
Please give more details on the following: since when the system is online, how
many predefined workflows are available, how many WFs have been executed by
users, how many workflows are planned to be added?
>>These statistics are not recorded (unlike access to data sets which are
recorded as demanded by the data providers). There are 113 workflow-based
tools that are currently available in the platform. No further tools are expected,
however further data sets are requested since the main challenge facing the
urban community is access to data.
For case study 2, it would be interesting to know whether the existing
walkability application was re-coded or else how it was integrated.
>>This was described in section 4 as part of the discussion on how the workflow
environment is used to wrap existing applications.
As the GUIs are generated automatically, it is likely that they become quite
complex for complex workflows. Please give some information whether this
aspect been addressed and has system usability been evaluated?
>>This is not currently (directly) supported.
As various existing visualization components are re-used, it would be
interesting to know whether a common L&F to overcome their (presumably)
heterogeneous output styles has been addressed (and if so, how).
>>A reference has been added with more details on how the visualisation
aspects of the AURIN platform have been supported.
Check References, eg Reference 17 is missing
>>Fixed
Check carefully for typos and punctuation
>>Fixed
Ensure that all acronyms are introduced
>>Fixed