Friday 27 September 2013

Summary


We have involved representative users from targeted user communities through the whole development process. The users have been testing  the user-facing functionality of the Soils2Satellites product in an iterative format, with their input driving the ongoing development of the application. Mostly the user interface worked to meet user requirements, and the project team has been pleased to see the system being supported by potential users.

Our test users


User 1 is a postgraduate at University of Adelaide who specializes in ecological theory and genetic and genomic approaches to answering biological questions. They have worked in weed management and the impact of climate change on invasion patterns. We expected this user to represent researchers interested in species invasion and change in the face of climate change.

User 2 is the Science and Technology Lead for a TERN MSPN facility and has had a range of Scientific and Management roles in both the Northern Territory and the South Australian Government. We expected this user to represent a senior science focused researcher and University administrator exploring the information in S2S.

User 3 is an internationally respected environmental scientist who has spent many years working in the field of conservation ecology in Australia and the USA. Their primary research focus has been identifying ecological traits of species that are vulnerable to extinction. They have also worked for several state and federal government agencies. We expected this user to represent senior academics and researchers and policy making site users.

User 4 is a field ecologist and researcher. We expected this user to represent ecological data collectors and providers who could supply information.

Functional Testing


Prior to our user group becoming involved in testing a particular component of the application the project team conducted significant functional testing, to ensure that as many bugs as possible were squashed, leaving users to focus on the UX and the application’s functionality and workflow.

Use Cases


Both functional and User testing were performed with reference to the Use Cases listed below. User testing was performed in a far more “free form” manner than the Functional testing with less prescription upon specific activities and in a less structured environment.
  • Find Study Location
  • Find Study Location Visit
  • View Study Location Summary
  • View Study Location Visit Summary
  • View Sampling Unit Summary
  • Extract Study Location/Study Location Visit Data
  • Compare Study Locations
  • Compare Study Location Visit
Users also tested areas of the system, in particular with reference to the Mapping Interface without any real reference to Use Cases.

Find Study Location & Find Study Location Visit

This functionality allows a user to find one or more study location(s) matching the search criteria, they have entered. The Search criteria being either or both of
  • a spatial locational area defined by drawing a bounding box on a map or
  •  textual search criteria entered via a text box or selected from a pre-defined list
using this functionality returns and displays a list of matching study locations and visits that can be added to the study locations/visits list for further review.   

What succeeded:
The technique of searching for locations via a map interface and then filtering a set of returned values with further parameters is very familiar to most similar web applications and the users found this functionality intuitive and easy to use.

What we learned:
User feedback defined the amount of summary detail that was to be displayed for each returned study location and visit found by the search, they also particularly asked for a link to be included that provides swift access to the Study Location Summary details for a particular site.

View Study Location & Study Location Visits Summary

This functionality allows a user to view a set of summary data for either an individual study location or a study location visit. The user accesses the summaries via the same method of either clicking a specific link to the summary page from the list of Study Locations and Visits returned from a Search, or by accessing (via a link) the visit summary page from within the study location summary or vice versa

What succeeded:
Users easily understood the functionality, i.e. they were able to navigate to the summary pages they wished to see. Users were also happy with the level of detail in the summary pages after they had provided input. The Users saw the benefits of the data visualisations in quickly and simply representing the underlying data.

The users also contributed to the particular data visualisations in each summary page in terms of the business logic/rules for each graph/chart and also the design and layout of the page real estate.

What we learned:
Initially there were a number of data items missing from both summary data sets that we subsequently added to the data displayed when asked to by the users. The formatting and order of the data displayed was also updated based upon feedback.

We amended the navigation of the application too to make visits and study locations available in a more direct fashion by removing some of the hierarchical navigation that had initially been implemented. 


View Sampling Unit Summary

This functionality allows a user to view a particular sampling unit summary for a particular Study Location Visit. There are a number of different Sampling Unit types in Soils2Satellites
  • Basal Wedge
  • Point Intercept
  • Structural Summary
  • Soil Observation
  • Soil Bulk Density
  • Soil Character
Each of these Sampling Units types has a specific set of data and a set of data visualisations, the Sampling Unit Summary page is accessed via a link form either a Study Location or Study Location Visit Summary page.

What succeeded:
As with the other summary pages users easily understood the functionality, i.e. they were able to navigate to the summary pages they wished to see and understood the data they were being presented within each different Sampling Unit Summary page.

Users were also happy with the level of detail in the summary pages after they had provided input. The users saw the benefits of the data visualisations in quickly and simply representing the underlying data.
The users also contributed to the particular data visualisations in each summary page in terms of the business logic/rules for each graph/chart and also the design and layout of the page real estate.

What we learned:
Initially there were a number of data items missing from both summary data sets that we subsequently added to the data displayed when asked to by the users. The formatting and order of the data displayed was also updated based upon feedback.

The algorithms used for the Point Intercept Sampling Unit Summary in particular were significantly influenced by the user feedback as were the colours used in the visualisations that represent the transect layout – the colours were changed to the actual colours of substrates recorded in the field. The size of the points representing the individual Growth forms recorded in the transact are also due to direct input from the users.

The particular visualisation showing comparison between Auscover Fractional Cover layer data and equivalent “Ground Truthing” data from AusPlots/TREND  is also a result of direct input from users. 

Extract Study Location/Study Location Visit Data

A key requirement for Soils2Satellites was for the users to be able to simply create and extract a set of data derived from the whole S2S database. Data can be extracted at the Sampling Unit Summary level or for one or more Study Location Visits. At the visit level standard data about that visit is included plus data to be extracted can be further filtered to include one or more or all of the data for the Sampling Units performed during that visit. 

What succeeded:
The simple extract format of a single file in .csv format downloaded as a zip file was appreciated by the Users, as was the creation of a DOI to support re-use of the data. This functionality successfully re-used technologies from the ALA and the TERN HQ DOI creation functionality. 

What we learned:
The initial navigation process to access the extract functionality was altered based upon the user feedback. An Export button was added to the Search Results screen to allow users to quickly Search, Access and download data without having to access each individual Sampling Unit Summary page. 

The data made available from intersecting the environmental layers data with the location of a particular Study Location was added to the extract data due to User feedback.

Compare Study Locations

This functionality allows a user to view and compare a range of data values in their raw form and through visualisations for any number of Study Locations returned from the Searching functionality.

What succeeded:
The ability to be able to compare a standard set of data across sites in varying geographical locations in one screen and via a range of visualisations was seen by the users as a key function of the application.

What we learned:
The users contributed to the particular data visualisations in this page in terms of the business logic/rules for each graph/chart and also the design and layout of the page real estate. The identification of the types of data that would “work well together” as visualisations, that provide a meaningful “view” of a site were implemented and refined from this user input. 

General Mapping/Spatial Feedback


The S2S application contains a fairly standard Mapping interface using OpenLayers and the Google Maps API, this functionality was leveraged from the capability provided within the Atlas Of Living Australia’s spatial portal. This provides standard map navigation tools along with the ability to select points visible in the map and add them to a selection list.

At the request of the users the opacity of the map’s layers can be altered plus the layer attribute details can be viewed. The mapping functionality also allows users to build sets of environmental layers (derived from the full list of environmental layers provided by the ALA) that they can store and re-use as personal sets of contextual layers during their searches. 

Individual User Experiences


User 1

User 1 was predominantly involved in identifying the types of visualisations that would be most valuable to non-ecologically focused users through making the data in S2S discoverable and in defining how it would be best to display interesting and significant relationships across the different data types. This users input was key to defining the data outputs in the application.

“The intuitive navigation allows me to easily visualise trends in the data that would otherwise be extremely difficult and time consuming to consider, furthermore, results are presented in such a way that novel associations can present themselves – helping to stimulate new ways to think about ecological associations and identification of genuine and interesting knowledge gaps where new research can be focussed."

User 2

User 2 was key in providing the scientific understanding and business rules behind a number of the visualisations in the application in particular in the Point Intercept Sampling Unit Summary visualisations. This user was particularly pleased to see that the data generated by Auscover from its remote sensing activities can be ground truthed by data collected under AusPlots/TREND and the correlations or differences in the data be easily and clearly visualised. They are especially keen to be able to now take the application further in this area if possible.

User 3

This user was predominantly involved in detailing what raw data for each Study Location and Visit would be relevant for display and download and how this data was to be grouped together in the application.  As with the other users this user was also involved in providing feedback to the project team around the data visualisations.

User 4

User 4 commented on how following the implementation of the data visualisations in the application, it was now possible to demonstrate to the landowners and managers of the country where the data collection activity had taken place the value and usefulness of this type of data collection. This then encourages further access to areas for future re-visits and also for access to other areas where monitoring surveys may not have taken place before.

“The Soils toSatellites tool is important in fostering the continuing relationship between the scientific community and those who grant us access to their land. By allowing land managers to visualise the data shortly after we’ve collected it and see how it fits in to a bigger picture, it’s easy for us to show the value of the work. It can also help landowners to manage their lands, whether to maximise agricultural productivity, ensure strong conservation outcomes, or a mix of both.”

Tuesday 3 September 2013

S2S stands out at ALA Science Symposium

Congratulations to the Soils-to Satellites (S2S) team for an excellent example of a successful collaboration between National Collaborative Research Infrastructure Strategy(NCRIS) capability . The team showed how different types of data from different platforms can be integrated and visualised to support reuse by researchers for new scientific enquiry. Some participants considered the presentation by Martin Pullan, Project Manager at TERN’s Eco-informatics Facility and Peter Doherty, the Program Manager at the Atlas of Living Australia (ALA) to be among the best at the Symposium.

The ALA hosted its first Science Symposium recently at the CSIRO Discovery Centre in Canberra with 27 speakers presenting on topics from taxonomy to data infrastructure. Scientists attend many conferences to keep up-to-date with new developments in their field of study.  Often, the gems of new information at any one conference can be counted on one hand and the S2S incubator project stood out as one of these. A presentation that followed on from our talk by Dr Andrew Treloar, IT Director at the Australian National Data Service (ANDS) also highlighted the successful collaborative nature of our project and he stated it was his favourite ANDS application development project.

S2S project although a 'proof of concept' has the potential to be leading infrastructure for accessing data on Australia’s nature and environments. A strong data alliance is already in place for further roll out of the application and once it has been released in the next few days we hope you support its ongoing development via this blog.

Final Post

1.   Introductory Product Information


Soils-to-Satellites (S2S)

The tool brings disparate and diverse datasets together into an easy-to-use interface that enables visualisation and comparison. These datasets include ecological and genomics data collected through TERN’s Multi-Scale Plot Network and delivered through Eco-informatics’ Australian Ecological Knowledge and Observation System (ÆKOS), remote sensing datasets available through AusCover, alongside the spatial and biodiversity information available via the ALA.

S2S now enables researchers to explore and display the relationships between these diverse data sets in ways not previously experienced in a single application. For example, users of the tool can display layers of Australian environmental data such as elevation, temperature or soil type, and then ‘drill down’ to view and explore rich ecological and genomics data across those layers.

2.   Instructional Product Information


"Through AusPlots and the Australian Transect Network, we've now surveyed around 300 plots, which represents a wealth of ecological data that we hope will support scientists and managers across Australia. Our data is undeniably valuable, and represents thousands of man hours in some very inaccessible environments. But trying to show potential users the scope of the data we've collected and how it can be combined for useful analysis can be a real challenge. The ability of Soils to Satellites to display and combine the data we've collected in real time, from biomolecular sequences to landforms, allows users to do just that, and make sure that our data is used by others, to the maximum benefit of scientists and the community."
          Stefan Caddy-Retalic
               Terrestrial Ecosystem Research Network (TERN)

"Soils to satellites allows to me explore large and complex data sets in a meaningful and dynamic way. As we move towards greater levels of data sharing and maximisation of initial investment in data collection, projects like Soils to Satellites become increasingly important tools that allow us to navigate the wealth of existing data, assess its usefulness and fitness for repurposing, and ultimately access the raw data and all the associated contextual information. Without efficient and intuitive ways to explore the relationships between existing data sets at a range of scales, such as those provided through Soils to Satellites, we will not be able to take full advantage of the considerable scientific knowledge base we have at our finger tips. The intuitive navigation allows me to easily visualise trends in the data that would otherwise be extremely difficult and time consuming to consider, furthermore, results are presented in such a way that novel associations can present themselves – helping to stimulate new ways to think about ecological associations and identification of genuine and interesting knowledge gaps where new research can be focussed."
          Eleanor Dormontt
               Terrestrial Ecosystem Research Network (TERN)

"Working in projects which collect lots of data, it’s easy to lose sight of where and how the data will be used. As someone who is generating this data in the field, Soils to Satellites represents a major step forward in making sure that the data I collect will be used by others. By providing different ways of visualising complex data, scientists and land managers can understand the relationships between that data in a simple way. This means that data that might otherwise gather dust in an obscure database is available in a simple format for everyone to access. For example, previously labour-intensive comparisons – say between vegetation community structure and soil characteristics (like pH and salt content) are easy, either at a local or continental scale.

The Soils to Satellites tool is also important in fostering the continuing relationship between the scientific community and those who grant us access to their land. By allowing land managers to visualise the data shortly after we’ve collected it and see how it fits in to a bigger picture, it’s easy for us to show the value of the work. It can also help landowners to manage their lands, whether to maximise agricultural productivity, ensure strong conservation outcomes, or a mix of both.

Soils to Satellites will only increase in usefulness as a tool as more data becomes available. Information on soil metagenomics, population genetics and leaf area index will build on the current platform to make soils to satellites an indispensable tool for ecologists and land managers alike."
           Emrys Leitch
               Terrestrial Ecosystem Research Network (TERN)

2.2 Links to the actual product

2.3 Source code

2.4 User Guide

3.   Product (or Product Components) Re-usability Information


The Soils to Satellite application provides a service to researchers, that further represents the potential of integrating major research infrastructure.  Data available from TERN Eco-informatics' ÆKOS data portal (raw ecological data) and the Atlas of Living Australia (biodiversity data and spatial products) can be searched, displayed and visualised thanks to an alliance of data custodians who publish their data and products.

Any data downloaded from SoilsToSatellites can then be re-used outside of the system by ecologically focused researchers

4.   Contextual Product Information


4.1 Licensing of final product
  1. SoilsToSatellites is available under Mozilla Public Licence v1.1
  2. The data downloadable from the application is licenced through

Creative Common By Attribution (CC-BY) Copyright Licence v3.0. This licence lets others distribute, remix and build upon a work, even commercially, as long as they credit the original creator/s (and any other nominated parties)

4.2 Sustainability
  1. All the software and data components will be located on NeCTAR infrastructure, initially within the University of Melbourne and then with eResearchSA at the Adelaide node of NeCTAR.
  2. The SoilsToSatellites portal will be maintained in the public domain for 12 months by TERN Eco-informatics and ALA, with support from NeCTAR.
  3. New data will be added to SoilsToSatellites by TERN Eco-informatics staff.


Thursday 13 June 2013

S2S incubates a data alliance

Two horses by working together can pull a load of almost 4 tons. When you combine four horses, they’re not limited to 8 tons but can pull a whopping 123 tons -15 times as much because they work in unison and maximize synergies.  By forming an alliance between data custodians and infrastructure capabilities, the Soils to Satellites (S2S) incubator project has leveraged synergies in technical expertise and datasets to create a web service that’s greater than the sum of its parts.

Professor Andy Lowe at The University of Adelaide is a world renowned plant evolutionary biologist addressing ecological, evolutionary and biosecurity questions using molecular approaches, and he with funding support from  the Australian National Data Service, has sponsored joint development by TERN Eco-informatics and the Atlas of Living Australia’s (ALAof the project.

S2S is a ground breaking project because it brings together, through web services, different types of data from different sources, stored on different platforms, for researchers to reuse for different scientific purposes’, said Andy.

‘It represents the start of a strong data  alliance for environmental data in Australia, and this give us great opportunities to bring together cutting edge expertise in IT, Ecology and Data Management to advance science.’

Dr John La Salle, director of ALA, said working on the S2S project was a great opportunity to further demonstrate the openness of the ALA system.
For the first time, the system brings a range of data together from the ALA and TERN Eco-informatics’ Australian Ecological and Knowledge Observation System (ÆKOS). These data are a mix of biodiversity, ecological and genetics data and derived spatial data covering environmental layers such as landform or climate, and other features derived from remote sensing technologies and maps of soils.

‘Having this rich mix of data available via a single application is a huge benefit to science, as modellers can work immediately with integrated datasets and save lots of valuable time,’ says Andy.

‘We were able to achieve this breakthrough literally because of our fledgling alliance for sharing resources and leveraging the “openness” of our systems. Our work brought together volumes of data and specialist expertise - especially in the technology and technical infrastructure which underpins the ALA and ÆKOS.  The result is a service that furthers represents the potential of integrating major research infrastructure.’

How is this so? S2S brings in data from a broad range of custodians as shown in the diagram below. Open remote sensing products are available by the Bureau of Meteorology and TERN AusCover. These have been leveraged by CSIRO and the ALA, making available valuable spatial layers representing different types of climate and environmental data. These data can then be further overlaid with soil maps that are created by CSIRO and TERN’s Soils and Landscape Grid ofAustralia facility on demand by the user . Biodiversity data on plant species initially is also integrated into this system and these data have been contributed to the ALA by its partners  Commonwealth and State Herbariums and Museums, government agencies and the public. The raw ecological plot data available in ÆKOS presently has databases and datasets contributed by government agencies and TERN’s Multi Scaled Plot Network  (AusPlot Rangelands and the AustralianTransect Network TREND) and will also house a number of additional researcher datasets in the near future. The genetics data are contributed by the Barcode of Life, GenBank and BioPlatforms Australia to the ÆKOS.

S2S will be launched at the end of this month and is a strong ‘proof of concept’  demonstrating that a system can successfully integrate disparate data from different platforms. The system has been made possible through the generous funding support of ANDS and underpinned by strong collaboration between talented ecological, data and cyberinfrastructure experts.  This service will be developed further in the future.

The executive director of ANDS, Dr Ross Wilkinson, said ANDS was delighted in the way that the project demonstrates the power of bringing data together to answer new questions.


The Soils to Satellite application provides a service to researchers that further represents the potential of integrating major research infrastructure.  Data available from TERN Eco-informatics' ÆKOS data portal (raw ecological data) and the Atlas of Living Australia (biodiversity data and spatial products) can be searched, displayed and visualised thanks to an alliance of data custodians who publish their data and products.

Tuesday 16 April 2013

‘Soils to Satellites’ shines at TERN Symposium

The annual TERN Symposium is an opportunity for anyone and everyone ecologically-inclined to gather in a single location and discover what projects the various TERN facilities have been working on since the previous year. This year’s event was held over three days in February, at Old Parliament House, Canberra and featured a host of prominent guests including plenary speakers Lord Robert May (former Chief Scientist of the UK) and Professor Kris French (President of the Ecological Society of Australia) Dr David Schimel (Jet Propulsion Lab, NASA) and Professors Andy Lowe and Stuart Phinn (Associate Science Directors of TERN). The Symposium was attended by over 350 people from all across Australia, including many travelling from overseas, demonstrating an increasing interest in Australian ecosystem science infrastructure and science outputs across the country.

This year, a presentation on 'Soils to Satellite'(S2S), a collaborative project funded by the Australian National Data Service (ANDS) was given by Craig Walker (Director of the Eco-informatics Facility of TERN) and Peter Doherty (Atlas of Living Australia Programme Manager).  The presentation outlined the gains of greater synergies and collaborations between the NCRIS capabilities and how S2S addresses these. 

Craig described how the NCRIS capabilities had vast volumes of data and high levels of expertise especially in advanced technology and technical infrastructure covering the Australia's environmental information landscape. 

'For example, the Australian Ecological Knowledge and Observation System (ÆKOS) built by Eco-informatics has well-described, raw scientific ecological data as its focus and complements the Atlas of Living Australia which focuses on data and profiles of species and specimens (biodiversity data)', said Craig.

'ÆKOS as part of its development has leveraged ALA's expertise in systematics, biodiversity and spatial platforms.'

Peter said the S2S incubator project demonstrates the integration of disparate data (independent of structure and standards), as well as strong collaboration and cooperation between NCRIS capabilities and the value and power of the mature ALA web services. 

'It also contributes to and enriches ANDS Research Data Australia platform and generates 'real world' feedback for future iterations'.

'S2S brings together technical expertise and technological infrastructure for mutual benefit', agree Craig and Peter.

'There are many benefits enabling raw ecological plot and biodiversity data to be explored to support species, community and ecosystem distribution modelling to address climate change science questions in new ways not previously possible.'

Tuesday 15 January 2013

Post 6. PROJECT OUTPUTS & OUR PRIMARY PRODUCT

The Soils to Satellites Project delivers a web-based application that allows users to combine and explore data from the Atlas of Living Australia (ALA) and the Terrestrial Ecological Research Network (TERN) ÆKOS application. These data are associated with a range of survey work conducted by TREND and TERN AusPlots and also the ALA’s environmental and species distribution spatial layers. These data are aggregated in the application by reusing and re-implementing the functionality already present in the ALA. - please refer to other posts for details on these data.

This site will allow users to search for data in a number of ways either through “pre-canned” views of data or by building structured queries. Data can be viewed at a summary, aggregated level or at an individual field record level.   Result sets can be visualised through a set of  tools provided by the ALA whilst the results can also be exported in csv format.

Result sets will also be provided as RIF-CS collections to ANDS for further easy discoverability. The code used to create these websites will be available for reuse under a GNU General Public License. All components created by the project team will be available from this Bitbucket site along with documentation (Use Cases, Architecture diagrams etc. A user reference manual will address operational day to day usage procedures and will explain how to go about using application.) on Google Drive. The application will be hosted through a NeCTAR node at e-Research South Australia (eRSA) at the University of Adelaide


The product itself will be presented or publicised in a number of ways:
  • Press release(s) targeted at the Ecological and Conservation communities and DSIIRTE, that publicise the outcomes and collaborative achievements relating to the project and the associated software tool developed with ANDS funding. As well as highlighting the new research now possible by Prof Andy Lowe’s research group by bringing the Soils to Satellite data sets together in a single application.
  • Demonstration of the resulting solution at the TERN Annual Symposium, February 2013 by Prof Andy Lowe. Attendees include TERN staff, university researchers, Government decision-makers and policy experts, representatives from peak bodies, and industry sector staff. The value message will focus on potential research that can be conducted with the assistance of the tool that couldn't be done before.
  • Releases posted through social media channels such as Twitter, LinkedIn, Pintrest and Facebook.

Additionally, the following Demonstrations of Value may be pursued:

  • Demonstration of the value of the combinatorial solution for use in policy and natural resource management contexts, e.g. utilising the spatial power of the systematically collected genomics data over environmental gradients to show:
    • i) future management implications of temperature change, rainfall, drying for critical issues such as species genetic flow, refugia and evolutionary/extinction potential; &
    • ii) the value of integrating existing expertise from ALA and TERN for cross-cutting solutions.

In his roles as TERN Associate Science Director and Head Science advisor for the SA Department of Water, Environment and Natural Resources (DEWNR), Prof Andy Lowe can assemble key agency decision makers in both funding and policy contexts. Given the timing of governmental budgetary allocations, late 2012/early 2013 would appear to be the most suitable timing.
  • Endorsements from key direct and indirect stakeholders such as Commonwealth agencies
  • One or more academic papers submitted by the Science Stakeholders (Lowe, Cross, Guerin) by the 2nd quarter (calendar year), 2013 - in recognised journals which reference the integration and analysis enabled by project