Increasing GIS Web Services Capacity to Serve The Unidata Community.

William Gallus Jr
Daryl Herzmann

The recent few years have seen an explosion with the need and use of environmental data provided via web services and standard formats to Geographic Information Systems (GIS). Unidata has been a pioneer in developing software and stimulating its community to meet this growing need. To that end, the Unidata Equipment Grants program provided funding to the Iowa Environmental Mesonet (IEM) of Iowa State University to increase its capacity to serve Unidata provided datasets to the expanding GIS user base within the Unidata Community.

Hardware Purchased


Figure 1: IEM server configuration

Using funds provided by the Unidata Equipment Grant and supplemental local funds, the IEM was able to build a mostly fault tolerant and scalable infrastructure to support the mass dissemination of GIS products and web services. Specifically, two powerful Dell servers (PowerEdge 2850) along with nearly 1 terabyte of fast disk space were purchased with the Unidata grant. After some bureaucratic fun, these systems were placed in production on November 29, 2005.

These new servers were configured as participants in a Linux Virtual Server (LVS) cluster. Figure 1 shows the current topology of the cluster with the Unidata grant providing the "LDM Processor" and "Spatial Database" components. A LVS cluster provides a software based solution to do high availability and load balanced computing. Two "director" nodes create a redundant virtual presence providing services by brokering traffic between external clients and cluster internal services. These services include a Local Data Manager (LDM) instance, Spatial database, and web servers.

Delivering Unidata datasets to GIS

The project proposal outlined a number of Unidata provided datasets that would be made accessible via web services and GIS formats. During the past year, some of these datasets are now being provided in GIS formats/services by institutions such as the National Weather Service (NWS). The NWS should be commended for efforts such as their RIDGE RADAR data display system and the Precipitation Analysis website which provides the GIS community with operationally supported datasets.

Thanks to the Unidata grant, the IEM is now providing these services and data files for the communities use.

  1. CONUS NEXRAD base reflectivity composites
    Availability: LDM/IDD, OGC WMS-T, HTTP GeoTiff
    Notes: This dataset is by far the most requested and the most widely used dataset the IEM provides. It is produced by using the 'nex2img' GEMPAK program written by the Unidata Program Center. Nationwide ~1 km resolution products are generated every 5 minutes. The equipment grant allowed the IEM to provide a Web Map Service (WMS) that supports the time specification (WMS-T). This feature exposes our entire 5 minute interval archive dating back to 2003 to WMS-T aware GIS clients!
  2. NWS watch/warning products
    Availability: LDM/IDD, OGC WFS, HTTP Shapefile, XMPP Jabber
    Notes: Utilizing the robust and extremely fast LDM/IDD infrastructure provided by Unidata, the IEM wrote a number of decoders to ingest various NWS issued products into a spatial database (PostGIS). This software will be eventually released to the Unidata community under an Open Source license. Organizations interested in beta-testing this software should contact Daryl Herzmann.
  3. GOES satellite composites
    Availability: OGC WMS, HTTP GeoTiff
    Notes: GOES East and West, IR and VIS imagery is available. CONUS composites are also generated using a GDAL utility. An archive of these products will be exposed via a WMS-T instance later this summer.
  4. "Stage 4" NWS precipitation estimates
    Availability: NWS Precipitation Analysis
    Notes: The NWS has provided this wonderful service, so there was no need for the IEM to duplicate this effort. In the near future, the IEM will provide OGC WMS, WFS, and WCS services of this dataset.

Here is an example interface utilizing the WMS-T NEXRAD service. The interface code is a part of the OpenLayers toolkit. The default time display shows Hurricane Katrina making landfall.

Summary:

The Unidata equipment grant provided funding necessary to increase the capacity of the IEM to provide datasets to the GIS community. Since the installation, the IEM web farm is now handling 4 times the number of web hits than when it was previously at full capacity last fall. The infrastructure in place should allow another 8 fold increase in internet traffic before more substantial resources would need to be purchased.

As always, the IEM continues to work with other members of the Unidata community, NWS, and private/public sector to increase the use of these services and adoption of similar technologies. Please do not hesitate to contact Daryl Herzmann if you have any questions or suggestions.