Skip to end of metadata
Go to start of metadata

This page is for sharing LTS system development and maintenance plans for the Harvard Geospatial Library.

Charter

I. Problem/Value Statement

Problem Statement:

The Harvard academic and research community are increasingly using geospatial data analysis and visualizations to do research and to present visualizations of research results. Additionally, Harvard is a partner in the OpenGeoportal cross-institutional collaboration to share GIS data sets widely for reuse and create open source discovery and repository software. Metadata about Harvard GIS data sets, and previews of the data itself, has been made available through OpenGeoportal, and is directly searchable in library supported portals at Stanford, MIT, Tufts, UC Berkeley, and many others.

The Graduate School of Design teaches classes in geographic information systems (GIS) and requires students to learn and become fluent in geospatial analysis and mapping. The Center for Geographic Analysis includes links to library data sets in WorldMap. A centrally supported GIS data repository and search tool enables researchers to discover GIS datasets that are available on-line from Harvard and peer institutions, and to access and download datasets for use in a wide variety of research, teaching, and learning applications. Additionally, tools and GIS technical support is critical for library staff creating metadata for GIS data sets and to load metadata and GIS data into the repository.

HGL requires a major software upgrade because a key software component has reached end of life. This upgrade requires substantial software changes, as well as new installations of key software components and data migration. This work requires the collaboration of the HGL software developer, LTS operations, HUIT database analysts, and library experts from the Harvard Library Map Collection.

Business Value:

The Harvard Geospatial Library provides a centrally supported resource to faculty, students, GSD teaching and library staff, and HCL Maps and ITS Library staff. Without HGL, these staff could not offer services to meet their university missions. In addition, GIS partners around the world view Harvard's commitment to GIS data set discovery and reuse as exemplary and valuable.

II. Vision and Approach

Describe the solution:
HGL is a service provided centrally by the Harvard Library with critical support from LTS. The LTS portion is comprised of a constellation of software applications, servers, and technical support service to library staff. See this diagram of the HGL universe for an overview of the systems and services provided. HGL includes not only a web based discovery interface and repository for GIS data sets, but also back end servers accessed directly by GSD and Map Library and ITS staff to enrich and manage their GIS holdings.

Like any digital system, HGL requires ongoing maintenance, including software and server upgrades to supported and secure versions of software applications, operating systems, databases, and server hardware.

Deliverables/Work Products:

FY17

  • Configure a new database server for HGL with the latest Oracle release and latest ESRI ArcSDE software (mercator)
  • Migrate the database contents from the older HGL databases to the new database
  • Develop new HGL software for map tile caching and interaction with a new ESRI component, ArcGIS Server.
  • Update the user interface for HGL to use Open Geoportal 2.0 (pelham)
  • Publish all layers as new web services for OGP 2.0
  • Work closely with Map Library staff to test the migrated HGL data
  • Shut down the old HGL application server (calvert) and database server (alcott)

FY18

See HGL FY18 Roadmap

III. Stakeholders/People

Who is the work being done for? (Sponsor) Harvard Geospatial Working Group of the Research, Teaching, and Learning Standing Committee, Harvard Library

Who is funding the work? HL through the LTS operating budget

Who will accept the work? Bonnie Burns, chair, Harvard Geospatial Working Group

Who is the project manager? Randy Stern

Who will be involved in doing the work (service area, department, etc …)? [Include name, project role, and estimated percentage of time per month for the project duration]

Resource NameRole(s)Project Estimate (through project CompletionOngoing supportWork
Randy SternPM4%0% 
Dave Siegeldeveloper and FY17 software support100%30% 
Tim ElliottData publishing and tile caching script development25% for 2 months.0%Productionize Bonnie Burns' prototype script
Bonnie BurnsHGL working group chair, publishing script requirementssmall Create prototype scripts, assess intermediate results
Danielle Browngeospatial data technician, ITS Metadata Managementsmall  
Marc McGeemetadata analyst, ITS Metadata Managementsmall  
Venu RamaswamyHUIT DBAas much as needed Migrate the databases fro 3 ArcSDE instances to new versions of ArcSDE
SharonDev/ops managersmall review architecture for security, scalability, sustainability, cost
Tom ScorpaDev/ops engineer5% for 1 year5%? 
Anthonyarchitect  review architecture for security, scalability, sustainability, cost
LTS Support (tbd)Support lead  define and implement support policies, public support links, and internal documentation

IV. Schedule and Cost

Schedule: [include project phases, known reporting dates, and delivery deadlines]

(see  HGL 2016 - New Architecture Project Checklist for detailed work checklist)

2016-2017 HGL Checkin Meeting Notes

See HGL disk storage and costs for detail.

PhaseitemWhoCompletion DateStatusPurpose
Prioritization and other workPrioritization by Harvard LibraryAnn Whiteside4/21/15doneBonnie and Janina wrote up a one page document describing the need for HGL technical work, the broad spectrum of users, the connection (and/or future connection) to CGA, the importance of working within the Open GeoPortal consortium, the work that is needed to bring HGL up to the right version and other needs. A working group will write a report for a TBD standing committee and LLT.
 HOLLIS+ GIS integration and embedded mapsDave7/1/15doneSupport for HOLLIS team in adding Geospatial data from the Harvard Geospatial Library (HGL) to HOLLIS+.
 HGL developer available after DRS/FITS work reassignedDave8/1/15doneHGL developer assigned
 Migrate HGL to tomcat7/java7Dave12/22/15doneHUIT required security update to HGL
 HGL Working Group prioritizes OGP 2.0HGL Working Group1/4/16doneThe HGL Working Group performed usability testing of the HGL 2.0 interface.  The group strongly feels that we should move ahead with this.
Hardware/software configurationMigrate HGL QA environment from ArcSDE 9.3.1/Oracle to ArcSDE 10.2/Postgres w ArcGIS server with OpenGeoportal 2.0Dave2/24/16doneProof of concept development
 Attempt to migrate new HGL servers to the AWS Cloud 2/1/16doneLTS operations configures new HGL in the cloud. Decision is to revert to existing VM architecture.
 QA testing complete on VM architecture, request for LTS OPS production machine setupDave3/23/16doneInitiate production deployment planning with LTS operations group
 Setup base software on new machinesLTS OPS5/6/16doneMachines had been setup on by Feb 3, before AWS attempt
 Configure machines for access as neededLTS OPS6/10/16done 
Database migrationHUIT DBA help requested in Service Now - INC01797979Randy6/29/16doneVenu Ramaswamy assigned 7/13/16 to perform database migration to new ArcSDE schema
 Develop and test upgrade path for data migration from old HGL to new HGLHUIT DBA (Venu), Dave, LTS OPS10/15/16doneMany iterations with ESRI, HUOT DBA, and Dave to configure ESRI and Oracle database
 Database migrated for first SDE instanceVenu, Dave10/31/16doneTook 3.5 months
 Database migrated for second SDE instanceVenu, Dave11/20/16failedShould have been to copy procedure from first migration. Has run into many issues with HUIT, LTS, ESRI.
 Database migrated for PROD SDE instanceDave, Tim, Bonnie1/3117done (2/3/17)Latest proposal is to try to bypass the HUIT database migration.
Data publicationData publication with Map LibraryDave4/1/17? "Publishing" the data makes it available to the HGL user interface. Sub projects: Script development, raster data publication, vector data copy and publication
 Testing completeBonnie Burns4/15/17?  
LaunchLaunch new HGL and shut down old HGLBonnie Burns, LTS OPS5/1/17?  


V. Other

Constraints: Coordinating LTS OPS, HUIT DBAs, ESRI tech support, and the HGL developer is a challenge.

Assumptions: Map library can create publish scripts and execute them in a timely manner. HUIT DBA can upgrade last 2 ArcSDE instances in a timely manner.

Dependencies: Dependent on HUIT DBA schedule and Map Library publishing script creation and execution

Risks (description, plan, impact, owner): There is a single knowledgeable HGL developer in LTS, so illness, vacation, etc can delay projects.  HGL development is lower priority than production system issues both for LTS OPS and for HUIT DBAs.

 

  • No labels