Data Integration

We designed, developed, and implemented a single data repository for veteran identification data, pulling together a number of data sources to consolidate, clean up, and merge information such as personal data, activation and deployment history, educational benefits, and spousal data. This single repository, named VADIR, enabled the VA to administer benefits more efficiently. Rather than relying on a number of separate, conflicting, and duplicative sources, the VA now owns a one-stop, authoritative source for veteran's data.

 

Challenges:

Our challenges stemmed from:

  1. The sheer size and volume of the data
  2. The consolidation of duplicate data sources that often contained conflicting information

Solution:

We followed a systematic approach to build this repository: analysis, design, code development, testing (both data validation and performance testing), and implementation.  For every data feed that the VA wanted consolidated, we met with VA and DoD stakeholders to learn what they wanted captured, what the data values meant, what sources they considered authoritative, and how to rectify differences between the sources in conflict situations.

Once we understood VA needs, we created design documents to:  1) verify that we indeed understood the VA requirements; 2) verify that we understood the nature of the data being extracted, and; 3) propose how we would efficiently and correctly pull this data from its original source.  We created a set of Oracle-based PL/SQL programs to retrieve, validate, clean up, and transfer the data elements in question. We then tested and re-tested these programs, making sure that the data values being pulled and transferred were indeed correct. We also tested for performance, making sure these programs ran quickly, and if they encountered any abnormalities, that these abnormalities were documented and handled.

Once both Oak Grove and the VA approved the program and test results, we worked together to schedule a go-live date for the interface, implementing and executing the interface on said date