Saturday, January 30, 2010

The Creation of a Historical Database and Geomapping Project

This post may be interesting if you're also working on a similar project.

Over the past year, I had the pleasure of working for the Promise Land Project. I was involved with completing the database and creating a geomapping project. It contains all information from the original land record books for the Dresden Ontario area, dating from the 1840's until 1901. It is the goal that these tools will provide researchers with a visual understanding of the history and it may provide further insight into African Canadian history.

Designing the database

The database was designed to maintain the integrity of the original records holding to the "protect the fonds" theory of archival management. The main benefit of this database is that it is searchable.

The design of the database included: City, Area, Plan Number, Lot Number, Page Number, Guarantor (first, middle and last name, occupation, name code) Guarantee (first, middle and last name, occupation, name code), Lands, Instrument, Instrument Date, Registration Date, Book, Folio Number, Registration Number, Dollar or Pound amounts, and Lot Notes. These fields in the database all represent columns in the original land records books. The database was also structured according to books. Each land record book was given a tab and the database and page numbers were added. This way, researchers can easily refer to the original records without any trouble. In creating separate tabs for each book and including the page numbers, the information was organized in a very clear way and anyone without a research background can easily manoeuvre the data without difficulty.

Technical Challenges

The original database was created using Open Access Base and it became clear that Base would not be compatible with the GIS mapping software Arcmap. In order for the database to talk to the map it needed to first be converted into Microsoft Excel.
Not only would Excel be compatible with the mapping software, Excel has many other benefits over Open Access Base. Excel is overall a more powerful program and it can easily handle large amounts of data without slowing down. It has more features which make it both easier to use and provide more options to the user. For example, Excel provides the option to freeze panes, in doing so selected columns can always be visible. This makes it easier to input data if the column headings are always visible. Base did not offer this feature.

After conducting research online, my co-worker was able to convert the database into Excel; however, Excel did not recognize any of the numbers in the database including the dates or land costs. This information would all need to be re-entered by hand, as it became clear that there was no other way to accomplish this using any other method, even the copy and paste method would not recognize the numbers from Base. The only way to correct this conversion problem was to re-enter manually, every date and dollar amount. First, middle and last names also needed to be re entered into separate fields, that had originally shared the same field.

The use of name codes, were also introduced into the database. This was done because the original writing in the land record books was often difficult to read due to the flowery writing of the time. The code 1 was used to represent names that we were, almost certain to contain no errors, and the number 99, was used when errors in the name were suspected. This way, users could easily identify where there may be problems with the data and use their own judgement and knowledge of family names to recognize individuals despite potential inputting errors in the database. Name codes will also be a useful way for project partners to quickly refer to problem names and use their knowledge from research and by referring to the original records to correct any errors.

With the database complete data transcribing policies were created which outlined the methods used for designing and maintaining the database.

Geomapping Project

I then used this database to create a geomapping project in Arcmap. For the purposes of this project, I took the original plans, scanned them, and placed them in Arcmap. I then needed to provide them with spatial references and georeference them with a satellite image of the area. I then created a shapefile and with this shapefile, I was able to give each lot its own identity in the program, that I could then label with the appropriate lot and plan number. With this information in the program I could then join the information in the database with the map.

Project Uses

With these tools, which will be made available online, researchers will be able to search the database for particular historical figures or even family members. The mapping project will present the data visualy and may provide historians with new insights.

Note

Accuracy, is the most important thing when creating a database. Data entry is tedious and needs to be done meticulously; any error can undermine the integrity of the data, which is why it is important to give it the time that it needs. This project is now available to be seen online at: http://plp.uottawa.ca/

3 comments:

Anonymous said...

After reading through that thorough telling of what it takes do make all of the maps fit each other, I have a whole new respect for it!

You brought up a lot of problems that most people don't even know about!

Are there going to be more issues with converting maps and files as time goes on?

uhaul said...

good job
it's nice how you describe some of the difficulties you encountered and how you solved them
it,s a learning tool

Melissa Robinson said...

You're probably right. Who knows what advancements will be made in the next few years with mapping programs. But it's good to know that the informaiton has been put in digital format because accessiblity of public records is important and the information now exists outside of the archives as well, which is also comforting to know.