Data Normalization, Geocoding, and Error Assessment

Goal

The goal of this assignment is to geocode locations of known sand mines located through out Wisconsin and compare them to both our classmates geocoded mine locations but the actual mine locations as well. 

Objectives

In this assignment there were six main objectives that needed to be completed, the objectives are listed below:

1. Normalize the mines MS Excel table
2. Connect to the geocoding service from Esri and geocode your mines
3. Connect to the department ArcGIS server and add the public Land Survey System (PLSS) feature class
4. Manually locate all mines that have a PLSS location
5. Compare your results with the results of your colleagues in class
6. Write a technical report

Methods

In order to complete this assignment the first step was to figure out which sixteen mines located in Wisconsin were my mines to geocode. For each of the mines that was assigned to me I had to normalize all of the information that was listed. To normalize each of the mines I would need to separate all of the data that was originally listed into different columns in an excel spreadsheet. This will help organize the data so than once I need to geocode the mines it will be easier to find the closest mine to the location that will be given to me. The data would be normalized into an extra eight columns for each of the sixteen mines.We would normalize the data by separating out the facility address field. Once that step was completed the next step was to add the table to ArcMap and start geocoding the mines that were assigned to us. In order to start geocoding we need to open up the address locator program that is available in ArcMap, This is where we look to see which mines have a PLSS location or a street address, if it has a PLSS location only then we are going to geocode those locations later in the assignment. The mine locations that have a street address are the first to geocode we find the data point that is represented on the map from the table that we added to ArcMap earlier. For us to begin geocoding we need to use this option in address locator named "Pick Address from Map" this will help us geocode the locations with a street address much easier. All of the initial data points are not located at the mine locations so we need to search for mines that are closest to the initial point or we can use google maps to help locate the mines. The next step was for us to manually locate the mines that just have a PLSS locations listed, in order to better locate the mines that just have a PLSS location we need to add a shapefile named "plss_qq_sects" this shapefile represents all of the different sections that are in Wisconsin. To easily find the PLSS location, it would most likely be placed in the middle of each section. Once we had all of the mines geocoded we would then compare our locations to the actual mine locations, and to our colleagues mines as well to see how accurate and precise our locations were. 

Results

Below are figures 1 and 2, figure 1 is representing four of the mines that needed to be normalized and geocoded. Figure 1 below is the excel file of the mines when they are not normalized. 

Figure 1: Excel file of Mines that are not normalized

Figure 2 represents the same exact mines as  figure 1 but instead it is showing how the mines have been normalized in an excel file. 

Figure 2: Excel file of Mines that are normalized

As you can see some of the mines don't have a PLSS location, while others may only have a PLSS location and some may have both. The next step was to create a map comparing both the real mines to the mines that I geocoded, and another map that is comparing my mines to my classmates who geocoded the same mines, so than we can compare to see how accurate each one of us were. Below are figures 3 and 4, figure 3 is representing the mines that I geocoded to the mines of my colleagues. 

Figure 3: Class Geocoded Mines compared to Individual Geocoded Mines

As you can see by looking at the figure above one can see that most of the mines that were geocoded by the class and the individual are mostly in the same area except for a few outliers that are located more towards the southern area of Wisconsin. Figure 4 represents the mean, standard deviation, maximum and minimum of all the distances from the individual geocoded mines to the class geocoded mines. 

Figure 4: Statistical Information of distance from Individual Geocoded Mines to Class Geocoded Mines

As you can see from the diagram above there were some mines that were extremely close to one another and others that were very far apart. The closest was 78 m and the farthest was 149,244 m but that was probably a different mine that wasn't supposed to be geocoded. Figure 5 is the table of the actual distance for each of the mines that were geocoded. 

Figure 5: Distance from Class Geocoded Mines

The figure above shows the distance between each of the mines, the distance is recorded in meters and is comparing every mine that was individually geocoded and every mine that was geocoded by my colleagues. Figure 6 is a map representing the individual geocoded mines compared to the actual mine locations in Wisconsin.

Figure 6: Real Mines compared to Individual Geocoded Mines

The map above is showing how close the individual geocoded mines were to the actual mines, as you can see most of the mines are in the general area, while others are much further away than expected. Figure 7 and 8 are below, figure 7 represents the statistical information for the distance between the real mines and the individual geocoded mines. While figure 8 is the actual table of the information listed in figure 7.

Figure 7: Statistical Information of distance from Individual Geocoded Mines to the Actual Mine Locations

Figure 8: Distance from Actual Mine Locations

The two figures above represent basically the same information but just show it in different ways, figure 7 shows what the mean/average distance was from the individual geocoded mines to the actual mines which came out to 22,572 m. While figure 8 lists all of the distances for each of the individual geocoded mines to the actual mine locations and lists the distance between the two in meters. 

Discussion

The next step is to discuss why the distance was so great between the individual geocoded mines to both the class's geocoded mines and the actual locations. The reason why there was such a large distance between some of the mines is because of three different things; accuracy, precision, and errors. All three of these things go hand in hand, for example if one is to have high accuracy data or create high accuracy data than it should be free of errors. But in this case we were trying to geocode the locations of these mines by just looking for the closest mines to the data point that were given to us. In other words we were not creating high accuracy data because we didn't have the exact locations of each mine and we were just trying to find the closest mine to the locations that were provided. The errors that most likely occurred during this process of geocoding these mine locations is known as operational errors. These errors are mainly user or processing errors that end up resulting in a imperfection in the data, it mainly occurs during the processing of the geographic data. It would be hard to figure out which points are actually correct and which ones aren't. The main errors that complicated things in this project were that of original source map and data processing and analysis. There were problems in field survey measurements, photogrammetric measurements and image analysis. The reason why these had the most errors is because when geocoding we were analyzing the land around the data points provided surveying for the mines that were closest, that would have caused the most problems when trying to correctly locate the actual mine locations. 

Conclusion

This assignment was the most beneficial assignment yet, because it made us work by ourselves and get a taste of what it is like not to follow step by step directions. Overall it was a great learning experience and gave us some hands on work for geocoding different locations and making our own data points. After completing this assignment I found out how important geocoding is when working on GIS and how important accurate data is and how easy it is not to create accurate data and create errors instead.