TOPIC 2 DATA SOURCES AND INPUT

Prepared by HAK

Topic outline
2.1 Spatial data sources for GIS 2.1.1 Describe the readily available data o Digitized data o Satellite Images o Air Photos o GPS Data o Old maps and plans 2.1.2 Describe the new data from fieldwork 2.2 Attribute data sources 2.3 Methods of GIS data input

2.3.1 Explain the digitizing method o Tablet digitizing o On screen digitizing

2.3.2 Explain The Scanning Method o Raster scanner o Vector Scanner 2.3.3 Explain the keying in methods

2.2.1 Describe the readily available data o Digital data o Hardcopy data
2.2.2 Describe the new data from fieldwork

Topic outline
2.4 Elements of data quality 2.4.1 Prescribe the following elements
o o o

2.5 Error sources in a GIS 2.5.1 State the following error : o Error during data acquisition o Error during data entry o Error during data storage o Error during data Processing

Spatial accuracy elements Attributes accuracy elements Consistency elements

o
o

Clarification elements
Current elements

2.4.2 Completeness elements

o o o

Completeness in coverage
Completeness in classification Completeness in authentication

DIGITIZED DATA

Often, digital data for a GIS project is not available, so it must be created from other existing sources like paper map
Digitizing is the process where features on a map or image are converted into digital format for use by a GIS Process by which coordinates from map, image or other sources are converted digital format Digitizing converts the features on the map into three basic data types: i. Points zero dimensional objects ii. Lines one dimensional objects iii. Polygons two dimensional objects

..cont.

Points, lines and areas (polygon) on maps or images represent real-world entities or phenomena There are three major steps to digitize spatial data: i. Prepare map or other image for digitizing. ii. Enter (digitize) features from the map. iii. Edit the digitized map for accuracy and to ensure quality.

Digitize data will produce vector data.

Source: Defense Mapping School National Imagery and Mapping Agency

..cont.

Real World
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 G B G G B B BG G B G G B G BK B G B B B B
600 500 400 Y-AXIS 300 200 River 100 100 200 300 400 500 X-AXIS 600 Trees

Trees House

Raster Representation

Vector Representation

Source: Defense Mapping School, National Imagery and Mapping Agency

Digitizing Errors, Node, & Line Snapping

Undershoot - nodes that do not quite reach line or another node - cause unconnected networks and unclosed polygons ii. Overshoot - lines that cross over existing nodes or lines How to overcome the above problems use node snapping and line snapping
i. iii. iv. v.

Dangling node Line crossing Missing line / label point

..cont.

..cont.

..cont.

1.

Missing line

..cont.

2.

Undershoot

..cont.

3.

Overshoot

..cont.

4.

Multiple Identifies (attributes) associated with undershoots or missing lines

SATELLITE IMAGES

Vector Data

SATELLITE WILL PRODUCE A RASTER IMAGE Raster images come in the form of individual pixels Each spatial location or resolution element has an associated pixel value, which indicates the coordinates, elevation, and any relevant attribute data, such as a color or ID number. VECTOR DATA Vector data comes in the form of points and lines that are geometrically and mathematically associated. The points are stored using their coordinates, and the lines are stored as a series of point pairs. Vector data structures produce smaller file sizes than raster images because only point coordinates are stored

digital image of the original photograph

A photograph could also be represented and displayed in a digital format by subdividing the image into small equal-sized and shaped areas, called picture elements or pixels.

REMOTE SENSING IMAGERY ( PREPARATION AND USE )

CLEAN DATA REMOVE NOISE

Preprocessing

IMPROVE CONTRATST AND INFORMATION MAKE THEMES THEME CATEGORIES TRANSFER INTO GIS

Enhancement

Thematic Analysis Classification Integration into GIS

The process for using remote sensing imagery

THE BASIC PROCESS IMAGERY AND PRE-PROCESSING GIS

Clean the data by removing electronic noise and correcting mistakes, such as missing scan lines. Often Radiometric and Geometric correction Data quality is improved and the image is then distributed to users.

ENHANCEMENT To enhance the data so that better information can be obtained This often includes improving visual contrast, such as changing subtle differences in gray tones into shades that are more distinctive, as an aid to interpretation. MAKE THEMES ( LAYER ) Enhanced data into selected themes For example, landscape images contain a variety of land covers ( such as various vegetation types ) that can be separated to provide

..cont.
CLASSIFICATION A vegetation theme, for example might include a sequence of classification, starting at overall land cover ( e.G forest, grassland, agricultural land and barren area. Classifying data according to a projects specefic purpose is an important data reduction step in keeping the amount of data manageable and focused. INTEGRATION INTO GIS Transferring imagery into GIS use as a background or complete integration as a full data set. That image used as background for on-screen digitizing are merely digital pictures, requires proper georeferencing, projection and database development.

OLD MAPS AND PLANS

Peta :

Satu lakaran yang dibuat atas kertas bagi mengambarkan sesuatu kawasan berdasarkan bentuk simbol piawai.

Pelan Satu lukisan yang diplot mengandungi butiran asal dan disediakan mengikut peraturan tertentu.Contoh Pelan kadaster.

Perbezaan peta dan pelan Peta

Kawasan besar Tidak detail Contoh : Peta jalan, peta topografi

Pelan
Kawasan kecil ( lebih khusus ) Lebih detail eg, nama pemilik, luas, jarak dll. Contohnya : Pelan Kadaster, Pelan enginering

CONTOH lain HUBUNGAN PETA DENGAN GIS

Hubungan Peta Dengan GIS????

PETA ( SALINAN KERAS )

DIGITIZING ( PENDIGITAN )

Data Spatial

Attribute

GLOBAL POSITIONING SYSTEM (GPS) DATA

GPS determines a location on the surface of the Earth

FEATURES TYPES There are three types of Feature which can be mapped: Points, Lines and Areas. A Point Feature is a single GPS coordinate position which is identified with a specific Object A Line Feature is a collection of GPS positions which are identified with the same Object and linked together to form a line An Area Feature is very similar to a Line Feature, except that the ends of the line are tied to each other to form a closed area

AERIAL PHOTO

Before digitizing, we need to georeference aerial image ( photo ). Georeferencing is the process of defining how raster data is situated in map coordinates Use intersections of the road, could be easily identified on the photo.

THE NEW DATA FROM FIELDWORK

Field Survey ( Traversing )

Collect data ( bearing and distance )

Download data from equipment ( Total Station ) to computer ( CDS software )

Process data in CDS and Autocad software ( Produce vector data )

Export data vector using Universal Translator in MapInfo software

Key-in Attribute in MapInfo

Make an analysis in MapInfo

DIGITIZING
Process whereby an analog (or paper) map is converted into a digital format There are three primary methods for digitizing spatial information:
Two types of digitizing :
Tablet Digitizing On-Screen Digitizing

Tablet digitizing requires a person to enter coordinate information through the use of a digitizing tablet and digitizing puck.
A digitizing tablet is a hardened surface with a fine electrical wire grid under the surface. Digitizing tablets are either hardened, more stationary tables or rollup boards designed for portability

A digitizing puck is an electrical device with cross hairs and multiple buttons to
perform data entry operations An operator then enters the information using the puck.

Tablet digitizing

TABLET DIGITIZING HOW IT WORKS

When the user places the digitizing puck over a location on the tablet, and presses one of the buttons, the wire mesh beneath the tablet records the location of the puck Digitizing tablets are very accurate, with more expensive tablets able to measure objects to within 0.006 mm. This means that if you were to press the entry button on the puck continuously at one spot, the coordinate value received from the tablet would only vary by 0.006 mm. The coordinate, as referenced by the tablet is then

x,y

stored in the computer.

TABLET DIGITIZING THE PROBLEMS

tedious, and operator fatigue Due to environmental conditions (such as humidity), the source materials may actually change (due to shrinking and expansion). If you notice the person below, the actual map is usually a foot or two away from the operator. So, small subtitles in the geographic objects may be missed when you are so far away

ON-SCREEN DIGITIZING

On-screen digitizing is an interactive process in which a map is created using previously digitized of scanned information. Commonly called "heads-up" digitizing because the attention of the user is focused up on the screen, and not on a digitizing tablet

Advantages : Ease of zooming for detailed work Full array of editing ( delete, copy and so-on ) and text features ( font and text editing ) Quick and inexpensive method for producing GIS

SCANNING

Process of converting maps into digital form

Pengimbasan ( Scanning )
device that converts maps into digital form Prinsip: Mengimbas garisan-garisan secara berturutan Merekodkan jumlah cahaya yang dipantulkan

Perbezaan cahaya diskalakan kepada bi-level hitam atau putih (1 bit per piksel) atau multiple-gray levels (8,16 atau 32 bit)
Resolusi spatial adalah dari sekecil 100 dpi hinggalah kepada 1800 dpi dan lebih Alat pengimbas GIS dalam lingkungan 400 1000 dpi

Dots per inch (DPI) is a measure of spatial printing or video dot density

PROCESS OF SCANNING
PREPARE MAP

PRE-PROCESSING

Prepare Map Although scanners will copy any data, map scans produce best results using bi-tonal data ( balck-andwhite lines and features, not gray or color ) Select the area to be scanned, reviewing procedures, and selecting the desired resolution
Pre-processing Clean the map of stains, specks and other useless marks that make data noise Clean map reduce the editing process Finding real-world coordinate control points for georeferencing Scan the Map Each scanner has particular controls and procedures, but it is usually a standard process of either positioning the map in the scanner or feeding it into the system. Scanning produce a raster image

SCANNING

PROSES PENGIMBASAN
Penyediaan Peta -Walaupun alat pengimbas boleh menyalin semua data, bagi menghasilkan keputusan imbasan peta yang baik menggunakan bitonal data ( black and white lines and features, not gray or color ) -Pilih kawasan yang hendak diimbas, penelitian ( review ) dan pilih resolusi yang sesuai. Resolusi Penyusunan struktur grid ( baris dan lajur ) atau saiz sel dunia sebenar ( contoh 30m diatas permukaan bumi. )

Penyediaan Peta

Pra-pemprosesan

-Membesihkan peta daripada tompok kotor, bintik2 atau tanda-tanda lain yang x berguna data noise .

Pra-pemprosesan

- Pembersihan peta mengurangkan proses pengeditan. - Dapatkan koordinat titik kawalan untuk georefering

Pengimbasan

Pengimbasan

-Langkah terakhir buat pengimbasan - Setiap alat pengimbas mepunyai kaedah yang tertentu, tetapi biasanya proses yang biasa ialah meletakkan peta di atas alat pengimbas dan disalurkan kepada sistem

Tiga sebab utama pengimbasan dilakukan

Mengurangkan kesan lusuhan dan koyakan,memudahkan pengaksesan, menyediakan pangkalan data bersepadu dan mengindeknya secara geografi Filem, peta, fotoudara dan imej satelit diimbas untuk memberikan latarbelakang geografi bagi data vektor Peta dan imej diimbas sebelum ditukarkan kepada bentuk vektor (vectorization)

Keyboard Entry
Papan Kekunci :

Digunakan untuk memasukan data atribut sebagai tambahan kepada proses pendigitan dan pengimbasan.
Cara ini amat berkesan untuk perlaksanaan yang memerlukan ketepatan tinggi seperti penggunaan kadaster, bearing dan jarak sempadan lot-lot tanah dimasukkan dan koordinat batu sempadan dihitung. Kemasukan data (nilai ukuran atau koordinat) dengan papan kekunci akan menghasilkan pengkalan data GIS yang berketepatan tinggi.

ELEMENTS OF DATA QUALITY

SPATIAL ACCURACY - deviance in the geographic location of an object from its true ground position.
TWO COMPONENTS TO SPATIAL ACCURACY

relative accuracy
The positioning of map features relative to one another.

absolute accuracy
The accuracy of data elements with respect to a coordinate scheme Eg : UTM

Contoh Kedudukan relatif dan Kedudukan Mutlak

Kedudukan relatif

Kedudukan Mutlak

ATTRIBUTE ACCURACY Attribute accuracy is the closeness of attribute values to their true value Location does not change with time, but attributes often do Attributes may be discrete or continuous. Discrete attributes may have a finite number of values I.e., land use, vegetation type, etc. Continuous attributes may have an infinite number of values I.e., elevation, property value, isotherms, isohyets, etc.

Attribute Accuracy must be analyzed in different ways depending on the nature of the data

Continuous attributes (surfaces) such on a DEM or TIN:

Accuracy is expressed as measurement error e.g. elevation accurate to 1 m

In testing attribute accuracy: An error of omission occurs when a points class on the ground incorrectly recorded in the database An error of commission occurs when the class recorded in the database does not exist on the ground

CURRENT ELEMENTS
A critical factor of any database is Time. Demographic information is time-sensitive, changing significantly even over the course of a year

Land use information is also time-sensitive as many areas of the world

experience rapid urbanization especially in less-developed countries near large cities Time of year is another factor in data collection seasonal changes in crop type e.g, Spring wheat or Summer wheat, Summer vegetables or Fall vegetables The time aspect of data quality is usually the date of the source material Topographic maps may include the original source date, as well as any recent

dates
Date of acquisition is also a factor for geographic information that changes rapidly over time Forestry maps are generally updated on a 5-10 year basis Agricultural maps are updated as rapidly as weekly during growing seasons

LOGICAL CONSISTENCY
This component is concerned with determining the faithfulness of the data structure for a data set. This typically involves spatial data inconsistencies such as incorrect line

intersections, duplicate lines or boundaries, or gaps in lines.

These are referred to as spatial or topological errors.

Vector Topology helps deal with:

slivers

overshoots dangles

Not sharing border

COMPLETENESS
What is completeness ? Completeness concerns the degree to which the data exhausts the universe of

possible items.
Are all possible objects included in the database? Is the database affected by rules of selection, generalization and scale? Completeness consists of : Completeness in coverage Completeness in classification Completeness in authentication

COMPLETENESS IN COVERAGE
What is Coverage? A coverage is a georelational data model that stores vector datait contains both the spatial (location) and attribute (descriptive) data for geographic features. Completeness of coverage is the proportion of data available for area of interest Progressively updated data sets, a patchwork of more recent data, may work for current status of a resource Older, more complete, data sets work best for comparative analysis where consistency is important

COMPLETENESS OF CLASSIFICATION
Completeness of Classification assesses how well the chosen classification is able to represent the data All data should be encoded at the selected level of detail Differences may occur owing to the individual or organizations that produced the maps. Different government agencies mapping adjacent areas may have boundary-matching problems, even though the maps are accurate in terms of position and classification

COMPLETENESS OF VERIFICATION
Completeness of Verification examines the amount and distribution of field
measurement or other independent sources of information used to develop the data In Geology, standard field data techniques use solid lines for mapping visible and verifiable rock types boundaries, while inferred boundaries are indicated with dashed lines (Air Geology) A data quality check for the geologists field data

ERROR DURING DATA ACQUISITION

Geometrical (positional) and semantic (classification) errors in the compilation of the source maps Interpretation and classification errors in remotely sensed data causing attribute error. Inaccuracy in source data i.e. from field sampling.

Inaccuracies due to the range (fuzzy) character of natural boundaries, e.g.

vegetation or soil type. Errors due to source data being out of date. This happens frequently in case of maps. Other errors with maps may be due to scale and generalization. Limitation of survey equipment Improper georeferencing of remotely sensed and photogrammetric data.

ERROR DURING DATA ENTRY

Digitizing errors due to operator mistakes and limited precision of digitizer. Errors during digitizing may be due to limitations of maps as given earlier or due to operational error. These operational errors may include psychological, physiological and digitizing method error.

Errors in attribute data entry (typing errors).

ERROR DURING DATA STORAGE Errors due to the limited precision with which coordinates and other numerical data are stored. Errors arising from rasterization / vectorization.

ERROR DURING DATA PROCESSING a. Input: Digitizing: human error, the width of a line, spikes, knots, also entering attribute data. Dangling nodes (connected to only one arc): permissible in arc themes (river headwaters etc.). Pseudo-nodes (connected to one or two arcs) - permissible in island arcs, and where attributes change, e.g. road becomes paved from dirt or vice versa. Projection input error.

b. Manipulation Interpolation of point data into lines and surfaces. Overlay of layers, digitized separately, e.g. soils and vegetation.

Density of observations.
Inappropriate or inadequate inputs for models

c. Output Scale changes - detail and scale bars. Color palettes: intended colors don't match from screen to Printer

REFERENCE

www.utsa.edu/LRSG/Teaching/ES2113/L6_GIS_data_sour ce.ppt http://nptel.iitm.ac.in/courses/Webcoursecontents/IITKANPUR/ModernSurveyingTech/lectureE_40/E_ 40_3.htm