WO2010068185A1 - Method of generating a geodetic reference database product - Google Patents

A method of generating a geodetic reference database product is disclosed. The method comprises acquiring mobile mapping data captured by means of digital cameras, range sensors and position/orientation determination means mounted to a vehicle driving across the earth surface, the mobile mapping data comprising simultaneously captured image data, range data and associated position/orientation data in a geographic coordinate system. Stationary earth surface features are determined from the mobile mapping data by combining the image data, range data and associated position/orientation data. A 3D-orthorectified images for the stationary earth surface features is generated. At least one pixel of the 3D-orthorectified image has a corresponding xyz coordinate in the coordinate reference system. The 3D-orthorectified images are stored in a database to obtain the geodetic reference database product.

Method of generating a geodetic reference database product

Field of the invention

The present invention relates to the field of generating a geodetic reference database product.

The invention further relates to a computer implemented system for generating a geodetic reference database product, a geodetic reference database product, a computer program product and a processor readable medium provided with the computer program product or the geodetic reference database product. A geodetic reference database product can be useful when orthorectifying different images of the same geographic area

Background of the invention

Ground control points (GCP' s) are used in image rectification of satellite, aerial or aero survey imagery to standard map projections. Image rectification in GIS converts images to a standard map coordinate system. This is done by matching ground control points (GCP) in the mapping system to points in the image. These GCPs enable the calculation of necessary image transformations.

A ground control point can be any point on the surface of the earth which is recognizable on remotely sensed images, maps and aerial /satellite photographs and which can be accurately located on each of these. A ground control point is a point on the surface of the earth of known location (i.e. fixed within an established co-ordinate reference system). GCP' s are used to geo-reference image data sources, such as remotely sensed images or scanned maps. A GCP could be:

- a copy of a part of a paper map showing a selected point and its surrounding;

- an image chip from a scanned map showing a selected point and its surrounding;

- an image chip from a digital map showing a selected point and its surrounding;

- a written description or sketch of the selected point;

- an image from an aerial/satellite or ground based photo showing a selected point and its surrounding; or - any other representation of a specific location suitably documented so as to be recognizable in an aerial/satellite image and suitably documented so as to convey a set of coordinates defining its location accurately according to a coordinate system..

A GCP comprises associated precise X, Y and Z coordinates in a coordinate reference system. A GCP describes an earth surface feature which is clearly identifiable in a satellite or aerial imagery. The most significant requirement for a GCP is it's visibility in the image to be orthorectified. A secondary characteristic is that it be durable. A GCP should ideally have a size which is at least 4 times the size of a pixel in the image to be orthorectified. Earth surface features used for defining GCP' s can be cultural features, line features and natural features.

A cultural (man made) feature is usually the best point to use as GCP. It covers road intersections, road and rail road intersections, road and visible biogeographic boundary intersections, such as the intersection of a road and the boundary line between a forest and an agricultural field, river bridges, large low buildings (hangars, industrial buildings, etc), airports, etcetera.

Line features could be used when they have well defined edges in the imagery. The GCP is normally selected as a center of the intersection of two line features. The two line features forming the intersection have to cross with an angle larger than 60 degrees.

Natural features are generally not preferred because of their amorphous shapes. It may however be necessary to use natural features in areas lacking suitable built features. If a natural feature has well defined edges, it may be used as a GCP. It could be forest boundaries forest paths, forest clearings, river confluence, etc. When selecting such points it must to be taken into account that certain boundaries can be subject to variations (forest, water bodies) in time. In situations where there are insufficient suitable features, it is possible for the surveyor to create an observable feature for the purpose of identifying a GCP.

Primary difficulties in the rectification process occur when the accuracy of the map points are not well known and/or when the images lack clearly identifiable points to correspond to the maps. Image rectification is a standard feature available with commercial GIS software packages. An orthophoto or orthophotograph is an aerial photograph that has been geometrically corrected ("orthorectified") such that the scale of the photograph is uniform, meaning that the photo can be considered equivalent to a map. Unlike an uncorrected aerial photograph, an orthophotograph can be used to measure true distances, because it is an accurate representation of the earth's surface, having been adjusted for topographic relief, lens distortion, and camera tilt.

Orthographic views project at a right angle to the data plane. Perspective views project from the surface onto the datum plane from a fixed location.

Aerial photographs are useful for providing spatial information, but they usually contain geometric distortion. Maps that are geometrically precise are called planimetric or orthographic maps. An orthographic map plots the position of objects after they have been projected onto a datum or reference plane. Spatial features above or below the plane are projected up or down in a vertical format onto the horizontal plane. Most aerial photographs unfortunately show a non-orthographic perspective view.

To geo-reference or rectify aerial or satellite imagery, a set of GCP' s has to be selected for each image. The GCP' s of a set should be uniformly selected in the image. Points near the edges of an image should be selected and preferably with even distribution in the image. The set of GCP' s should preferably also respect terrain variations in the scene, i.e. select point at both highest and lowest elevations.

GCP' s could be generated by a human going into the field and gathering both an image or corresponding description of the GCP and the corresponding X, Y and Z coordinate in a coordinate reference system by a position determination means of for example a GPS receiver. In "Accurate mapping of Ground Control Point for Image Rectification and Holistic Planned Grazing Preparation" by Jed Gregory, et al., GIS Training and Research Center, Idaho State University Pocatello, ID 83209-8130, Oct 2006, GCP's had to be established and their exact spatial location recorded to ensure accurate georectification of the imagery. Ten GCP's were setup strategically throughout the area to be georectified. The GCP's were setup using two strips of plastic, six inches wide and six feet long, laid across each other in the shape of a cross (+). All GCP's were oriented with each arm of the cross pointing in one of the four cardinal directions (north, south, east, west). After placement of each GCP a GPS location was recorded at the center of the cross using a Trimble GeoXT GPS unit. Said document makes clear the huge amount of time and effort that is necessary to collect accurate GCP' s. Only after such a set up the aerial images can be taken and must be done before weather and other effect move or obliterate the markers.

There are basically two corrections that are made in an orthorectification process. Orthorectification is the transformation of a perspective view image into an image wherein each pixel has a known XY-position on the geoid describing the earth surface and wherein each pixel is regarded to be viewed perpendicular to the earth surface in said XY-position. First, any shifts (translation and rotation errors) tilts or scale problems can be corrected and second the distortion effects of elevation changes can be corrected. In current orthorectification processes applied to images, elevation distortion is the major cause of horizontal errors. This is illustrated in figure 1. A camera mounted in an aircraft 1 records perspective view images of the earth surface 2 (shown here in profile). However, only one pixel in the image can be representing an orthogonal view of the earth surface and the other pixels are all angled view representations of the earth surface. Figure 1 shows a profile of the earth surface for a given y coordinate. Horizontal line 3 is assumed to represent a profile of a reference surface of the earth for the given y coordinate in a coordinate reference system, for example WGS84 or any other geoid describing the earth surface in a coordinate reference system. Shown is a building structure 4, for example a bridge, on the earth surface whose xyz positions on the earth surface 2 and height are known. Furthermore, the position and orientation in the coordinate reference system of the capturing point 5 of the aerial image is known (for example by means of accurate GPS and/or other position/orientation determination means). By means of geometry, it is possible to determine the pixels of the upper side of the building structure and to determine the corresponding x,y position. However, if the height, i.e. z coordinate, of the earth surface with respect to the reference surface 3 is not known, a first terrain-induced error 6 will be introduced in the orthorectified image. Similarly, if also the height of the building structure is not known an additional building height-induced error 7 will be introduced in the final orthorectified image. In that case the upper side or the building structure can be projected meters aside the correct xy position in the orthorectified image. In case the building structure is a bridge, the road on the bridge will be projected erroneously if the elevation information with respect to the reference surface is not (accurately) known. Figure 2 illustrates this type of error.

Figure 2 shows an orthorectified image wherein a digital elevation model (DEM) is used to oithorectify the aerial image. A DEM, or "bald earth" , which it is often referred to as, is created by digitally removing all of the cultural/built features inherent to a digital surface model DSM by exposing the underlying terrain. A DSM is a first surface view of the earth containing both location and elevation information in a coordinate reference system. The USGS 10m National Elevation Data Set (NED) is a cost-effective DEM available but fails to allow for accurate orthorectification for bridges, buildings and elevated structures as shown in figure 2. By not taking into account the height of the bridges, the upper sides of the bridges are shifted with respect to the real location of the bridges. The real location of the bridges in figure 2 are indicated by the white lines superimposed on the orthorectified image. Figure 3 shows an orthorectified image wherein an accurately geo-coded DSM is used to oithorectify the aerial image. In can be seen that by using the correct heights of the building structures, the upper surface of the building structures, i.e. the road surfaces, are correctly projected on the orthorectified image space. The upper surface of building structures are correctly projected when the white lines indicating the outlines of the building structures coincide with the visual outlines in the orthorectified image.

It should be noted that both DEMs and DSMs provide only a model of the earth surface. They do not comprise information which is easy recognizable on sensed images, maps and aerial photographs. Without GCP's associated with a DEM or DSM, they cannot be used to orthorectify such images. The accuracy of the GCP's used will determine the accuracy of the resultant image or orthorectification process.

Geographic Information Systems often combine both digital map information and orthorectified images in one view. Information from the image can be extracted or analyzed to add to, correct or validate the digital map information. Similarly, orthorectified images could be used to extract digital map information for use in a navigation device. In both situations it is important that the location of features in the orthorectified images correspond to their real locations on the earth. In the first case, due to incorrect heights, the position of road surfaces in the orthorectified image do not coincide with the corresponding road surfaces from the digital map. For an example see figure 2. In this case, the navigation device could measure positions that are different from those in its map database that were extracted from the poorly orthorectified image and could provide an alarm erroneously informing the user of the navigation device about unsafe driving conditions.

A requirement for generating a correct orthorectified image of the road surfaces from an aerial image or satellite image is that sufficient GCP' s related to the road surface are present within the area represented by the orthorectified image. At present, the costs of orthorectification increase linearly with the number of GCP' s to be captured by humans. The more, GCP' s are needed to obtain the required accuracy of an orthorectified image, the more human effort is needed.

There is a lack of cheap, accurate and well distributed ground control points to help control positionally accurate navigation and mapping applications. Furthermore, Advanced Driver Assistance Systems (ADAS) require accurate 3D positional information about the road to control such systems. For these applications it is important that the road surface is correctly positioned in the orthorectified image. To be able to do this, elevation information about the road surface is needed especially the elevation information of bridges, banks, elevated highways and flyovers.

The current state of ground control products for calibration and rectification of geospatial imagery is patchy and inconsistent in almost all areas of the globe.

There is need for a geodetic reference database product, that comprises sufficient GCP' s or ground control information to orthorectify aerial or satellite imagery with enough accuracy to use the orthorectified images as a reliable data source for GIS applications at least as it applies to the surface of roads.

Summary of the invention

The present invention seeks to provide an alternative method of generating a geodetic reference database product, that could be used in numerous GIS application such as: Image orthorectification, base mapping, location-based systems, 3D- visualisation, topographic mapping, vehicle navigation, intelligent vehicle systems, ADAS, flight simulation, in-cockpit situational awareness.

According to the invention, the method comprises: - acquiring mobile mapping data captured by means of digital cameras, range sensors and position determination means including GPS and IMU mounted to a vehicle driving across the earth surface, the mobile mapping data comprising simultaneously captured image data, range data and associated position data in a geographic coordinate system;

- determining stationary earth surface features from the mobile mapping data by combining the image data, range data and associated position data;

- generating 3D-orthorectified images for the stationary earth surface features, wherein at least one pixel of the 3D-orthorectified image has a corresponding xyz coordinate in the coordinate reference system;

- storing the 3D-orthorectified images in a database to obtain the geodetic reference database product.

The invention is based on the recognition that to accurately oithorectify sensed aerial and satellite images an accurate 3D model of the earth surface is needed. Furthermore, the relation of the sensed image and the 3D model has to be determined. Current 3D models such as DSM and DEM describe the earth surface in terms of 3D coordinates. These 3D coordinates do not have an associated color value corresponding the earth surface when viewed from above. Therefore, it is not possible to align the 3D models and the sensed images. Furthermore, Pixel size of commercially available images is 5.0 m with a horizontal accuracy RSME of 2.0 m and a vertical accuracy RMSE of 1.0 m. These resolution and accuracy limit orthorectification processes to generate orthorectified images with a higher accuracy.

Mobile mapping vehicles capture mobile mapping data captured by means of digital cameras, range sensors such as laser/radar/sonar sensors, and position determination means including GPS and IMU mounted to a vehicle driving across the road based earth surface, the mobile mapping data comprising simultaneously captured image data, range data and associated position data in a geographic coordinate system. Position determining means enables us to determine the position with a horizontal absolute accuracy of 0.5 - 2 m l sigma and a vertical accuracy of 1.5 - 3 m. By means of the laser/radar sensor data in combination of the associated determined position data it is possible to create a surface model with a pixel size from 2 cm x 2 cm, a relative horizontal accuracy of 0.5m over 100m cm and a relative vertical accuracy of 0.35m/ over 100m. The surface model could be used to transform the image data into oithorectified images of the earth surface with a pixel size from 2 cm, a relative horizontal accuracy of 0.5m over 100m. The height information from the surface model could be added to each pixel of the oithorectified image to obtain a 3D oithorectified image having a relative vertical accuracy of 1.5m. From these 3D orthorectified images, stationary earth surface features or Ground Control Objects GCO, such as road paintings, could be extracted and stored as GCP' s in a database for orthorectification of imagery. A characteristic of the stationary earth surface feature is that it has a shape and size that the feature could be recognized in the imagery to be rectified. The present invention enables us to generate a huge amount of GCP' s in an easy way and short time period. It is not necessary that a human first identifies locations suitable for orthorectification, goes into the field to set-out the mark to visually identify the location or goes to the identified location and measures the accurate position of the GCP. Another advantage is that it uses ground control objects, i.e. road markings, that could not be selected in exiting paper maps or digital maps as they are not present in said maps. The method helps us to capture GCP' s which could up till now only be collected manually by humans making a survey.

The method according to the invention combines the best of three worlds, accurate position determination, processing of high resolution laser/radar data or other range data and processing of high resolution images. Both the range data and image have a high resolution and accuracy as they represent data captured at relative short distance to the recorded surface compared to aerial imagery. This allows us to use less expensive digital cameras and laser sensors.

A stationary earth surface feature could be any physical and visual feature in the earth's surface selected from a group comprising at least one of: road paintings, stark changes in pavement type, monument bases, sharp curb edges, metal caps for man-hole covers and sewer grates, unique low-lying geological features and the like. A stationary earth surface feature can be any linear features, such as lane markings, and may even be road segments. Whereas in the prior art a GCP point is used to identify one point in a coordinate reference system, a linear feature of for example a dashed lane marking, extends the single point approach of a GCP to identify an in principle near infinite number of accurately positions along the line and thus providing a corresponding number of traditional GCP points each identifying only one single reference point in a coordinate reference system. Furthermore, in case of the dashed lane marking, they are basically uniform distributed along the linear feature and provide a set of GCP for generating a oithorectified image with uniform accuracy in the area of the image corresponding to the line feature.

In a further embodiment the 3D-orthorectified images are combined to obtain a continuous linear control network, wherein branches of the network correspond to road segments; and storing the continuous linear control network in the geodetic reference database product. The continuous linear control network, provides us not individual GCP' s but a continuous network of GCP' s which can be detected and matched with the aerial or satellite imagery. By means of the continuous linear control network, it is possible to significantly improve the orthorectification of the roads in the imagery. The continuous linear control network provides a very accurate DEM or DSM of the surface of the roads and road structures with a resolution which is at present 5 times better than commercially available DSMs or DEMs.

In an embodiment, the stationary earth surface features are determined by generating oithorectified images from the mobile mapping data by combining the image data, range data and associated position data. Then, the oithorectified images are processed to determine areas of interest having high contrast in the oithorectified images. Finally, areas of interest having predefined characteristics are selected as the stationary earth surface features. Road markings such as "Warning of 'Give Way' just ahead,", Stop lines, guidance arrows, pedestrian crossings, tapered road edge lines at highway exits, hatched markings, chevron markings, etc., should be visible at any time and can be easily detected by image processing. This enables us to generate cheap, accurate and well distributed ground control points to help control positionally accurate navigation and mapping applications.

It is further an object of the invention to provide a method which enables a computer implemented system to generate content to be stored in a ground control database. Short description of drawings

The present invention will be discussed in more detail below, using a number of exemplary embodiments, with reference to the attached drawings, in which

Fig. 1 shows schematically a source of distortion in the orthorectification process;

Fig. 2 shows an oithorectified image with use of a DEM;

Fig. 3 shows an oithorectified image with use of a DSM;

Fig. 4 shows a flow diagram of the method according to the invention;

Fig. 5 is a block diagram of an exemplar computer system for implementing the method according to the invention;

Fig. 6 shows a MMS system with a camera and a laser scanner;

Fig. 7 shows a diagram of location and orientation parameters;

Fig. 8 shows some examples of stationary road surface features.

Detailed description of exemplary embodiments

Figure 4 shows a simplified flow diagram of the method according to the invention. The method start with action 400, by acquiring mobile mapping data. Mobile mapping data is captured by means of digital cameras, laser sensors and position/orientation determination means including GPS and IMU mounted to a vehicle driving across the earth surface, the mobile mapping data comprising simultaneously captured image data, laser data and associated position/orientation data in a geographic coordinate system. A vehicle provided with position/orientation determination means, laser sensors and digital cameras for collecting mobile mapping data is called a mobile mapping system MMS. It should be noted that in stead of laser sensors any other range sensor, such as a LADAR, LIDAR and RADAR, could be used to capture range data that can be used to generate a 3D model or image.

Figure 6 shows a MMS system that takes the form of a car 20. The car 20 is provided with one or more cameras 29(i), i = 1, 2, 3, ... I and one or more laser scanners 23(j), j = 1, 2, 3, ... J. The looking angle of the one or more cameras 29(i) can be in any direction with respect to the driving direction of the car 21 and can thus be a front looking camera, a side looking camera or rear looking camera, etc. The viewing window(s) of the camera(s) 29(i) cover(s) the whole road surface in front the vehicle. Preferably, the angle between the driving direction of the car 21 and the looking angle of a camera is within the range of -45 degree - +45 degree on either side. The car 21 can be driven by a driver along roads of interest.

The car 21 is provided with a plurality of wheels 22. Moreover, the car 21 is provided with a high accuracy position/orientation determination device. As shown in figure 6, the position/orientation determination device comprises the following components:

• a GPS (global positioning system) unit connected to an antenna 28 and arranged to communicate with a plurality of satellites SLi (i = 1, 2, 3, ...) and to calculate a position signal from signals received from the satellites SLi. The GPS unit is connected to a microprocessor μP. Based on the signals received from the GPS unit, the microprocessor μP may determine suitable display signals to be displayed on a monitor 24 in the car 1, informing the driver where the car is located and possibly in what direction it is traveling. Instead of a GPS unit a differential GPS unit could be used. Differential Global Positioning System (DGPS) is an enhancement to Global Positioning System (GPS) that uses a network of fixed ground based reference stations to broadcast the difference between the positions indicated by the satellite systems and the known fixed positions. These stations broadcast the difference between the measured satellite pseudoranges and actual (internally computed) pseudoranges, and receiver stations may correct their pseudoranges by the same amount.

• a DMI (Distance Measurement Instrument). This instrument is an odometer that measures a distance traveled by the car 21 by sensing the number of rotations of one or more of the wheels 22. The DMI is also connected to the microprocessor μP to allow the microprocessor μP to take the distance as measured by the DMI into account while calculating the display signal from the output signal from the GPS unit.

• an IMU (Inertial Measurement Unit). Such an IMU can be implemented as three gyro units arranged to measure rotational accelerations and translational accelerations along three orthogonal directions. The IMU is also connected to the microprocessor μP to allow the microprocessor μP to take the measurements by the DMI into account while calculating the display signal from the output signal from the GPS unit. The EvIU could also comprise dead reckoning sensors.

It will be noted that one skilled in the art can find many combinations of Global Navigation Satellite systems and on-board inertial and dead reckoning systems to provide an accurate location and orientation of the vehicle and hence the equipment (which are mounted with know positions and orientations with references to a reference position and orientation of the vehicle).

The system as shown in figure 6 is a so-called "mobile mapping system" which collects geographic data, for instance by taking pictures with one or more camera(s) 29(i) mounted on the car 21. The camera(s) 29(i) are connected to the microprocessor μP. The camera(s) 29(i) in front of the car could be a stereoscopic camera. The camera(s) could be arranged to generate an image sequence wherein the images have been captured with a predefined frame rate. In an exemplary embodiment one or more of the camera(s) are still picture cameras arranged to capture a picture every predefined displacement of the car 21 or every interval of time. The camera(s) 29(i) send the images to the μP. In an embodiment, the mobile mapping vehicle comprises three cameras, one front looking camera and a camera at each side having a looking axis within a range of 30 - 60 degree and preferably 45 degree, with respect to the heading direction of the vehicle. In that case, the front looking camera captures images especially suitable for detecting road directions above the road surface and the side looking cameras captures images especially suitable for detecting objects, such as road signs, along the road.

Moreover, the laser scanners 23(j) take laser samples while the car 21 is driving along roads of interest. The laser samples, thus, comprise data relating to the environment associated with these roads of interest, and may include data relating to the road surface, building blocks, trees, traffic signs, parked cars, people, direction signposts, the road side etc. The laser scanners 23(j) are also connected to the microprocessor μP and send these laser samples to the microprocessor μP.

It is a general desire to provide as accurate as possible location and orientation measurement from the three measurement units: GPS, EvIU and DMI. These location and orientation data are measured while the camera(s) 29(i) take pictures and the laser scanners 23(j) take laser samples. Both the pictures and laser samples are stored for later use in a suitable memory of the μP in association with corresponding location and orientation data of the car 21, collected at the same time these pictures were taken. The pictures include visual information, for instance, as to the road surface, building blocks, trees, traffic signs, parked cars, people, direction signposts, monuments, etc. The laser scanners 23(j) provide a cloud of laser scanner points dense enough to visualize in a 3D representation of along-the-road information. In an embodiment, the laser scanner(s) 23 (j) are arranged to produce an output with minimal 50 Hz and 1 deg resolution in order to produce a dense enough output for the method. A laser scanner such as MODEL LMS291-S05 produced by SICK is capable of producing such output. In a minimal configuration, one laser scanner sensing the road surface after or ahead the car 21 could be mounted on the car. In another configuration, two laser scanners should be used, and preferably four. In an embodiment, two laser scanners positioned at the front of the car have a rotation axis having an angle of about 45 degree with respect to the driving direction of the car and two laser scanners positioned at the back of the car have a rotation axis having an angle of about 45 degree with respect to the driving direction. Unpublished International Application PCT/NL2007/050541 discloses further advantages of using a set-up wherein two laser scanners scan the same surface at different time instants. It should be noted that in stead of laser scanners any other range sensor could be used that provides distance information or a dense point cloud.

Figure 7 shows which position signals can be obtained from the three measurement units GPS, DMI and EVIU shown in figure 6. Figure 7 shows that the microprocessor μP is arranged to calculate six different parameters, i.e., three distance parameters x, y, z relative to an origin in a predetermined coordinate system and three angle parameters ω_x, ω_y, and ω_z, respectively, which denote a rotation about the x-axis, y-axis and z-axis respectively. Preferably, the z-direction coincides with the direction of the gravity vector. The global UTM or WGS84 coordinate system could be used as predetermined coordinate reference system. It should be noted that the method according to the invention can be used with a local coordinate reference system, such as NAD 83 and other national grid systems. The six different parameters provide the 6- degree of freedom which is needed to track the position and orientation of the vehicle in time. The camera(s) and laser scanners have a fixed position and orientation with respect to the car 21. This enables us to determine accurately from the six parameters the position of each laser sample in the coordinate reference system and the position and orientation of the camera in the coordinate reference system at the moment of taking an image or laser sample. In action 402, the position data, image data and laser data is combined and processed to generate orthorectified images. An orthorectified image is an image representing an area of the surface of the earth. Generally, a geoid describes a model of the earth surface related to a geographic reference system. Each pixel of an orthorectified image has an associated xy-position in the geographic reference system. Furthermore, each pixel value is regarded to represent the earth surface as seen perpendicular to the orientation of the surface at the xy-position.

International Application WO08044927 discloses a method to generate orthorectified tiles and mosaics from Mobile Mapping Images. The images are projected on a virtual plane representative of the road surface ahead the mobile mapping vehicle. A real surface model of the road surface can be derived from the laser data. A skilled person can adapt easily the method disclosed in WO08044927 to project the images on the real surface model instead of the virtual plane to produce the orthorectified images.

The orthorectified images represent the surface of the earth where the MMS system has been driven across the earth surface. In most cases they represent the road surface, pavement and a part of the road side along the road.

In action 404, the orthorectified images are applied to image processing to determine areas of interest which are easy to detect in images. Preferably, an area of interest AOI is a region having high contrast with the surrounding are selected. As the color of road markings differs from there road background color, an image of the road surface can be applied to a threshold algorithm to determine the areas of the road markings. Unpublished International Application PCT/NL2007/050569 discloses an improved method of filtering orthorectified images and producing linear lane information, such as lane dividers, edge lines, from them. A characteristic of road markings is that they are stationary earth surface features. Furthermore, there are road markings that can be identified in aerial imagery. This makes those road markings very suitable to be used as GCP. Furthermore, there are many road markings having a shape that can be used to identify the orientation of the road markings. This makes it possible to define uniquely the location and orientation of said road marking in a coordinate reference system. These makes them suitable to be used as individual GCP. Furthermore, any other road painting, stark changes in pavement type, monument bases, sharp curb edges, metal caps for man-hole covers and sewer grates, unique low- lying geological features could be used as photo identifiable objects. The centre pixel of the group of pixels in the oithorectified image of the stationary earth surface features could be used to define the position of the feature in the coordinate reference system. Also a visual characteristic point of the group of pixels could be used. A visual characteristic point could be any position, i.e. pixel, having a predefined visual characteristic in the image, for example centre of feature, point of guidance arrow, point where tapered road edge lines touches each other, etc. In principle any pixel could be used to define the position of the feature. The group of pixels should comprise associated data, indicating which of the group of pixels is used to define the position of the feature to a position in the coordinate system.

In action 406, the areas of interest are analyzed whether they have sufficient size and unique character to be recognized in imagery. Those, that comply with these requirement are selected as stationary earth surface feature. For those in action 408 a 3D-orthorectified image is generated. A 3D-orthorectified image is an oithorectified image wherein at least one pixel has an associated height value, i.e. z coordinate, in the coordinate reference system. This means that the oithorectified image has a XYZ- position in the coordinate reference system. One pixel having an associated height value is sufficient for paint markings on the road, as these are on smooth surfaces and not likely to have z values that differ much. However, if the stationary earth surface feature has much differing z-values it is advantageous that the 3D-orthorectified image comprises an oithorectified image with metadata describing an associated height value for each pixel. This could be done by determining for each pixel and associated height value. If the stationary earth surface feature could be approximated by an inclined plane, a vector could be used to describe the orientation of the surface. In another embodiment, the metadata comprises associated height values for the corner pixels. By means of linear interpolation between the XYZ-coordinates of the corner pixels the height value for the other pixels could be determined. In yet another embodiment, the metadata described the contour of the image in XYZ-coordinates by means of a polyline. There are many other implementations of metadata possible to describe the surface of the 3D-orthorectified image in xyz coordinates of a coordinate reference system. Figure 8 shows some examples of stationary road surface features that could be used as GCP. It shows a stop line 81, "Warning of 'Give Way' just ahead" 82, guidance arrows 83, sewer grates 84, speed limits 85, pedestrian crossings 86, tapered road edge lines at exits 87, sharp curb edges 88, metal caps for man-hole covers 89 and any other direction indications 90. Other road surface features (not shown in figure 8) are hatched markings or chevron markings, reflection arrows, bifurcation arrows. Traffic Signs Manual 2003, Chapter 5, Road Markings, ISBN 0 11 552479 7, provides a complete overview of road markings that can be used as GCP.

Some stationary road surface features could exist in the same appearance in for example a squared area of 50 x 50 meters, for example guidance arrows, metal caps for man covers and sewer grates. In that case, a feature cannot be used as individual GCP to identify uniquely the corresponding feature in an image to be rectified. However, in combination with other nearby stationary road surface feature they can form a unique footprint to identify the position in the image to be rectified. It should be noted, that such a footprint will have both a unique position and orientation on the earth, which enables an orthorectification process to correct image errors by translation, scaling and rotation.

In action 408 for each selected stationary earth surface feature a 3D-orthorectified image is generated. The 3D-orthorectified image could be an orthorectified image in the form of a rectangle showing the stationary road surface feature and its surrounding background. In another embodiment, the pixels of the 3D-orthorectified image describes only the stationary earth surface feature. The 3D-orthorectified image could be generated by selecting the pixels representative of the stationary earth surface feature or GCO and determining the corresponding height information for each pixel and storing them as an image chip in a chip database. A chip database is a collection of image chips or subsets of images where each image is linked to a coordinate reference system. The image chip could have a reference to the original orthorectified image or tile. The image chip comprises a snapshot image of the location from the orthorectified image and metadata representative of the XY position in the coordinate reference system and elevation of height information. According to an embodiment of the invention, each pixel of an image chip is associated with an XYZ-position in the coordinate reference system. The size of an image chip depends on the size of the stationary road surface feature and the pixel size. Necessary pixel size depends on the resolution of the image to be rectified. From the mobile mapping data, image chips with a pixel size of 2cm could be generated. For rectifying aerial images, an image chip preferably comprises pixels which each representing an area of 7 by 7 cm, and having an absolute horizontal accuracy up to 0.5 - 2m 1 sigma and comprising associated elevation information having an absolute vertical 0.5 - 3m 1 sigma in a coordinate reference system .

In another embodiment, action 408 performs the actions: deriving 3D-polylines defining the position of edge lines of the surface of a stationary earth surface features in the coordinate reference system, generating an orthorectified image for the surface defined by the 3D-polylines and linking the 3D-polylines and orthorectified image to obtain a 3D-oithorectified image. In computer graphics, a polyline is a continuous line composed of one or more line segments. A polyline is specified by the endpoints of each line segment. This embodiment is very suitable for planar surfaces such as road segments, upper side of bridges. In that case, the surface can be approximated by the planar surface defined by the polylines. The elevation information, i.e. the Z- coordinate, can easily be derived by interpolation between the polylines. An advantage of the polylines is that they could be used as break lines when triangulating digital elevation models, i.e. generating the surface for a DEM represented by elevation points on a raster, forming a grid of squares.

In action 410 the 3D-orthorectified images are stored in a geodetic reference database, for example a chip database. This means that an image is stored along with associated positioning data comprising a description of coordinates in a coordinate reference system to determine for each pixel of the image their XYZ-position in a coordinate reference system. In an embodiment, the description is for each pixel an XYZ-position. In another embodiment, the data comprises a description describing the XY-position of the corners of the image and for each pixel a Z-coordinate. In yet another embodiment, the data comprises a description describing the XYZ-position of the edges of the image by means of polylines. In this embodiment, the XYZ-position of each pixel could be calculated by means of linear interpolation. A skilled person is able to find easily several alternative description, for example using a mixture of absolute and relative coordinates in the coordinate reference system. In action 412, the orthorectified images generated by action 402 or the 3D- orthorectified images generated by action 408, are combined to generate a continuous linear control network. In an embodiment, the continuous linear control network is an earth surface model like a DSM or DEM, wherein the model represents all earth surface captured by the digital cameras and laser sensors by means of orthorectified images, wherein each pixel has a XYZ-position in a coordinate reference system. The continuous linear control network is continuous as subsequent, neighboring or overlapping images of the network perfectly match with each other and form seamlessly a part of the earth surface, without discontinuities. The continuous linear control network is linear as the roads between junctions have a linear appearance and form a strip of the model of the earth surface. The continuous linear control network is geographic as each pixel of the orthorectified images forming the earth surface model is associated with a XYZ-coordinate in a coordinate reference system. Furthermore, the continuous linear control network is a network as the strips representative of the road surfaces form together a network, wherein nodes of the network are junctions and the branches of the network represent the road surface of the road segments between the junctions. The continuous linear control network establish a consistent countrywide 3D surface network with details which can be identified in aerial and satellite images.

In action 414, the continuous linear control network is stored in the geodetic reference database product.

The method according to the invention generates a geodetic reference database product from data that has been captured by means of a relatively inexpensive vehicle which could be provided with relative inexpensive digital cameras, laser sensors and position determining means. The method creates photo-identifiable data that can be used as ground control objects in rectification processes. The invention allows us for high volume collection of GCP' s which is orders of magnitude greater than traditional ground control production. The invention delivers a continuous road network surface of the road corridor in the form of image chips mosaic augmented with corresponding surface model, which could be in the form of break lines. The method has a consistent and verifiable accuracy profile in all geodetic dimensions. The method does not need that first special photo-identifiable earth surface marks are created in the field to be used to orthorectify future aerial imagery. Furthermore, the database product comprises substantially photo-identifiable material that will exist for many years. As the database product comprises 3D information by means of elevation information, it can be used to correct 3D surfaces as well.

Another advantage of the usage of MMS data is that in one mobile mapping session, the images data as well as the laser data records areas of the earth surface more than once when crossing a junction or traveling a road segment more than once. These areas could comprise a stationary road surface feature that can be used as a ground control object. In reality, the stationary road surface feature has the same location in the coordinate reference system. However, the position determining could have some absolute and relative inaccuracy within one mobile mapping session. The method according to the invention will select these stationary road surface features two or more times and corresponding XY position and elevation information Z-coordinate will be determined each time. For each determined stationary road surface feature a record could be made in the database comprising an image chip and metadata describing the XYZ position and optionally a reference to the original orthorectified image. By analyzing records related to the same earth surface feature, redundant information can be removed from the database. For example, by combining, i.e. averaging, or anomaly exclusion, the images and metadata of the same stationary road surface features, redundant information can be removed. By averaging the XY position and elevation information, meta data with averaged values for the XYZ coordinates could be calculated. Averaged values will in general more accurately define the position of the GCO in the coordinate reference system.

In figure 5, an overview is given of a computer arrangement 500 suitable for implementing the present invention. The computer arrangement 500 comprises a processor 511 for carrying out arithmetic operations. The processor 511 is connected to a plurality of memory components, including a hard disk 512, Read Only Memory (ROM) 513, Electrical Erasable Programmable Read Only Memory (EEPROM) 514, and Random Access Memory (RAM) 515. The memory components comprises a computer program comprising data, i.e. instructions arranged to allow the processor 511 to perform the method for generating a spatial-data-change message or the method for processing a spatial-data-change message according to the invention. Not all of these memory types need necessarily be provided. Moreover, these memory components need not be located physically close to the processor 511 but may be located remote from the processor 511. The input data and output data associated with the methods may or may not be stored as part of the computer arrangement 500. For example, the input data may be accessed via web services. It might even be possible, that an action is performed by a process running on another processor.

The processor 511 is also connected to means for inputting instructions, data etc. by a user, like a keyboard 516, and a mouse 517. Other input means, such as a touch screen, a track ball and/or a voice converter, known to persons skilled in the art may be provided too.

A reading unit 519 connected to the processor 511 may be provided. The reading unit 519 is arranged to read data from and possibly write data on a removable data carrier or removable storage medium, like a floppy disk 520 or a CDROM 521. Other removable data carriers may be tapes, DVD, CD-R, DVD-R, memory sticks, solid state memory (SD cards, USB sticks) compact flash cards, HD DVD, blue ray, etc. as is known to persons skilled in the art.

The processor 511 may be connected to a printer 523 for printing output data on paper, as well as to a display 518, for instance, a monitor or LCD (liquid Crystal Display) screen, head up display (projected to front window), or any other type of display known to persons skilled in the art.

The processor 511 may be connected to a loudspeaker 529 and/or to an optical reader 531, such as a digital camera/web cam or a scanner, arranged for scanning graphical and other documents.

Furthermore, the processor 511 may be connected to a communication network 527, for instance, the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), Wireless LAN (WLAN), GPRS, UMTS, the Internet etc. by means of I/O means 525. The processor 511 may be arranged to communicate with other communication arrangements through the network 527.

The data carrier 520, 521 may comprise a computer program product in the form of data and instructions arranged to provide the processor with the capacity to perform a method in accordance to the invention. However, such computer program product may, alternatively, be downloaded via the telecommunication network 527 into a memory component.

The processor 511 may be implemented as a stand alone system, or as a plurality of parallel operating processors each arranged to carry out subtasks of a larger computer program, or as one or more main processors with several sub-processors. Parts of the functionality of the invention may even be carried out by remote processors communicating with processor 511 through the telecommunication network 527.

The components contained in the computer system of Figure 5 are those typically found in general purpose computer systems, and are intended to represent a broad category of such computer components that are well known in the art.

Thus, the computer system of Figure 5 can be a personal computer, a workstation, a minicomputer, a mainframe computer, etc. The computer can also include different bus configurations, networked platforms, multi-processor platforms, etc. Various operating systems can be used including UNIX, Solaris, Linux, Windows, Macintosh OS, and other suitable operating systems.

The foregoing detailed description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The described embodiments were chosen in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims appended hereto.

1. Method of generating a geodetic reference database product, the method comprising:

- acquiring mobile mapping data captured by means of digital cameras, range sensors and position/orientation determination means mounted to a vehicle driving across the earth surface, the mobile mapping data comprising simultaneously captured image data, range data and associated position/orientation data in a geographic coordinate system;

- determining stationary earth surface features from the mobile mapping data by combining the image data, range data and associated position/orientation data;

- generating 3D-orthorectified images for the stationary earth surface features, wherein at least one pixel of the 3D-orthorectified image has a corresponding xyz coordinate in the coordinate reference system;

- storing the 3D-orthorectified images in a database to obtain the geodetic reference database product.

2. Method according to claim 1, wherein a stationary earth surface features is physical and visual feature in the earth's surface selected from a group comprising at least one of: road paintings, stark changes in pavement type, monument bases, sharp curb edges, metal caps for man-hole covers and sewer grates, unique low-lying geological features.

3. Method according to claim 1 or 2, wherein the stationary earth surface features are linear features.

4. Method according to claim 3, wherein the stationary earth surface features are road segments.

5. Method according to claim 4, wherein the method further comprises:

- combining the 3D-orthorectified images to obtain a continuous linear control network, wherein branches of the network correspond to road segments; and - storing the continuous linear control network in the geodetic reference database product.

6. Method according to any of the claims 1 - 5, wherein the determining stationary earth surface features comprises:

- generating orthorectified images from the mobile mapping data by combining the image data, range data and associated position data;

- processing the orthorectified images to determine areas of interest having high contrast in the orthorectified images; and

- selecting area of interest having predefined characteristics to obtain the stationary earth surface features.

7. Method according to any of the claims 1 - 6, wherein generating 3D- orthorectified images for the stationary earth surface features, comprises:

- deriving 3D-polylines defining the position of edge lines of the surface of a stationary earth surface features in the coordinate reference system;

- generating an orthorectified image for the surface defined by the 3D-polylines;

- linking the 3D-polylines and orthorectified image to obtain a 3D-orthorectified image.

8. A computer implemented system for generating a geodetic reference product, the system comprising a processor (511) and memory (512; 513; 514; 515) connected to the processor, the memory comprising a computer program comprising data and instructions arranged to allow said processor (511) to perform any of the' methods according to claims 1 - 7.

9. A geodetic reference database product comprising 3D-orthorectified images that have been produced by any of the methods according to claims 1 - 7, wherein each 3D- orthorectified image represents a stationary earth surface feature and at least one pixel of the 3D-orthorectified image has a corresponding xyz coordinate in the coordinate reference system.