logo separator

[mkgmap-dev] Errors when using --gmapi-minimal

From Gerd Petermann gpetermann_muenchen at hotmail.com on Fri May 27 08:33:37 BST 2022

Hi Dave,

I agree that the --gmapi-minimal option is quite special. The code
to implement the optional pattern stores the pattern in a variable
called mustWritePattern. Then, mkgmap first determines that a file
should be written because it is in the list of input files,
 then - because of the gmapi-minimal option - it detects that
the output file already exists and would skip it but the mustWritePattern
says that it should still be overwritten. Hope this makes it clearer?

Gerd




________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von dfjkman at gmail.com <dfjkman at gmail.com>
Gesendet: Freitag, 27. Mai 2022 07:55
An: 'Development list for mkgmap'
Betreff: Re: [mkgmap-dev] Errors when using --gmapi-minimal

Hi Gerd,

In my situation the problem is solved, your understanding of my case is
correct. I always knew that if I removed the overview files from the input
folder I would have no problem, your suggestion of a better mask means I do
not have to remove these files, thanks for that. I was just thinking that
if, for example, you had an input folder containing contour files for a
large area and only wanted to create a map of a much smaller area the
include pattern was a way to do this. This in fact works but does mean that
the files that are in the input folder but are skipped in the process are
still referenced in the tdb. I suspect, but am no expert obviously, the
content of the tdb is governed by the initial input instruction and mask,
then later the --gmapi-minimal pattern skips some files that have been
already referenced in the tdb. While this may be something that is never
used it is something that can be done and the resultant missing references
are not fatal and do not cause BaseCamp to shut down. References to missing
overview files however are fatal as I have discovered.

Thanks for all your work on mkgmap, the maps I produce using it help me to
hunt down the many mapping errors made by remote mappers and the many
inexperienced mappers mapping in Zambia.

Kind regards,
Dave

-----Original Message-----
From: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> On Behalf Of Gerd
Petermann
Sent: 27 May 2022 07:19
To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
Subject: Re: [mkgmap-dev] Errors when using --gmapi-minimal

Hi Dave,

does that mean the problem is completely solved now?
My current understanding is that you created the gmapi folder, later created
gmapsupp and overmap files in the folder with the DEM tiles and that those
additional files were selected as input files with the *.img mask.

The pattern for the --gmapi-minimal option really just forces a copying of
files, it has no influence on the tdb content.

Gerd



________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
dfjkman at gmail.com <dfjkman at gmail.com>
Gesendet: Donnerstag, 26. Mai 2022 23:24
An: 'Development list for mkgmap'
Betreff: Re: [mkgmap-dev] Errors when using --gmapi-minimal

Hi Gerd,

Having a more specific mask as you suggest solves the problem of any
overview map or gmapsupp being included as input, but it does not get round
the fact that any files excluded by --gmapi-minimal, with include-pattern
specified, are still included in the TDB file but not present in the map. It
does mean that BaseCamp does not crash as any missing files are not overview
ones. It does not really affect me as I will not be filtering my input files
it is just that I noticed this behaviour when using the include-pattern with
--gmapi-minimal, using the improved mask as you suggested will exclude any
overview map anyway in my situation, I have just checked.

When I used --gmapi to combine  the maps I did not have the overview or
gmapsupp files in my input folders, only the files I required, so did not
come across the problem. It was only after I had recreated the DEM and
contour maps that I came across the problem as new overview maps and
gmapsupp had been created  and I had not removed them. I then had a look at
using the include-pattern with --gmapi-minimal to prevent the files being
used as input and so started to play around with various patterns to see the
effects. I just thought I would let you know my findings. Perhaps I
misunderstood the purpose of the include-pattern.

Dave



-----Original Message-----
From: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> On Behalf Of Gerd
Petermann
Sent: 26 May 2022 20:35
To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
Subject: Re: [mkgmap-dev] Errors when using --gmapi-minimal

Hi Dave,

so I think your problem is not the --gmapi-minimal option. You probably get
the same problems with --gmapi.
Make sure that you don't include files which are not wanted in the target
directory.
So, instead of C:\Users\Dave\Documents\Maps\Data\DemImg\*.img use a more
specifc mask like C:\Users\Dave\Documents\Maps\Data\DemImg\3641*.img so that
only the wanted tiles and not the overview map or a gmapsupp is included.

Once this works well with the --gmapi option you can use the --gmapi-minimal
option to avoid the redundant copying of those tiles.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
dfjkman at gmail.com <dfjkman at gmail.com>
Gesendet: Donnerstag, 26. Mai 2022 19:05
An: 'Development list for mkgmap'
Betreff: Re: [mkgmap-dev] Errors when using --gmapi-minimal

Hi Gerd,

I have previously created  a separate DEM map containing only DEM data and
all .img files are stored in a folder called '
C:\Users\Dave\Documents\Maps\Data\DemImg\' as well as a separate map
containing only contours with all .img files stored in a folder called '
C:\Users\Dave\Documents\Maps\Data\ContImg\', both are transparent and I had
used the individual renamed gmapsupp.img maps on my Etrex to be able to show
DEM or contour maps together with my normal map of Zambia by enabling or
disabling them on the device.

I tried my normal map building procedure using the  --gmapi-minimal option
to combine the existing DEM and Contour maps with my standard map for use in
BaseCamp at first just as an experiment and did not use any filters and had
no problems as the DemImg and ContImg folders only had the required .img
files, no overview map. I then Recreated the DEM and contour maps changing
the DEM distances and contour intervals and then recreated my combined map
which is when I noticed the error message. On investigating I realized the 2
folders now contained the overview maps, '
C:\Users\Dave\Documents\Maps\Data\DemImg\Zambia_Dem.img' and
C:\Users\Dave\Documents\Maps\Data\ContImg\Zambia_Contours.img'. Thinking it
was because I had not filtered the --gmapi-minimal I used the following
filter, .*[0-9]{8}\.img, to select only .img files with names made up of
numbers so it would ignore 'Zambia_Contours.img' and 'Zambia_Dem.img'. This
it did but there was still the error message from mkgmap and JaVaWa GMTK
reported this error ' TDB error:        One or more map segments in the TDB
file are missing from disk.' and listed the following:
63240000: Overview Map (Zambia_C) (missing)
63240000: Overview Map (Zambia_D) (missing)

As a trial I then created a filter, ' .*[0-8]{8}\.img ' to skip .img files
containing a 9, so this filter would skip 36410009.img, 36410019.img and so
on. Mkgmap reports it is skipping these files as expected but once again
JaVaWa GMTK reports:
Name on computer:       Zam_OSM_Dem_26-05-2022
Name on device: Zam_OSM_Dem_26-05-2022
Family ID (FID):        2636
Product ID (PID):       1
Version:        1.00
Size:   428.62 MiB
Location:       C:\ProgramData\Garmin\Maps\Zam_OSM_Dem_26-05-2022.gmap\
TYP file:
C:\ProgramData\Garmin\Maps\Zam_OSM_Dem_26-05-2022.gmap\dfj.typ
Map type:       .gmap map (global profile)
Protected:      No

TDB error:      One or more map segments in the TDB file are missing from
disk.
Errors recoverable:     Maybe

Map segments (94):      26350001: ZM-Copperbelt Province (26350001)
        26350002: ZM-Western Province (26350002)
        26350003: ZM-Chibombo District (26350003)
        26350004: ZM-Southern Province (26350004)
        26350005: ZM-Mazabuka (26350005)
        26350006: ZM-Mongu (26350006)
        26350007: ZM-Livingstone (26350007)
        26350008: ZM-Northern Province (26350008)
        26350009: ZM-Kitwe (26350009)
        26350010: ZM-Mpika (26350010)
        26350011: ZM-Muchinga Province (26350011)
        26350012: ZM-Mkushi (26350012)
        26350013: ZM-Eastern Province (26350013)
        26350014: ZM-Central Province (26350014)
        26350015: ZM-Lusaka Province (26350015)
        26350016: ZM-Republic of Zambia (26350016)
        36410001: Zambia_Dem (36410001)
        36410002: Zambia_Dem (36410002)
        36410003: Zambia_Dem (36410003)
        36410004: Zambia_Dem (36410004)
        36410005: Zambia_Dem (36410005)
        36410006: Zambia_Dem (36410006)
        36410007: Zambia_Dem (36410007)
        36410008: Zambia_Dem (36410008)
        36410009: Zambia_Dem (36410009) (missing)
        36410010: Zambia_Dem (36410010)
        36410011: Zambia_Dem (36410011)
        36410012: Zambia_Dem (36410012)
        36410013: Zambia_Dem (36410013)
        36410014: Zambia_Dem (36410014)
        36410015: Zambia_Dem (36410015)
        36410016: Zambia_Dem (36410016)
        36410017: Zambia_Dem (36410017)
        36410018: Zambia_Dem (36410018)
        36410019: Zambia_Dem (36410019) (missing)
        36410020: Zambia_Dem (36410020)
        36410021: Zambia_Dem (36410021)
        36410022: Zambia_Dem (36410022)
        36410023: Zambia_Dem (36410023)
        36410024: Zambia_Dem (36410024)
        36410025: Zambia_Dem (36410025)
        36410026: Zambia_Dem (36410026)
        36410027: Zambia_Dem (36410027)
        36410028: Zambia_Dem (36410028)
        36410029: Zambia_Dem (36410029) (missing)
        36410030: Zambia_Dem (36410030)
        36410031: Zambia_Dem (36410031)
        36410032: Zambia_Dem (36410032)
        36410033: Zambia_Dem (36410033)
        36410034: Zambia_Dem (36410034)
        36410035: Zambia_Dem (36410035)
        36410036: Zambia_Dem (36410036)
        36410037: Zambia_Dem (36410037)
        36410038: Zambia_Dem (36410038)
        36410039: Zambia_Dem (36410039) (missing)
        86410001: Zambia_Contours (86410001)
        86410002: Zambia_Contours (86410002)
        86410003: Zambia_Contours (86410003)
        86410004: Zambia_Contours (86410004)
        86410005: Zambia_Contours (86410005)
        86410006: Zambia_Contours (86410006)
        86410007: Zambia_Contours (86410007)
        86410008: Zambia_Contours (86410008)
        86410009: Zambia_Contours (86410009) (missing)
        86410010: Zambia_Contours (86410010)
        86410011: Zambia_Contours (86410011)
        86410012: Zambia_Contours (86410012)
        86410013: Zambia_Contours (86410013)
        86410014: Zambia_Contours (86410014)
        86410015: Zambia_Contours (86410015)
        86410016: Zambia_Contours (86410016)
        86410017: Zambia_Contours (86410017)
        86410018: Zambia_Contours (86410018)
        86410019: Zambia_Contours (86410019) (missing)
        86410020: Zambia_Contours (86410020)
        86410021: Zambia_Contours (86410021)
        86410022: Zambia_Contours (86410022)
        86410023: Zambia_Contours (86410023)
        86410024: Zambia_Contours (86410024)
        86410025: Zambia_Contours (86410025)
        86410026: Zambia_Contours (86410026)
        86410027: Zambia_Contours (86410027)
        86410028: Zambia_Contours (86410028)
        86410029: Zambia_Contours (86410029) (missing)
        86410030: Zambia_Contours (86410030)
        86410031: Zambia_Contours (86410031)
        86410032: Zambia_Contours (86410032)
        86410033: Zambia_Contours (86410033)
        86410034: Zambia_Contours (86410034)
        86410035: Zambia_Contours (86410035)
        86410036: Zambia_Contours (86410036)
        86410037: Zambia_Contours (86410037)
        86410038: Zambia_Contours (86410038)
        86410039: Zambia_Contours (86410039) (missing)

Other essential
C:\ProgramData\Garmin\Maps\Zam_OSM_Dem_26-05-2022.gmap\Product1\Zambia.tdb
files/folders:
C:\ProgramData\Garmin\Maps\Zam_OSM_Dem_26-05-2022.gmap\Product1\Zambia
        C:\ProgramData\Garmin\Maps\Zam_OSM_Dem_26-05-2022.gmap\Zambia.mdx
        C:\ProgramData\Garmin\Maps\Zam_OSM_Dem_26-05-2022.gmap\Zambia_mdr
        C:\ProgramData\Garmin\Maps\Zam_OSM_Dem_26-05-2022.gmap\info.xml.
JaVaWa can repair these errors by rewriting the TDB file to exclude the
missing files.

I had removed any overview files (Zambia_Contours.img and Zambia_Dem.img)
from the input folders, if they are left in they are skipped and mkgmap will
report:

gmapi-minimal: Skipping file
C:\Users\Dave\Documents\Maps\Data\DemImg\Zambia_Dem.img
gmapi-minimal: Skipping file
C:\Users\Dave\Documents\Maps\Data\ContImg\Zambia_Contours.img
WARNING (global): Could not copy 63240000.RGN
uk.me.parabola.imgfmt.FileExistsException: File 63240000.RGN already exists
WARNING (global): Could not copy 63240000.TRE
uk.me.parabola.imgfmt.FileExistsException: File 63240000.TRE already exists
WARNING (global): Could not copy 63240000.LBL
uk.me.parabola.imgfmt.FileExistsException: File 63240000.LBL already exists
SEVERE (global): Invalid mapname for
C:\Users\Dave\Documents\Maps\Data\DemImg\Zambia_Dem.img, perhaps it is not a
.img file SEVERE (global): Invalid mapname for
C:\Users\Dave\Documents\Maps\Data\DemImg\Zambia_Dem.img, perhaps it is not a
.img file

As you can see from the attached screengrab from BaseCamp of the trial map
it behaves as you would expect with the DEM and contour data missing in the
area of the skipped file, BaseCamp will load Ok even if the TDB file is not
corrected by JaVaWa but it will not load without correction if any of the
skipped files are overview files.

My normal instruction to create the combined map is as follows:

Java -jar -Xmx7g `
        C:\Users\Dave\Documents\Maps\Utils\mkgmap\mkgmap.jar `
        --style-file=C:\Users\Dave\Documents\Maps\Styles\dfj `
        -c C:\Users\Dave\Documents\Maps\Options\Zambia.cfg `
        --output-dir= "C:\Users\Dave\Documents\Maps\Temp\Output Map" `
        --family-id= 2635 `
        --family-name=" Zam_OSM_Dem_26-05-2022" `
        --series-name=" Zam_OSM_Dem_26-05-2022" `
        --gmapi-minimal='.*[0-9]{8}\.img' `
        -c C:\Users\Dave\Documents\Maps\Temp\Split_files\template.args `
<----- new map input
        C:\Users\Dave\Documents\Maps\Data\DemImg\*.img  `
<----- pre compiled DEM map input
        C:\Users\Dave\Documents\Maps\Data\ContImg\*.img `
<----- pre compiled contour map input
        -n 26350001 `
        --description=Zambia `
        C:\Users\Dave\Documents\Maps\Typ\dfj.txt `

The commands to create the initial DEM maps and contour maps are the same as
above except not using the --gmapi-minimal command and using
C:\Users\Dave\Documents\Maps\Data\DemImg as the output directory for the DEM
maps and C:\Users\Dave\Documents\Maps\Data\ContImg for the contour maps and
the input files obviously provide the elevation data and nothing else.

I hope this is clear.

Kind regards,
Dave


-----Original Message-----
From: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> On Behalf Of Gerd
Petermann
Sent: 26 May 2022 14:27
To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
Subject: Re: [mkgmap-dev] Errors when using --gmapi-minimal

Hi Dave,

I don't know if it's an error in mkgmap or maybe a wrong expectation.
Please tell us the complete command(s) to create the initial map and the 2.
command to uppdate it with --gmapi-minimal.
What do you expect to happen with the --gmapi-minimal option?

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
dfjkman at gmail.com <dfjkman at gmail.com>
Gesendet: Donnerstag, 26. Mai 2022 12:36
An: 'Development list for mkgmap'
Betreff: [mkgmap-dev] Errors when using --gmapi-minimal

Hi Gerd,

I decided to have a go using the --gmapi-minimal option and may have come
across a potential bug.

When using the include-pattern and some .img files are skipped because they
do not match the pattern, but exist in the input folder containing pre
written files, these skipped files are still written to the *.tdb file but
not included in the map.

In other words using the pattern example given
--gmapi-minimal=.*4711[0-9]{4}\.img, matching  files between 47110000.img
and 47119999.img, if there is also a file 47120000.img this will be skipped
but there will still be a reference to the skipped in the *.tdb file the
same with any overview files that may exist in the input folder.

I think having any overview maps in the input folder will generate the
following error messages:

gmapi-minimal: Skipping file
C:\Users\Dave\Documents\Maps\Data\DemImg\Zambia_Dem.img
gmapi-minimal: Skipping file
C:\Users\Dave\Documents\Maps\Data\ContImg\Zambia_Contours.img
WARNING (global): Could not copy 63240000.RGN
uk.me.parabola.imgfmt.FileExistsException: File 63240000.RGN already exists
WARNING (global): Could not copy 63240000.TRE
uk.me.parabola.imgfmt.FileExistsException: File 63240000.TRE already exists
WARNING (global): Could not copy 63240000.LBL
uk.me.parabola.imgfmt.FileExistsException: File 63240000.LBL already exists
SEVERE (global): Invalid mapname for
C:\Users\Dave\Documents\Maps\Data\DemImg\Zambia_Dem.img, perhaps it is not a
.img file SEVERE (global): Invalid mapname for
C:\Users\Dave\Documents\Maps\Data\DemImg\Zambia_Dem.img, perhaps it is not a
.img file I discovered the errors while using JaVaWa GMTK when checking the
maps for errors.

I am creating a map including pre written DEM and contour information to try
find badly mapped rivers that go uphill and over ridges in BaseCamp.

Kind regards,
Dave

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


More information about the mkgmap-dev mailing list