[mkgmap-dev] Usage of DEM files within zip

Thu Jun 11 09:57:48 BST 2020

Hi

I finished some testing on the DEM files in zip not found by mkgmap.

It has something to do with the zip-tool used.

When internal Windows 10 zip is used through explorer or vbscript, you can create ZIP’s of 17GB but only zips up to 2Gb can be read  “by mkgmap”.

Even when removing files from such big zip until it drops under the 2Gb does not help.

As soon as it is re-created from scratch and smaller then 2Gb its oke again.

Also tested with winzip 11 (pretty ancient version which I had licence for) and the usage of 7zip, and both work fine above 2Gb.

Winzip was tested up to the 2,3Gb and 7zip up to 17Gb

Pretty useless info, but then you know 😊

Hgt Files
Tool
Result
Remark
Zipped size
2
Windows 10
ok
2
7-zip 18.05 extension .7z
not found
2
7-zip 18.05 extension .zip
ok
2
VBA Script Windows 10
ok
1154
VBA Script Windows 10
not found
From scratch
2,24 GB (2.407.959.231 bytes)
1154
Windows 10
not found
From scratch
2,24 GB (2.407.959.231 bytes)
1094
Windows 10
not found
Internal delete
1,93 GB (2.078.032.075 bytes)
1079
Windows 10
not found
Internal delete
1,90 GB (2.047.599.717 bytes)
1063
Windows 10
not found
Internal delete
1,87 GB (2.013.685.658 bytes)
1059
Windows 10
not found
Internal delete
1,86 GB (2.007.054.183 bytes)
1052
Windows 10
not found
Internal delete
1,84 GB (1.978.252.178 bytes)
1047
Windows 10
not found
Internal delete
1,82 GB (1.960.539.426 bytes)
1019
Windows 10
not found
Internal delete
1,73 GB (1.862.120.309 bytes)
1000
Windows 10
not found
Internal delete
1,69 GB (1.819.411.992 bytes)
1000
Windows 10
ok
Above unzipped then from scratch
1,76 GB (1.894.852.116 bytes)
1154
Winrar
Warning before start > too big for zip
1154
Winzip 11 2006
ok

2,26 GB (2.436.489.540 bytes)

-----Oorspronkelijk bericht-----
Van: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> Namens Joris Bo
Verzonden: maandag 8 juni 2020 11:21
Aan: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
Onderwerp: Re: [mkgmap-dev] Usage of DEM files within zip

Thanks, i'll do some more testing.

It was indeed a single zip with 17Gb hgt.

Strange thing is that it for sure works with a 17Gigabyte single zip containing 26.000 mixed 1arc/3arc hgt files. (Windows 10) It still finds all files from New Sealand, to Zambia and Europe as long as it contains a subfolder in the zip.

But don't bother, I did not take performance into account. And my tests where not that scientifically correct. Because of the huge sizes... I only tried only until it worked again.

Kind regards

Joris

-----Oorspronkelijk bericht-----

Van: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> Namens Gerd Petermann

Verzonden: maandag 8 juni 2020 07:57

Aan: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>

Onderwerp: Re: [mkgmap-dev] Usage of DEM files within zip

Hi Joris,

I cannot reproduce the problem. I've created a srtm3.zip containing two files N52W003.hgt and N53W003.hgt and --dem=c:\temp\srtm3.zip works fine.

My assumption: Your world-hgt.zip file really contained all hgt files and got larger than maybe 2G and the zipfile class in java cannot process it.

mkgmap tries to support the file formats which are provided by the various DEM providers. I think I once tested performance with a large zip containing many hgt files and it was worse compared to single zipped files.

IIRC the better alternative was to compress the folder with the hgt files (not using zip but the OS methods) or to have each hgt file zipped.

Gerd

________________________________________

Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Joris Bo <jorisbo at hotmail.com<mailto:jorisbo at hotmail.com>>

Gesendet: Sonntag, 7. Juni 2020 20:45

An: Development list for mkgmap

Betreff: [mkgmap-dev] Usage of DEM files within zip

Hi

I struggled today with the DEM/hgt files in a zip. After downloading some extra and checking/rezipping finally my maps did not work anymore.

I do not have the exact error anymore but it was clear: DEM HGT files not found

Now it looks like if the ZIP does not have a folder but only the HGT files on the root of the zip then mkgmap does not recognize them If I rezip everything again and put the files within a subfolder first, it is oke.

For me its fine, I solved 'my' issue, but if this is true it could better be in the manual or solved in code.

Looks like this does not work:

world-hgt.zip\N00E006.hgt

world-hgt.zip\N00E007.hgt

world-hgt.zip\N00E008.hgt

And if I do it like this its oke

world-hgt.zip\world-hgt\N00E006.hgt

world-hgt.zip\world-hgt\N00E007.hgt

world-hgt.zip\world-hgt\N00E008.hgt

Kind regards,

Joris Bo

_______________________________________________

mkgmap-dev mailing list

mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>

http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________

mkgmap-dev mailing list

mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>

http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20200611/34935f65/attachment-0001.html>