<div dir="ltr">.img files are always reused and never rewritten - it's only about the gmapi files (a gmapsupp.img has to be generated in one go anyhow - so it is not in question here). So I do not see the sense of making it more complicated? It would be fine with me too that way - but I think it's simpler to just assume all .img files already have the converted files - because you could do it when generating those .img files. The problem with your approach is - that when you have a long list of tiles that were crashing due to too little maxnodes - then regenerating it - will make it very complicated. It's much easier to assume that all .img files are already converted. All other input not. If you need to convert .img files too - then you just use the normal --gmapi option instead.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 13 Sept 2021 at 18:58, Thomas Morgenstern <<a href="mailto:webmaster@img2ms.de">webmaster@img2ms.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<font face="Arial">I suggest the ne<font face="Arial">w</font>
option should be named <i>reuse=<c<font face="Arial">om</font>ma
separated list of files, w<font face="Arial">ildcar<font face="Arial">ds e<font face="Arial">nabled>. </font></font></font></i><font face="Arial"><font face="Arial"><font face="Arial">exampel <i>reuse=400<font face="Arial">00001</font>.img, <font face="Arial">40005*.img.
</font></i><font face="Arial">If this option is <font face="Arial">used, then mkgmap should not <font face="Arial">over</font>write the listed folders in
the <name>gma<font face="Arial">p</font>.
Naturally mkgmap should write<font face="Arial"> a new
tdb, preview.img, mdr and idx. <font face="Arial">I</font>n
most cases the <i>reuse folders </i><font face="Arial">contains Contourlines or other static
content.<br>
<br>
<font face="Arial">Thomas</font><br>
</font></font></font></font></font></font></font></font><br>
<div>Am 13.09.2021 um 15:38 schrieb Felix
Hartmann:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Gerd, yes exactly. I have a large collection of
.img files of contourlines. When I realized that I trash my NVME
disk very fast if I continue the way I did I got into thinking
and analysing what happens. It was clear that it all comes down
to the --gmapi option. I had believed that if I run several
passes with .o5m and .img files as input - only the .o5m input
is converted into new .gmapi folders - while the existing ones
are not overwritten. Then I checked the created date/last
modified date (because windows has in my eyes a bug that if you
replace a file with a near identical file - it just shows a new
modified date - but keeps the original created date - even
though it was a full overwrite).
<div><br>
</div>
<div>That got me thinking that in order to save writes - I have
to find a way to not only not recalculate the .img files - but
also create a static set of .gmapi folders. Those I just
mklink into the directory name that will be used on the next
run of mkgmap.jar - I managed to do this by uncommenting this
one line. Because I noticed that the symlinked (by mklink)
files are not rewritten I changed my scripts to move them away
and symlink them back. Then at the end delete all symlinks -
and move the files back (or to the location that I will use
for compressing). This step is a bit stupid if mkgmap could
just have a --gmapi-minimal mode in which only those files are
converted - that are also written out as .img files (if given
--tdb-file option). </div>
<div><br>
</div>
<div>I know that many people keep a static set of contourlines
.img files (also containing DEM). You just add the
show-profiles=1 option in case you include contourlines - and
leave it out if not. But actually it does not matter if the
contourlines files contain DEM or not.</div>
<div><b>I think the easiest way is the principle -
--gmapi-minimal only converts those files, it would write
out as .img files if --tdb-files option were given (or is
given). --gmapi on the other hand should convert the all
input files to .gmapi format.</b> Then mgkmap does not even
need to test if those files are there or not. This not only
saves a lot of writes - but also a lot of compile time.</div>
<div><br>
</div>
<div>Because essentially if you only provide .img input files
(including of course the ovm-img for the overview map if you
want) you only create a new set of mapset files. The exception
to this is the toolchain in which when a tile failed to
compile - you resplit the input files - and parse them again
with the same arguments so you have input of new o5m files and
old .img files. But the principle stays the same. If on the
rerun of the failed tiles now newy split with higher map-id
you give --gmapi-minimal and tdb-file - only those new tiles
are written - while the old .img and old ovm.img are supplied
to create the correct mapset files. With this procedure you
don't even need to put a symlink for the contourlines into the
gmap folder. As the input data to create the contourlines
rarely change - you can offer the contourlines as a separate
download. </div>
<div>Makes it much easier and faster.</div>
<div><br>
</div>
<div>On the map compilation server you then do not even need to
have a copy of the .gmapi contourlines files. You just need
the new input data and the static contourlines .img files.
Thomas Morgenstern for example had also not realized that he
is writing the contourlines .gmapi files each time without any
need. I think many/most providers of garmin maps who offer
many regions/worldwide coverage put the contourlines separate.
It's just a huge amount of compile time if you merge the
contourlines in o5m format with the map data in o5m data each
time before running mkgmap. The only actual advantage of this
being that the contourlines do not overlap roads (as they are
supplied as transparent .img files in another layer) - AND
that when people select maps with a single click instead of
drag in MapInstall/Mapsource they get all maps they think they
get (though there is a different shading - each layer
increases the darkness of the shading for selected). And of
course that the very old Mapsource still has problems
correctly showing the contourlines if they are in a separate
layer. However nearly everyone moved on to Basecamp which is
fully compatible with layered maps. The advantage of
contourlines as separate layer is for the user he can switch
them on / off independant of the maps and does not need to
download and transfer them each time. So I think for most the
advantages heavily outweigh the disadvantages - hence
contourlines into a separate transparent layer. Same can be
done of course for other things like buildings or vegetation -
though the advantage here is much less on the compile side
(while worldwide 10m coontourlines are 200GB of data - the
same extracts are only 15GB in data of buildings - if
buildings are shown only at resolution 24)</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, 13 Sept 2021 at 14:24,
Gerd Petermann <<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi
Felix,<br>
<br>
I have no clue how exactly your scripts work, how you manage
to reuse *.img files and so on. Also, I want to find a
solution that works for all users, not just you.<br>
<br>
So, I expect that you have one step that compiles frequently
changing OSM data and other steps which are used to compile
static data like contourlines or DEM. I don't know if you do
the latter for each country / continent or once for planet and
I don't care as long as it works for you.<br>
My understanding is that you have a large collection of *.img
files at some stage and that you run mkgmap multiple times
with different combinations of those *.img files as input to
produce different zip-files with gmapi format or gmapsupp
format.<br>
I think that's the normal way to do it, so I try to support
that way.<br>
<br>
Gerd<br>
<br>
________________________________________<br>
Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>
im Auftrag von Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><br>
Gesendet: Montag, 13. September 2021 12:28<br>
An: Development list for mkgmap<br>
Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and
converting to disk if used with --gmapi option<br>
<br>
I move away the info.xml - and use a different name for the
mapset files and then just have mkgmap any file that is input
in osm.pbf / osm / o5m format. Gmapi-minimal should not
convert any file that is supplied as .img - as it can be
assumed that those exist already (if they do not - then create
them with --gmapi). That is in my opinion the best approach.
So mkgmap does not even try to convert them.<br>
<br>
Afterwards I distribute a gmapi folder that includes all the
data - and by replacing the info.xml you can enable or disable
contourlines. For big countries the contourlines would be a
separate download anyhow - so the user only needs to download
the maps (including mapset files) but not redownload the
contourlines. Same principle as in offering the contourlines
as a separate gmapsupp.img file. so you have maps.img
contourlines10m.img contourlines20m.img buildings.img ....<br>
<br>
<br>
On Mon, 13 Sept 2021 at 13:17, Gerd Petermann <<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>>>
wrote:<br>
Hi Felix,<br>
<br>
sorry, the Data Deduplication as implemented in Windows Server
would not help here. It works after data was written.<br>
<br>
And yes, files which are not just copies of the *.img are
written as before. My understanding is that you have different
product directories in the gmapi folder and that you write
protect those files which should be kept.<br>
If you have a script to zip the gmapi directory you have to
exclude the unwanted folders.<br>
Didn't try it. Does it make sense?<br>
<br>
Gerd<br>
<br>
________________________________________<br>
Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>>
im Auftrag von Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>><br>
Gesendet: Montag, 13. September 2021 12:09<br>
An: Development list for mkgmap<br>
Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and
converting to disk if used with --gmapi option<br>
<br>
Oh - but data was certainly written - A rename will not show
as data written in both Task manager on Windows, as well as in
the smart data (I'm using Windows 10 pro not Windows server
however - maybe that functionality is limited to windows
server?)<br>
<br>
On Mon, 13 Sept 2021 at 13:06, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>>
wrote:<br>
Not sure if it makes it possible to use read only attribute
instead of moving and mklink. Maybe yes - because that was not
possible before. So it then would be set read only - instead
of of move & mklink - and at the end remove read only
instead of moving back.<br>
<br>
On Mon, 13 Sept 2021 at 12:59, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>>
wrote:<br>
Thanks Gerd,<br>
<br>
but that is just removing the warning if it cannot overwrite,
correct?<br>
If it can overwrite the gmap file it will still overwrite it
as I see.. So in essence just a bit more intelligent then my
disabling that line.<br>
<br>
I think it should not overwrite at all if present and
--x-gmapi-minimal (then you don't have to move away the files
and link them back to the original folder).<br>
<br>
On Mon, 13 Sept 2021 at 10:43, Gerd Petermann <<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>>>>
wrote:<br>
Hi all,<br>
<br>
attached is a quick patch that implements experimaltal option
--x-gmapi-minimal.<br>
<br>
If used instead of --gmapi mkgmap will not fail if a
write-protected output file exists in the gmapi output folder
and mkgmap copies data from *.img. It should still crash when
other files like Info.xml are written.<br>
<br>
BTW: no conversion is done when those files are written. I
think mkgmap simply copies data from sub files in *.img into
single files. So, the same sequence of Bytes exists two or
more times on the Computer. Windows Server seems to support
automatic data deduplication (1). I am sure Linux offers
similar support. No idea how effective or reliable it is, but
it might be worth trying.<br>
<br>
Gerd<br>
(1)<br>
<a href="https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/install-enable" rel="noreferrer" target="_blank">https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/install-enable</a><br>
<br>
________________________________________<br>
Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>>>
im Auftrag von Carlos Dávila <<a href="mailto:carlos@alternativaslibres.org" target="_blank">carlos@alternativaslibres.org</a><mailto:<a href="mailto:carlos@alternativaslibres.org" target="_blank">carlos@alternativaslibres.org</a>><mailto:<a href="mailto:carlos@alternativaslibres.org" target="_blank">carlos@alternativaslibres.org</a><mailto:<a href="mailto:carlos@alternativaslibres.org" target="_blank">carlos@alternativaslibres.org</a>>>><br>
Gesendet: Sonntag, 12. September 2021 22:45<br>
An: <a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><br>
Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and
converting to disk if used with --gmapi option<br>
<br>
I think it's a good idea if mkgmap checks the required files
are present.<br>
<br>
El 12/9/21 a las 21:16, Gerd Petermann escribió:<br>
> Hi Felix,<br>
><br>
> so you just want a new option so that mkgmap doesn't fail
if it cannot overwrite (write-protected) files in the output
directory, right? No need to verify the content of the exiting
file(s)?<br>
><br>
> Gerd<br>
><br>
> ________________________________________<br>
> Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>>>
im Auftrag von Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>><br>
> Gesendet: Sonntag, 12. September 2021 19:10<br>
> An: Development list for mkgmap<br>
> Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing
and converting to disk if used with --gmapi option<br>
><br>
> And well - it is burning SSD for the contourlines
currently even if not calling multiple times. 130GB minimum
per worldwide map update for all geofabrik extracts at 20m
equidistance. 260GB at 10m. With calling multiple times and
10m and 20m I had about 2TB of useless writes per weekly map
update. I've got rid of all of them with my uncommenting the
line, plus saved about 500GB of writes by now doing all the
gmapsupp.img and gmap stuff in Ramdisk So now instead of 4TB
of disk writes per map update I'm down to 1TB.. (and yeah the
resplitting of tiles added another 5TB of writes - but that
could have been fixed easily by changing order of commands
too).<br>
><br>
> On Sun, 12 Sept 2021 at 20:04, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>>>
wrote:<br>
> because you need to do it if you want to show
contourlines. Now compiling the worldwide contourlines each
time again - would burn SSD as well - besides taking longer
than just compiling the maps. So everyone who offers maps for
many countries as download puts the contourlines separate. If
you want to offer the choice of showing contourlines or not -
mkmgap will write the gmap contourlines once not needed, and
once the maps not needed. If you don't want to offer that
option - it's only the contourlines being written without
need. Contourlines for all geofabrik extracts 20m equidistance
are about 130GB, 10m would be 260GB.<br>
> If you offer 10m and 20m it will be even more not needed
writes.<br>
><br>
> The only solution is to go for a Ramdisk instead -
However you likely will need 128GB of RAM to do that - because
for Asia or Europe you need a 64GB Ramdisk. Same for North
America extract (but few people offer that and just have
Canada and the US divided into the 4-5 zones). For other maps
you will get by with a 15-20GB Ramdisk (which I have resorted
to now for all but the windows installers because I don't want
to let the server wait for ages for the single thread NSIS
wrapping up the data and instead start the next country in
parallel). And yes going RAMDISK already using my patch and
symlinking those files so mkgmap cannot overwrite them (would
still be faster if mkgmap didn't try in first place I think).
If you want to write out the contourlines as well besides the
additional time - for Asia continent you may then go for a
90GB Ramdisk minimum if offering windows and gmap installers.
In this case gmapsupp.img donwnloads aren't possible anyhow
due to the huge data amounts. For sm<br>
> aller countries I have them too....<br>
><br>
> I coded around this now by using symlinks - but yeah that
will be quite a lot of work and is prone to break - while I
guess it's only 10 lines or so to add to mkgmap code to have a
mode that does not write them out if they are present - or if
you tell on commandline they are present already. For .img
files they aren't overwritten either...<br>
><br>
> On Sun, 12 Sept 2021 at 18:40, Gerd Petermann <<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>>><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>>>>>
wrote:<br>
> Hi Felix,<br>
><br>
> did not read all the posts in detail. I understand that
mkgmap is neither burning SSDs nor doing any excessive writing
unless you call it multiple times for the same input files. So
the question is: Why do you do that?<br>
><br>
> Gerd<br>
><br>
> ________________________________________<br>
> Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>>>>
im Auftrag von Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>>><br>
> Gesendet: Freitag, 10. September 2021 00:02<br>
> An: Development list for mkgmap<br>
> Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing
and converting to disk if used with --gmapi option<br>
><br>
> Well I think the .tmp files are just building up - and
the renamed. So they are not causing any actual excessive
write.<br>
> As for the gmap - it would be cool if there is a mode to
not write them.<br>
><br>
> Actually it would be great if mkgmap could write all in
one go. Because the thing that takes so much time - is the
address search - and that is always the same. The differences
are tiny (just because MapInstall is crashing when files are
missing) you need to compile them separately.<br>
><br>
> but maybe there could be a mode where mkgamp writes all
in one go.<br>
> So family-name / family-name1 / family-name2<br>
> description / description1 / description2<br>
> input input1 input2<br>
> family-name..<br>
> show-profiles<br>
> overview-mapname<br>
> product-id<br>
> (and maybe I missed some options are those that would
need to be given for each set of input tiles. And then just an
option where you tell mkgmap files starting with which first 4
numbers are relevant for address search. No need to analyze if
those other supplied .img (e.g. buildings or contourlines)
need to be added to address search.). I know coded around the
problem of the gmap files causing excessive writes. But yeah
that is actually really complicated be it on windows or
linux...<br>
><br>
> On Wed, 8 Sept 2021 at 17:07, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>>>>
wrote:<br>
> Well I could give it 20 GB ram disk, maybe 32 but then I
need to render on less than 12 processes 64GB ram available).
But that is not enough for Asia continent map, and I guess
super tight for Europe...<br>
><br>
> Mkgmap could definitely keep those .tmp files in memory.
But the important bit is the gmap files not needeed to be
written.... Also would save quite some CPU time.<br>
><br>
> On Wed, 8 Sep 2021, 14:31 ael <<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>>><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>>>><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>>><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a><mailto:<a href="mailto:witwall3@disroot.org" target="_blank">witwall3@disroot.org</a>>>>>>
wrote:<br>
> On Wed, Sep 08, 2021 at 02:10:52PM +0300, Felix Hartmann
wrote:<br>
>> Yes on an nvme disk you barely notice the conversion
- it's really quick.<br>
>> BUT it is not needed if you have the files and even
more - it burns your<br>
>> NVME SSD disk.<br>
> +1. I always use an old spinning rust disk when using
mkgmap to save ssd<br>
> write cycles, even without contours and such. It seems to
be profligate<br>
> in its use of disk cycles. I did try using RAM disk, but
even with 16GB<br>
> on a laptop, that was soon exhausted.<br>
><br>
> ael<br>
><br>
><br>
> _______________________________________________<br>
> mkgmap-dev mailing list<br>
> <a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>>>><br>
> <a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
><br>
><br>
> --<br>
> Felix Hartman - Openmtbmap.org & VeloMap.org<br>
><br>
> _______________________________________________<br>
> mkgmap-dev mailing list<br>
> <a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>>><br>
> <a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
><br>
><br>
> --<br>
> Felix Hartman - Openmtbmap.org & VeloMap.org<br>
><br>
><br>
><br>
> --<br>
> Felix Hartman - Openmtbmap.org & VeloMap.org<br>
><br>
> _______________________________________________<br>
> mkgmap-dev mailing list<br>
> <a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><br>
> <a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
<br>
<br>
_______________________________________________<br>
mkgmap-dev mailing list<br>
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><br>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
_______________________________________________<br>
mkgmap-dev mailing list<br>
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><br>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
_______________________________________________<br>
mkgmap-dev mailing list<br>
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><br>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
_______________________________________________<br>
mkgmap-dev mailing list<br>
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><br>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Felix Hartman - Openmtbmap.org & VeloMap.org<br>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
mkgmap-dev mailing list
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a></pre>
</blockquote>
<br>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div>