[mkgmap-dev] splitter questions

Thu Oct 22 11:43:34 BST 2020

Hi Karl

If you are developing these for a device that supports multiple .img
files, the option
  --add-boundary-nodes-at-admin-boundaries=NUM
allows you to build individual countries (or groups), have them all
loaded on the device, select the appropriate maps and have routing
between them.

Having larger tiles can cause 2 problems that I know of:
1/ mkgmap runs out of memory
2/ some tile .img data structures use a variable number of bytes
depending on the number of items and stepping over a threshold can
cause a jump in the final .img size and, if I remember correctly, a
more significant jump in size of some of the global indexes (MDR
structures) if just 1 tile needs more bytes.
mkgmap generates a warning when it needs to use 3 bytes to reference
Cities, Zip codes, Exit facilities or their linked highways.

Ticker

On Thu, 2020-10-22 at 09:41 +0000, Gerd Petermann wrote:
> Hi Karl,
> 
> do you want to distribute the maps for others or do you need them
> only for yourself?
> In the latter case you may use a poly file to cut out the area you
> are interested in. I do this with my maps (which I don't share with
> others)
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von 7770 <7770 at foskan.eu>
> Gesendet: Donnerstag, 22. Oktober 2020 11:36
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter questions
> 
> Hi Gerd.
> 
> I managed to run the splitter and it completed well with
> java -Xmx1800m  and setting the --max-nodes=1600000.
> 570 osm.pbf files were created.
> 
> For even larger maps
> --max-nodes= higher should use less memory, but will those tiles be
> slower on
> a GPS unit and how much larger value is reasonable?
> 
> 
> The main reason for creating such a large mapdata set is to make
> cross border
> routing to work.
> Is there some other way i could produce the maps?
> For example split and build map for each country separately (based on
> data
> from geofabrik) and then stitch the countries and routing data
> together, is
> this possible?
> 
> Regards
> Karl.
> 
> 
> On torsdag 22 oktober 2020 kl. 11:17:00 CEST Gerd Petermann wrote:
> > Hi Karl,
> > 
> > the main purpose of --keep-complete is to have complete
> > multipolygons in
> > each tile to avoid empty parts of large lakes or flooded areas near
> > large
> > lakes. There is also an effect on routing regarding turn
> > restrictions and
> > route relations.
> > 
> > Anyhow, if you really want to produce a very large map on such a
> > limited
> > system you probably run into other problems when calculating the
> > global
> > index in mkgmap.
> > 
> > Gerd
> > 
> > 
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von 7770
> > <7770 at foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:59
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] splitter questions
> > 
> > Hi Gerd.
> > Yes, it is 64-bit, but i don't have a lot of memory.
> > --keep-complete=false, can it impact routing possibilities (routing
> > may not
> > work well) or is it mainly affecting how the map will look?
> > 
> > 
> > Regards
> > Karl
> > 
> > On torsdag 22 oktober 2020 kl. 10:49:23 CEST Gerd Petermann wrote:
> > > Hi Karl,
> > > 
> > > if you are running a 64 Bit OS make sure to install a 64 Bit java
> > > runtime.
> > > This allows to increase the -Xmx option. You probably need > 5G
> > > for this.
> > > With a 32 Bit OS you can only try to disable the keep-complete
> > > option.
> > > 
> > > Gerd
> > > 
> > > ________________________________________
> > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > Auftrag von
> > > 7770 <7770 at foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020
> > > 10:45
> > > An: mkgmap-dev at lists.mkgmap.org.uk
> > > Betreff: [mkgmap-dev] splitter questions
> > > 
> > > Hi.
> > > 
> > > I am trying to split data for some 15 countries. First step is to
> > > combine
> > > the country data (with osmconvert) to one large o5m file and then
> > > try to
> > > split the combined file.
> > > The o5m file is about 14 GB in size.
> > > 
> > > When running splitter i give java -Xmx1600m
> > > options to splitter are: --max-nodes=1500000 --no-trim
> > > 
> > > Splitter calculates around 640 areas/tiles to be created.
> > > If i understand correctly, ways are parsed completely and then
> > > starts
> > > relations.
> > > 
> > > After 100 000 relations i get:
> > > ***** Full GC *****
> > > and soon thereafter:
> > > Exception in thread "main" Exception in thread "worker-0"
> > > 
> > >  java.lang.OutOfMemoryError: GC overhead limit exceeded... and
> > > many more
> > > 
> > > lines.
> > > 
> > > After this, one thread (out of two) dies but the other seems to
> > > hang doing
> > > nothing... and i have to manually kill it later.
> > > 
> > > 
> > > 
> > > Now, i can give java a little bit more memory to java, say up to
> > > 1800m.
> > > 
> > > setting  --max-nodes= larger should use less memory says
> > > documentation.
> > > With a larger max-nodes, will those tiles be faster or slower to
> > > display
> > > on
> > > a GPS unit?
> > > What is a reasonable value, 10 000 000 nodes or 2 000 000?
> > > 
> > > 
> > > Would --max-areas help my situation?
> > > If i use --max-areas=350 (approx half of total expected tiles),
> > > based on
> > > your experience would it be enough, or shall i better try with a
> > > value of
> > > 220 (~1/3 of the total tiles)?
> > > 
> > > Regards
> > > Karl
> > > 
> > > 
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev