[mkgmap-dev] Another splitter bug

Sat Aug 15 11:57:29 BST 2009

That doesn't really buy us much unfortunately, given the way the parsing 
works the checks need to be scattered about in various places (otherwise 
a separate validation pass would be required, taking even longer. I think 
a separate validation step is outside the scope of the splitter).

Experience in dealing with huge volumes of unreliable external data feeds 
at my current job has taught me there's only so much you can do to tollerate 
and recover from bad input. If the source data is corrupt/unexpected then 
it's often better to fail-fast and let a human deal with the problem rather 
than make assumptions about what is wrong with the data and carry on since 
that can just introduce further problems downstream. And something else I've 
learned is that no matter how many checks you have in place, something will 
always slip through the cracks anyway!

That aside, I don't think we have anything to worry about here. The splitter 
is already tollerant of unrecognised XML tags and attributes and for the 
most part this is all we really need. As I said in a previous post, any further 
problems that we hit of this nature are best dealt with on a case-by-case 
basis.

Chris

F> cant you put in the check as an option. With the -check_for_bad_date
F> all data will be past in the splitter for a growing number of know
F> xml bugs  and without the check .. you check will be done an the app
F> will be faster ?
F> 
F> lg Dirk