You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is incorrect. If two points are equal to a precision of 7, then by definition they must be equal to a precision of 6 as well. The problem here arises from the use of toFixed to round the coordinates. The problem is that this introduces rounding errors that then cause the equality check to fail.
I think a better way to check that values are within the required precision is something like:
Math.abs(a - b) < (10**-6) // returns true, as expected
Math.abs(a - b) < (10**-7) // returns true, as expected
Very happy to make a PR if you agree that this is a preferable approach.
The text was updated successfully, but these errors were encountered:
@solarissmoke my apologies for responding late. Thank you for pointing out this issue.
Your approach seems better in this case as well as in general. You can proceed with PR if still you have time or else fix this on coming weekend.
The way
GeojsonEquality
check handles precision can result in incorrect results. The following example illustrates the problem:This is incorrect. If two points are equal to a precision of 7, then by definition they must be equal to a precision of 6 as well. The problem here arises from the use of toFixed to round the coordinates. The problem is that this introduces rounding errors that then cause the equality check to fail.
I think a better way to check that values are within the required precision is something like:
Very happy to make a PR if you agree that this is a preferable approach.
The text was updated successfully, but these errors were encountered: