Data loss on big encoded polyline

[ismail BASKIN]

it is true as expected

--it is true as expected
SELECT ST_IsClosed(ST_LineFromEncodedPolyline('guhiFwmpqEs]fJwmAbmAcmAngAwGve@wGrcBg|@~bBo_@bQ{c@fTos@vLs]bGscBsD{uAkf@cuB{aAkMsD{kAc[os@rDs`AzOs{@zTwj@zOkCnZoPntBoPf|@oKzOseAj}A{c@zw@c[bj@s]bmAcQvj@oAfEcVfw@s{@vvDce@~gBwQfr@_b@rv@s]ve@s{@nU{}B~C{c@kCsl@gEkCSo|CsDkHoAk_A_SklBw`@s`AgJchA?kH?_eAoUon@{Os]sIkiASs`Ab[cpBf|@_`Ajf@wBz@kMvGgYvrAbBz^rDn}@jf@znBbLrhBjCjwDg@bBsD~McBnF{Jf^wVr{@{EfO{zAbuBcG~H{zA~bB{Of|@_b@jxAos@f|@od@jH_q@bLowCnZkbB?kk@Ss`Ab[s`A~bBwhA~oCco@f{CgOb`@kMb[{JvL{JvLkRnUcrAzOwmAgJ_`AzT_b@zaAwj@~p@gYfw@wGbrAbBrq@rDrhBvGf|@nAjnArNf^fw@zEjf@nF~f@zErv@v~@jz@rgEvV~eCfOrkCvLrIsDj`HrNzY~p@bwAjRb`@zJ{@jW_DvrAgOfEoAbcAgY~a@g^jCcGjWco@nPw[zOoZrXg^jH{JfYs]zJkMbo@sb@rcBkbBnZ{YfYgObGSzm@oA~oCbhAj_An_@faF~iArI~Cve@bQrrBbt@bQnUjRvVbLzO~sArtAnx@nU~a@be@z^be@ni@fh@rXfc@jWvVjRvo@zTfw@~Wnx@fc@v[be@bVvVzJb~@j\jp@R~Hzc@rIja@z@rIjCrSrDbj@rD~f@zE~a@vBb[vBj\bBz^nAnZSz^?~\Srb@wBnn@oArXcBni@kCf^?fc@SnFcBb`@g@~f@_Dbe@_Drg@kCn_@{Ej\_Drb@kCfYwBfJz^fmEr]~sFzTrgE?z_G??jC~oCbLnUvj@rDbrArDraCfJjpEsD~bBjH??zYz@zxBfEbpBzOjlBnUr~Ave@jlBvrAjiAz|@nvAjsAzr@nlAf|@~p@vj@fJ~_Ab[zh@r]z^vVjHzEzxBjxAznBbcAju@n_@f}BngA~gBnUve@jf@vmAngAbQnZ~\~k@r~AfJzw@sDvmAkk@nn@kk@fYcmAjCkxAkMseAwBcQcGkf@oPscBzT{fAbL_l@vj@gOzY{@r{@wB~_AgJbj@?~C?nqA_l@jH_DjbBogA~iAnUr{@fJ~nAr]nFbBvmAsD~M_DnbAoUjf@_l@zw@_q@vhA{T~dAgJzzAfJr~A~k@~dA~p@vmArq@fEz@vy@~Rbo@_Dbo@w`@?kk@oPw`@co@{Ts`A{OcmAkf@{EkH_q@wcAwGkxAbL{sBvj@gnBvhAkxAzE{@zc@kHns@g|@jk@kRbGoAbcAgfAf@g@ns@ogAfYogA?cmAwGclDcLwdCwGgnBoPcmA?_~AkHg^oKgh@_b@_~As]sIgYf@gxBv`@kdAbt@oyBbwAgc@Rw[{JoZgY_`AkeCos@okDk_AkkEkM{m@cBcGkp@gbCs]suC?scBbLcrAbo@_~Azw@shBvQkf@zh@stAjCwGjCcmAkf@czBgw@_cB_eAczB_eAgnBwL_l@gT{aA{TwdCkCwdC?_bERsDnK_~Ar]ogAns@g|@zEwBjsAsg@jlB_q@rg@_g@nZs]nU_X~\seArIgh@kHgwEwGkk@_b@{aAcLsq@{w@kxAcLogAka@g|@{^cj@w`@gm@{^os@k\co@oPg|@wBwLcQwy@wGwrARgEvBo~BkCwrA{TgnB_Ik\sS_{@?we@oPsq@sXc`@os@wrAg_Bwy@od@oUsg@_X{zAsv@crA{|@gw@scB_l@ovA{JwV{aAsrBkCwGgfAczBwt@c|AS_NbGos@g@cVwG_b@oP_SkRgTg_Bc[s~A_l@s`AgJwj@rDco@rv@g^j_AsSfh@g|@rq@_eAS{ToZ{TwrAoKkf@s]cmA{JcQcVod@wpB?gOkHkk@kWgYkk@oPsq@'));
 

it is false, because ST_AsEncodedPolyline-ST_LineFromEncodedPolyline corrupt data.

--it is false, because ST_AsEncodedPolyline-ST_LineFromEncodedPolyline corrupt data.
SELECT ST_IsClosed(ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(ST_LineFromEncodedPolyline('guhiFwmpqEs]fJwmAbmAcmAngAwGve@wGrcBg|@~bBo_@bQ{c@fTos@vLs]bGscBsD{uAkf@cuB{aAkMsD{kAc[os@rDs`AzOs{@zTwj@zOkCnZoPntBoPf|@oKzOseAj}A{c@zw@c[bj@s]bmAcQvj@oAfEcVfw@s{@vvDce@~gBwQfr@_b@rv@s]ve@s{@nU{}B~C{c@kCsl@gEkCSo|CsDkHoAk_A_SklBw`@s`AgJchA?kH?_eAoUon@{Os]sIkiASs`Ab[cpBf|@_`Ajf@wBz@kMvGgYvrAbBz^rDn}@jf@znBbLrhBjCjwDg@bBsD~McBnF{Jf^wVr{@{EfO{zAbuBcG~H{zA~bB{Of|@_b@jxAos@f|@od@jH_q@bLowCnZkbB?kk@Ss`Ab[s`A~bBwhA~oCco@f{CgOb`@kMb[{JvL{JvLkRnUcrAzOwmAgJ_`AzT_b@zaAwj@~p@gYfw@wGbrAbBrq@rDrhBvGf|@nAjnArNf^fw@zEjf@nF~f@zErv@v~@jz@rgEvV~eCfOrkCvLrIsDj`HrNzY~p@bwAjRb`@zJ{@jW_DvrAgOfEoAbcAgY~a@g^jCcGjWco@nPw[zOoZrXg^jH{JfYs]zJkMbo@sb@rcBkbBnZ{YfYgObGSzm@oA~oCbhAj_An_@faF~iArI~Cve@bQrrBbt@bQnUjRvVbLzO~sArtAnx@nU~a@be@z^be@ni@fh@rXfc@jWvVjRvo@zTfw@~Wnx@fc@v[be@bVvVzJb~@j\jp@R~Hzc@rIja@z@rIjCrSrDbj@rD~f@zE~a@vBb[vBj\bBz^nAnZSz^?~\Srb@wBnn@oArXcBni@kCf^?fc@SnFcBb`@g@~f@_Dbe@_Drg@kCn_@{Ej\_Drb@kCfYwBfJz^fmEr]~sFzTrgE?z_G??jC~oCbLnUvj@rDbrArDraCfJjpEsD~bBjH??zYz@zxBfEbpBzOjlBnUr~Ave@jlBvrAjiAz|@nvAjsAzr@nlAf|@~p@vj@fJ~_Ab[zh@r]z^vVjHzEzxBjxAznBbcAju@n_@f}BngA~gBnUve@jf@vmAngAbQnZ~\~k@r~AfJzw@sDvmAkk@nn@kk@fYcmAjCkxAkMseAwBcQcGkf@oPscBzT{fAbL_l@vj@gOzY{@r{@wB~_AgJbj@?~C?nqA_l@jH_DjbBogA~iAnUr{@fJ~nAr]nFbBvmAsD~M_DnbAoUjf@_l@zw@_q@vhA{T~dAgJzzAfJr~A~k@~dA~p@vmArq@fEz@vy@~Rbo@_Dbo@w`@?kk@oPw`@co@{Ts`A{OcmAkf@{EkH_q@wcAwGkxAbL{sBvj@gnBvhAkxAzE{@zc@kHns@g|@jk@kRbGoAbcAgfAf@g@ns@ogAfYogA?cmAwGclDcLwdCwGgnBoPcmA?_~AkHg^oKgh@_b@_~As]sIgYf@gxBv`@kdAbt@oyBbwAgc@Rw[{JoZgY_`AkeCos@okDk_AkkEkM{m@cBcGkp@gbCs]suC?scBbLcrAbo@_~Azw@shBvQkf@zh@stAjCwGjCcmAkf@czBgw@_cB_eAczB_eAgnBwL_l@gT{aA{TwdCkCwdC?_bERsDnK_~Ar]ogAns@g|@zEwBjsAsg@jlB_q@rg@_g@nZs]nU_X~\seArIgh@kHgwEwGkk@_b@{aAcLsq@{w@kxAcLogAka@g|@{^cj@w`@gm@{^os@k\co@oPg|@wBwLcQwy@wGwrARgEvBo~BkCwrA{TgnB_Ik\sS_{@?we@oPsq@sXc`@os@wrAg_Bwy@od@oUsg@_X{zAsv@crA{|@gw@scB_l@ovA{JwV{aAsrBkCwGgfAczBwt@c|AS_NbGos@g@cVwG_b@oP_SkRgTg_Bc[s~A_l@s`AgJwj@rDco@rv@g^j_AsSfh@g|@rq@_eAS{ToZ{TwrAoKkf@s]cmA{JcQcVod@wpB?gOkHkk@kWgYkk@oPsq@'))));

[woodbri]

The encode and decode function take a precision=5 argument. This means that it truncates the floating point numbers at that number of significant decimal point digits. So if you encode and decode a linestring you will not get back "exactly" the same linestring. This is the definition of the encoding and decoding functions. So there is no error here other then understanding what the function is doing.

[ismail BASKIN]

Actually this problem happened on 155+ point.

This is true as expected (155 point)

SELECT ST_IsClosed(ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(ST_GeomFromText('LINESTRING(34.4958 38.3882,34.494 38.3931,34.4815 38.4057,34.4699 38.4182,34.4637 38.4196,34.4476 38.421,34.4316 38.4308,34.4287 38.436,34.4253 38.4419,34.4231 38.4503,34.4218 38.4552,34.4227 38.4713,34.429 38.4852,34.4397 38.5041,34.4406 38.5064,34.4451 38.5187,34.4442 38.5271,34.4415 38.5376,34.438 38.5473,34.4353 38.5543,34.4309 38.555,34.4121 38.5578,34.4023 38.5606,34.3996 38.5626,34.3845 38.5739,34.3754 38.5798,34.3685 38.5843,34.356 38.5892,34.349 38.5921,34.348 38.5925,34.339 38.5962,34.3096 38.6059,34.2928 38.612,34.2846 38.615,34.2757 38.6206,34.2695 38.6255,34.2659 38.6352,34.2651 38.6555,34.2658 38.6614,34.2668 38.6687,34.2669 38.6694,34.2678 38.6946,34.2682 38.6961,34.2714 38.7064,34.2768 38.7239,34.2786 38.7344,34.2786 38.7461,34.2786 38.7476,34.2822 38.7588,34.2849 38.7664,34.2866 38.7713,34.2867 38.7832,34.2822 38.7937,34.2724 38.8118,34.2661 38.8222,34.2658 38.8228,34.2644 38.8251,34.251 38.8293,34.2459 38.8288,34.2359 38.8279,34.218 38.8216,34.2011 38.8195,34.1716 38.8188,34.1711 38.819,34.1687 38.8199,34.1675 38.8204,34.1625 38.8223,34.1528 38.8261,34.1502 38.8272,34.1313 38.8419,34.1297 38.8432,34.1137 38.8579,34.1039 38.8606,34.0896 38.8662,34.0798 38.8746,34.0783 38.8806,34.0762 38.8886,34.0718 38.913,34.0718 38.9289,34.0719 38.936,34.0674 38.9465,34.0514 38.957,34.0282 38.9688,34.0032 38.9765,33.9979 38.9791,33.9934 38.9814,33.9912 38.9833,33.989 38.9852,33.9854 38.9883,33.9827 39.0016,33.9845 39.0142,33.981 39.0246,33.9703 39.0302,33.9623 39.0372,33.9533 39.0414,33.94 39.0428,33.9319 39.0423,33.915 39.0414,33.9052 39.04,33.8925 39.0396,33.8875 39.0371,33.8864 39.0281,33.8852 39.0218,33.8841 39.0154,33.8739 39.0065,33.8418 38.997,33.8202 38.9932,33.7977 38.9906,33.796 38.9884,33.7497 38.9893,33.7454 38.9868,33.7313 38.9788,33.726 38.9757,33.7263 38.9738,33.7271 38.9699,33.7297 38.9565,33.7301 38.9555,33.7343 38.9446,33.7393 38.939,33.7406 38.9383,33.7483 38.9344,33.7529 38.9316,33.7573 38.9289,33.7623 38.9248,33.7642 38.9233,33.7691 38.9191,33.7714 38.9172,33.7771 38.9095,33.793 38.8934,33.7973 38.889,33.7999 38.8848,33.8 38.8835,33.8004 38.876,33.7887 38.8528,33.7835 38.8425,33.7715 38.8063,33.7707 38.8046,33.7678 38.7984,33.7593 38.7799,33.7557 38.777,33.7519 38.7739,33.7492 38.7718,33.7355 38.7582,33.7319 38.749,33.7258 38.7434,33.7197 38.7383,33.7131 38.7315,33.7073 38.7274,33.7035 38.7235,33.6957 38.7204,33.6867 38.7169,33.6775 38.7129,33.6729 38.7071,33.6692 38.701,33.6673 38.6972,34.4958 38.3882)',4326))))));

I just added POINT(33.6626 38.687) at last (before ending point) then it returns false (156 point)

SELECT ST_IsClosed(ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(ST_GeomFromText('LINESTRING(34.4958 38.3882,34.494 38.3931,34.4815 38.4057,34.4699 38.4182,34.4637 38.4196,34.4476 38.421,34.4316 38.4308,34.4287 38.436,34.4253 38.4419,34.4231 38.4503,34.4218 38.4552,34.4227 38.4713,34.429 38.4852,34.4397 38.5041,34.4406 38.5064,34.4451 38.5187,34.4442 38.5271,34.4415 38.5376,34.438 38.5473,34.4353 38.5543,34.4309 38.555,34.4121 38.5578,34.4023 38.5606,34.3996 38.5626,34.3845 38.5739,34.3754 38.5798,34.3685 38.5843,34.356 38.5892,34.349 38.5921,34.348 38.5925,34.339 38.5962,34.3096 38.6059,34.2928 38.612,34.2846 38.615,34.2757 38.6206,34.2695 38.6255,34.2659 38.6352,34.2651 38.6555,34.2658 38.6614,34.2668 38.6687,34.2669 38.6694,34.2678 38.6946,34.2682 38.6961,34.2714 38.7064,34.2768 38.7239,34.2786 38.7344,34.2786 38.7461,34.2786 38.7476,34.2822 38.7588,34.2849 38.7664,34.2866 38.7713,34.2867 38.7832,34.2822 38.7937,34.2724 38.8118,34.2661 38.8222,34.2658 38.8228,34.2644 38.8251,34.251 38.8293,34.2459 38.8288,34.2359 38.8279,34.218 38.8216,34.2011 38.8195,34.1716 38.8188,34.1711 38.819,34.1687 38.8199,34.1675 38.8204,34.1625 38.8223,34.1528 38.8261,34.1502 38.8272,34.1313 38.8419,34.1297 38.8432,34.1137 38.8579,34.1039 38.8606,34.0896 38.8662,34.0798 38.8746,34.0783 38.8806,34.0762 38.8886,34.0718 38.913,34.0718 38.9289,34.0719 38.936,34.0674 38.9465,34.0514 38.957,34.0282 38.9688,34.0032 38.9765,33.9979 38.9791,33.9934 38.9814,33.9912 38.9833,33.989 38.9852,33.9854 38.9883,33.9827 39.0016,33.9845 39.0142,33.981 39.0246,33.9703 39.0302,33.9623 39.0372,33.9533 39.0414,33.94 39.0428,33.9319 39.0423,33.915 39.0414,33.9052 39.04,33.8925 39.0396,33.8875 39.0371,33.8864 39.0281,33.8852 39.0218,33.8841 39.0154,33.8739 39.0065,33.8418 38.997,33.8202 38.9932,33.7977 38.9906,33.796 38.9884,33.7497 38.9893,33.7454 38.9868,33.7313 38.9788,33.726 38.9757,33.7263 38.9738,33.7271 38.9699,33.7297 38.9565,33.7301 38.9555,33.7343 38.9446,33.7393 38.939,33.7406 38.9383,33.7483 38.9344,33.7529 38.9316,33.7573 38.9289,33.7623 38.9248,33.7642 38.9233,33.7691 38.9191,33.7714 38.9172,33.7771 38.9095,33.793 38.8934,33.7973 38.889,33.7999 38.8848,33.8 38.8835,33.8004 38.876,33.7887 38.8528,33.7835 38.8425,33.7715 38.8063,33.7707 38.8046,33.7678 38.7984,33.7593 38.7799,33.7557 38.777,33.7519 38.7739,33.7492 38.7718,33.7355 38.7582,33.7319 38.749,33.7258 38.7434,33.7197 38.7383,33.7131 38.7315,33.7073 38.7274,33.7035 38.7235,33.6957 38.7204,33.6867 38.7169,33.6775 38.7129,33.6729 38.7071,33.6692 38.701,33.6673 38.6972,33.6626 38.687,34.4958 38.3882)',4326))))));

Still true when not using encode-decode with extra point (156 point)

SELECT ST_IsClosed(ST_GeomFromText('LINESTRING(34.4958 38.3882,34.494 38.3931,34.4815 38.4057,34.4699 38.4182,34.4637 38.4196,34.4476 38.421,34.4316 38.4308,34.4287 38.436,34.4253 38.4419,34.4231 38.4503,34.4218 38.4552,34.4227 38.4713,34.429 38.4852,34.4397 38.5041,34.4406 38.5064,34.4451 38.5187,34.4442 38.5271,34.4415 38.5376,34.438 38.5473,34.4353 38.5543,34.4309 38.555,34.4121 38.5578,34.4023 38.5606,34.3996 38.5626,34.3845 38.5739,34.3754 38.5798,34.3685 38.5843,34.356 38.5892,34.349 38.5921,34.348 38.5925,34.339 38.5962,34.3096 38.6059,34.2928 38.612,34.2846 38.615,34.2757 38.6206,34.2695 38.6255,34.2659 38.6352,34.2651 38.6555,34.2658 38.6614,34.2668 38.6687,34.2669 38.6694,34.2678 38.6946,34.2682 38.6961,34.2714 38.7064,34.2768 38.7239,34.2786 38.7344,34.2786 38.7461,34.2786 38.7476,34.2822 38.7588,34.2849 38.7664,34.2866 38.7713,34.2867 38.7832,34.2822 38.7937,34.2724 38.8118,34.2661 38.8222,34.2658 38.8228,34.2644 38.8251,34.251 38.8293,34.2459 38.8288,34.2359 38.8279,34.218 38.8216,34.2011 38.8195,34.1716 38.8188,34.1711 38.819,34.1687 38.8199,34.1675 38.8204,34.1625 38.8223,34.1528 38.8261,34.1502 38.8272,34.1313 38.8419,34.1297 38.8432,34.1137 38.8579,34.1039 38.8606,34.0896 38.8662,34.0798 38.8746,34.0783 38.8806,34.0762 38.8886,34.0718 38.913,34.0718 38.9289,34.0719 38.936,34.0674 38.9465,34.0514 38.957,34.0282 38.9688,34.0032 38.9765,33.9979 38.9791,33.9934 38.9814,33.9912 38.9833,33.989 38.9852,33.9854 38.9883,33.9827 39.0016,33.9845 39.0142,33.981 39.0246,33.9703 39.0302,33.9623 39.0372,33.9533 39.0414,33.94 39.0428,33.9319 39.0423,33.915 39.0414,33.9052 39.04,33.8925 39.0396,33.8875 39.0371,33.8864 39.0281,33.8852 39.0218,33.8841 39.0154,33.8739 39.0065,33.8418 38.997,33.8202 38.9932,33.7977 38.9906,33.796 38.9884,33.7497 38.9893,33.7454 38.9868,33.7313 38.9788,33.726 38.9757,33.7263 38.9738,33.7271 38.9699,33.7297 38.9565,33.7301 38.9555,33.7343 38.9446,33.7393 38.939,33.7406 38.9383,33.7483 38.9344,33.7529 38.9316,33.7573 38.9289,33.7623 38.9248,33.7642 38.9233,33.7691 38.9191,33.7714 38.9172,33.7771 38.9095,33.793 38.8934,33.7973 38.889,33.7999 38.8848,33.8 38.8835,33.8004 38.876,33.7887 38.8528,33.7835 38.8425,33.7715 38.8063,33.7707 38.8046,33.7678 38.7984,33.7593 38.7799,33.7557 38.777,33.7519 38.7739,33.7492 38.7718,33.7355 38.7582,33.7319 38.749,33.7258 38.7434,33.7197 38.7383,33.7131 38.7315,33.7073 38.7274,33.7035 38.7235,33.6957 38.7204,33.6867 38.7169,33.6775 38.7129,33.6729 38.7071,33.6692 38.701,33.6673 38.6972,33.6626 38.687,34.4958 38.3882)',4326));

[woodbri]

Thanks for example, this will help someone narrow down the issue.

[ismail BASKIN]

Is there any improvement?

[robe]

No. Still get same answer in PostGIS 2.4 dev. @dbaston any thoughts on this. I vaguely remember a limitation of representing things like polygons so wondering if that affects closed rings as well though why LineFrom wouldn't suffer the same issue doesn't make sense.

[robe]

I'm guessing this won't be done anytime soon. Push back if you can get to it in next day or 2.

[pramsey]

Here's an example that is small enough to debug:

with d as (select 'SRID=4326;LINESTRING(33.6729 38.7071,33.6692 38.701,33.6673 38.6972,33.6626 38.6871)'::geometry as g),
e as (
   select g, ST_LineFromEncodedPolyline(ST_AsEncodedPolyline(g)) as gl
   from d
   ) 
select 
  st_astext(g), ST_AsEncodedPolyline(g), st_npoints(g), 
  st_astext(gl), st_npoints(gl)
from e

yuck.

[pramsey]

In 16130:

Support google line encodings that include a backslash character (References #3713)

[pramsey]

In 16131:

Support google line encodings that include a backslash character (References #3713)

[pramsey]

In 16132:

Support google line encodings that include a backslash character (References #3713)

[pramsey]

There were actually two things going on. The original author thought that it would be smart to escape the
character in the output, but actually the format didn't like that, it preferred the raw character. Also the postgis/ side caller was prematurely freeing the database memory underneath, which caused failures *sometimes*. Fix applied to trunk, 2.4, 2.3.