Implementing SLIC image segmentation

[SCrommelinck]

Would someone be interested in implementing Simple Linear Iterative Clustering (SLIC) image segmentation in GRASS?

References:

• ​http://ivrl.epfl.ch/research/superpixels
• ​http://docs.opencv.org/trunk/df/d6c/group__ximgproc__superpixel.html
• ​http://scikit-image.org/docs/dev/api/skimage.segmentation.html#skimage.segmentation.slic

[tgrippa]

Hi,

First of all, thank to GRASS GIS contributors for the great jobs they make, especially for recent advances in segmentation :D.

In our lab (ANAGEO, Free University of Brussels), we moved recently from proprietary software (eCognition) to GRASS GIS and we are very happy of this change. As we are regularly confronted with large image datasets (>2 billion pixels with multispectral bands), we have to face quite important processing time, which is an obstacle for dealing with "real" big datasets. We hope that SLIC segmentation will be implemented in GRASS GIS, which would allow us to save processing time and improve our ambitions.

Thank you

[mlennert]

After some discussion with MarkusM, I see three options, all based on the source code published at ​http://ivrl.epfl.ch/research/superpixels, which is GPL3:

• Integrate the code as is into GRASS. Disadvantage: it's C++ and we would have to maintain it. Advantage: code exists, so less work, we would "only" have to write the GRASS io wrapper
• Translate the original code to C and integrate into GRASS. Disadvantage: more work, we have to maintain; Advantage: no C++, no installation by user, native implementation in GRASS
• Write a GRASS module that calls the compiled command line tool. Disadvantage: the user would have to install and compile the code. Advantage: less work for us, others maintain

[rashadkm]

Replying to mlennert:

After some discussion with MarkusM, I see three options, all based on the source code published at ​http://ivrl.epfl.ch/research/superpixels, which is GPL3:

• Integrate the code as is into GRASS. Disadvantage: it's C++ and we would have to maintain it. Advantage: code exists, so less work, we would "only" have to write the GRASS io wrapper
• Translate the original code to C and integrate into GRASS. Disadvantage: more work, we have to maintain; Advantage: no C++, no installation by user, native implementation in GRASS
• Write a GRASS module that calls the compiled command line tool. Disadvantage: the user would have to install and compile the code. Advantage: less work for us, others maintain

mortiz, the second option seems better than others. right?

first is okay, if we need to drag in c++. but still there needs to be some additional work. why not write new grass module. we can port/copy/translate those c++ methods into using grass api wherever possible.

[mlennert]

Replying to rashadkm:

Replying to mlennert:

After some discussion with MarkusM, I see three options, all based on the source code published at ​http://ivrl.epfl.ch/research/superpixels, which is GPL3:

• Integrate the code as is into GRASS. Disadvantage: it's C++ and we would have to maintain it. Advantage: code exists, so less work, we would "only" have to write the GRASS io wrapper
• Translate the original code to C and integrate into GRASS. Disadvantage: more work, we have to maintain; Advantage: no C++, no installation by user, native implementation in GRASS
• Write a GRASS module that calls the compiled command line tool. Disadvantage: the user would have to install and compile the code. Advantage: less work for us, others maintain

mortiz, the second option seems better than others. right?

I personally would prefer it, yes, but it means more work and someone has to do it. Are you volunteering ? :-)

why not write new grass module. we can port/copy/translate those c++ methods into using grass api wherever possible.

+1 (with the above reservations).

Moritz

[rashadkm]

Replying to mlennert:

Replying to rashadkm:

Replying to mlennert:

After some discussion with MarkusM, I see three options, all based on the source code published at ​http://ivrl.epfl.ch/research/superpixels, which is GPL3:

• Integrate the code as is into GRASS. Disadvantage: it's C++ and we would have to maintain it. Advantage: code exists, so less work, we would "only" have to write the GRASS io wrapper
• Translate the original code to C and integrate into GRASS. Disadvantage: more work, we have to maintain; Advantage: no C++, no installation by user, native implementation in GRASS
• Write a GRASS module that calls the compiled command line tool. Disadvantage: the user would have to install and compile the code. Advantage: less work for us, others maintain

mortiz, the second option seems better than others. right?

I personally would prefer it, yes, but it means more work and someone has to do it. Are you volunteering ? :-)

I am interested to work.. given help from all grass devs ;).

why not write new grass module. we can port/copy/translate those c++ methods into using grass api wherever possible.

+1 (with the above reservations).

Moritz

[mlennert]

Replying to rashadkm:

Replying to mlennert:

Replying to rashadkm:

Replying to mlennert:

After some discussion with MarkusM, I see three options, all based on the source code published at ​http://ivrl.epfl.ch/research/superpixels, which is GPL3:

• Integrate the code as is into GRASS. Disadvantage: it's C++ and we would have to maintain it. Advantage: code exists, so less work, we would "only" have to write the GRASS io wrapper
• Translate the original code to C and integrate into GRASS. Disadvantage: more work, we have to maintain; Advantage: no C++, no installation by user, native implementation in GRASS
• Write a GRASS module that calls the compiled command line tool. Disadvantage: the user would have to install and compile the code. Advantage: less work for us, others maintain

mortiz, the second option seems better than others. right?

I personally would prefer it, yes, but it means more work and someone has to do it. Are you volunteering ? :-)

I am interested to work.. given help from all grass devs ;).

Well, maybe not all, but I'm willing to help, and we should definitely have MarkusM on board.

[rashadkm]

Thanks. I will be back hopefully with some code :-). I will definitely ping back if have questions. Either here or on grass-dev.

[mmetz]

Replying to mlennert:

Replying to rashadkm:

Replying to mlennert:

Replying to rashadkm:

Replying to mlennert:

After some discussion with MarkusM, I see three options, all based on the source code published at ​http://ivrl.epfl.ch/research/superpixels, which is GPL3:

• Integrate the code as is into GRASS. Disadvantage: it's C++ and we would have to maintain it. Advantage: code exists, so less work, we would "only" have to write the GRASS io wrapper
• Translate the original code to C and integrate into GRASS. Disadvantage: more work, we have to maintain; Advantage: no C++, no installation by user, native implementation in GRASS
• Write a GRASS module that calls the compiled command line tool. Disadvantage: the user would have to install and compile the code. Advantage: less work for us, others maintain

mortiz, the second option seems better than others. right?

I personally would prefer it, yes, but it means more work and someone has to do it. Are you volunteering ? :-)

I am interested to work.. given help from all grass devs ;).

Well, maybe not all, but I'm willing to help, and we should definitely have MarkusM on board.

I am on board. SLIC(O) is an interesting method, and I would be happy to help with the implementation. I would also prefer the second option, even if it means more work, but the result might be more stable/portable.

[mlennert]

Ok, so we agree on option 2, i.e. a native GRASS implementation translating the original C++ code.

And as Markus mentions it should include the SLICO version. Maybe as a flag ? Something like: i.superpixels.slic with a flag -o for SLICO ? Or i.slic -o ? Or i.segment.slic ? If we plan on implementing other superpixel approaches the i.superpixels.slic might be the way to go...

[rashadkm]

could i.segment have an option to choose between slic and other ones?

[mlennert]

Replying to rashadkm:

could i.segment have an option to choose between slic and other ones?

It's possible, but:

• SLIC itself is complex enough that MarkusM suggested it should go into a separate module to not overcharge i.segment
• From what I've seen up to now, the use of superpixels in remote sensing is mostly to reduce the number of "pixels" to treat for segmentation. In that sense the "segmentation" into superpixels is not exactly the same as segmentation using other algorithms. But my experience is limited and the differences I see are more semantic than anything else.

[rashadkm]

Okay. in that case we should decide on module name. i.superpixels.slic or i.segment.slic ?

[tgrippa]

Replying to rashadkm:

Okay. in that case we should decide on module name. i.superpixels.slic or i.segment.slic ?

I think i.superpixels.slic is more interesting in perspective of further development of other superpixels methods, but it's only my opinion.

[tgrippa]

For information, the Windows GUI which was provided in the EPFL website (​http://ivrl.epfl.ch/research/superpixels) didn't generate the .dat file containing the segment labels. I contacted the author (Achanta Radhakrishna) who fix this. The version on the website is the fixed one.

[mmetz]

Replying to tgrippa:

For information, the Windows GUI which was provided in the EPFL website (​http://ivrl.epfl.ch/research/superpixels) didn't generate the .dat file containing the segment labels. I contacted the author (Achanta Radhakrishna) who fix this. The version on the website is the fixed one.

I suggest to continue the discussion about details of the implementation of SLIC(O) in GRASS in the mailing list or offlist in order to not crowd this ticket.

[neteler]

Ticket retargeted after milestone closed

[rashadkm]

here is first version on grass7 addons: ​https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic

can we close this ticket?

[mlennert]

Replying to rashadkm:

here is first version on grass7 addons: ​https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic

can we close this ticket?

There are still one ​comment from Markus that you didn't apply: the memory allocation for distvec. Just uncomment ​lines 353 and following, not forgetting to add the G_free() call later (see the patch in #3247). Otherwise the memory is allocated in the stack, where there might not be enough available, rather than the heap.

When this is done, I agree that this ticket can be closed and we can open new tickets for any other problems.

However, while you're at it: could you also consider the two additions proposed in that same patch of #3247, assigning a color table and adding the module call to the raster map's history ?

[mlennert]

i.superpixels.slic is now available as an addon, so closing.