Sunday, October 6, 2013

PCL thread titled Keypoint indices

This is a discussion on PCL...

Hi Mustafa,
I was trying to do the same thing with you, in the end I ended up doing this:    (If I'm wrong please correct me!!!)
- Use the ponitCloud with the Keypoints (not with indices, with the XYZ information, just as you mention)
- Use them as input at the Feature Estimator, but I also set as *SearchSurface* the initial full pointcloud! that means that (ig I get it right) you find the keypoints, then for the XYZ location of each keypoint, it goes at the initial cloud (with the full information) and computes stuff there.

cloudVector_UNorg_PN - full pointCloud (actually vector of Pointclouds<>::Ptr), with Normals, it is unorganized, because I remove NaN points and Normals
KEYptVector_UNorg_P_ - Keypoint Pointcloluds (actually vector of Pointclouds<>::Ptr)
FEATtVector_UNorg_P_ - Feature Pointcloluds (actually vector of Pointclouds<>::Ptr)

**_PN** means Point and Normal
typedef pcl::SHOT1344  keyPointTYPE;       // float descriptor [1344]   ||   float rf [9]        
QVector< pcl::PointCloud::Ptr >  cloudVector_UNorg_PN;
QVector< pcl::PointCloud::Ptr >  KEYptVector_UNorg_P_;
QVector< pcl::PointCloud<     keyPointTYPE     >::Ptr >  FEATtVector_UNorg_P_;
(you can get away without using vectors of course, this is for my needs only)
Please have a look at the code snippet. The result that I'm getting is pretty expected.
After this stage, before finding correspondences, be sure to kick out all the Nan or Inf things at your Keypoints and Features!
If you delete a Feature, delete also the correspoding Keypoint!
There is an 1-1 relation between the Keypoint-Cloud and the Feature-Cloud

If other people have different opinion, please comment :)
If there is a way to work with indices, I would be interested too.

Best,
dimitris


                            std::cout << "//// KEYpoints" << std::endl;
                            {
                                // Fill in the model cloud
                                // Compute model_resolution
                                pcl::ISSKeypoint3D< pcl::PointXYZRGBNormal, pcl::PointXYZRGBNormal> keyPointDetector;
                                pcl::search::KdTree::Ptr tree (new pcl::search::KdTree );
                                model_resolution_Target = static_cast(  computeCloudResolution( cloudVector_UNorg_PN[iii] )  );
                                keyPointDetector.setSearchMethod(    tree);
                                keyPointDetector.setSalientRadius(   PARAM_KEYP_coeffSalientRadius * model_resolution_Target);
                                keyPointDetector.setNonMaxRadius(    PARAM_KEYP_coeffNonMaxRadiuss * model_resolution_Target);
                                keyPointDetector.setThreshold21(     0.975);
                                keyPointDetector.setThreshold32(     0.975);
                                keyPointDetector.setMinNeighbors(    5);
                                keyPointDetector.setNumberOfThreads( 4);

                                pcl::PointCloud::Ptr TMPPP_keyPt_UNorg_P_(new pcl::PointCloud);

                                keyPointDetector.setInputCloud(  cloudVector_UNorg_PN[iii] );
                                keyPointDetector.compute(       *TMPPP_keyPt_UNorg_P_ );
                                KEYptVector_UNorg_P_.append(     TMPPP_keyPt_UNorg_P_ );
                            }


                            std::cout << "//// Feature Estimation" << std::endl;
                            {
                                    pcl::search::KdTree::Ptr tree (new pcl::search::KdTree);
                                    pcl::SHOTColorEstimationOMP< pcl::PointXYZRGBNormal,pcl::PointXYZRGBNormal,keyPointTYPE > featureDescriptor;
                                    featureDescriptor.setSearchSurface( cloudVector_UNorg_PN[iii] );
                                    featureDescriptor.setInputNormals(  cloudVector_UNorg_PN[iii] );
                                    featureDescriptor.setInputCloud(    KEYptVector_UNorg_P_[iii] );     // --->> Comupte Features ONLY @ Keypoints !!!!!!!!!!!!!!!!!
                                    featureDescriptor.setSearchMethod(  tree );
                                    featureDescriptor.setRadiusSearch(  PARAM_FEAT_coeffRadiusSearch * model_resolution_Target ); 
                                    featureDescriptor.compute(         *TMPPP_featt_UNorg_P_ );
                                    FEATtVector_UNorg_P_.append(        TMPPP_featt_UNorg_P_ );
                            }

Dimitris Tzionas


On Sun, Oct 6, 2013 at 11:20 PM, Mustafa Mohamad <mustafa.mohamad@gmail.com> wrote:
Hi, 

I am trying to get the indices of the detected keypoints using the ISS keypoint detector.  The compute method of the ISS detector returns a cloud of the actual keypoint values.  However, I need the indices so that I can calculate local shape descriptors for those points in the original cloud.

There must be something very basic I am missing.

Mustafa

Saturday, October 5, 2013

Books Read

1. The Alchemist by Paulo Coelho (5 / 5)



2. The Fountainhead by Ayn Rand (4.5 / 5)

Ayn Rand was a great writer. This book cannot be read in few sittings. I must say that she makes her point so well in a profound and convincing manner. I must say that this is a must read book for any young minds.



3. Turning Points by APJ Abdul Kalam (ongoing)



4. What do you care others think by Richard Feynnman (on going, may be 4 / 5)

Here he makes a point "What "



5. The Last Lecture by Randy Pausch (4.75 / 5)

Great and inspiring. It was really a last lecture of a professor's life where he was asked to potray what he wanted to say into an hour lecture. It was last lecture of his life as he was suffering from terminating cancer. One can find the video in youtube and book has almost the same content of the video.


6. An Autobiography of a Yogi by Paramahamsa Yogananda (5 / 5)


7. The Kite Runner by Khaled Hosseini (4.5 / 5)

Tuesday, October 1, 2013

Discussions on FAST Corner Detection...Part 2

source:
http://fastcorner.wordpress.com/2010/09/09/fast-faq/
http://www.edwardrosten.com/work/fast.html


Great many thanks to Edward Rosten for creating a forum to ask questions and promptly answering them.

The below paragraphs are copied from the forum which I thought are helpful :)


Kang-Kook Kong says:
Hi, I am interested in FAST-ER
I run ‘./configure && make’ in fast-er-1.4, but I got the error message:
checking libCVD… yes
checking for dgetri_ in -lcvd… no
configure: error: libCVD must be compiled with lapack support.
Can you help me?
  • edrosten says:
    Do you have the LAPACK development libraries installed on your machine? If so, libCVD should pick them out automatically. Try the following:
    1. Check the configure output from libcvd. If it doesn’t find LAPACK, then install the LAPACK development libraries.
    2. Check the built libcvd using ldd /usr/local/lib/libcvd.so and see if it has a reference to lapack. If it’s not there, then there libCVD has failed to pick up lapack properly. If 1 is OK and 2 isn’t, then let me know.
    3. As a workaround, run export LDFLAGS=-llapack before running ./configure to get LAPACK manually.
    1. Vinny says:
      Hey
      I just wanted to tell you that you are my life saver. I just used your code for my final year project and it works like a charm. Actually I was on the verge of failing my project and was very worried about it until I saw your code. Thanks a lot. :’)
      But the thing is, I do not understand the program at all. At least, could you give me the gist of what the code actually does? Because I have to explain it in my report and also to my professor.
      Your help is greatly appreciated! :)
      Vinny
      • edrosten says:
        I’m glad it was so helpful.
        The code is almost impossible to read because it is machine generated.
        What it does is tell you if there is a contiguous arc of 9 or more pixels which are either all much brighter or all much darker than the centre pixel.
        Instead of looking at pixels in the circle in sequence, the program uses a decision tree in order to reject non-corners as quickly as possible.

      1. Perikles Rammos says:
        Hi, and thanks for this very nice algorithm.
        My questions are mainly on the construction of the decision tree from the training images.
        1. The minimization of the entropy function is used only for creating the decision tree?
        2. Does the training set needs to be annotated? (I guess not, since all 16 pixels are checked)
        3. I assume that with very few training samples (corners), a very simple decision tree will be created. When new corners are trained, how is the tree expanded? (maybe the answer is in the paper, I haven’t been able to figure it out yet)
        Thanks.
        • edrosten says:
          To answer your questions:
          1. yes
          2. The training set does need to be annotated. There are two ways of doing this.
            1. Classify points in an image using a slow, simple algorithm that checks all 16 pixels
            2. Generate all possible corners and non corners. Since each pixel can have 3 states, there are 3^16 ~= 46 million possible corners and non corners.
            I actually use a combination of both methods. I generate all possible corners to get a complete coverage of the data. I then extract corners and non corners from a number of images, so that the pixel statistics are represented in the detector.
            Bear in mind that when it gets as far as training, the pixels are either b (brighter), d (darker) or s (similar) relative to the centre, so corner candidates can be thought of as a list of 16 elements likebsdbbddbsdbsbdsb
            There is program in the FAST-ER distribution to generate exactly one of each type of corner and non-corner, along with the classification.
          3. Yes, if you have too few training samples, then the tree will not be representative of the segment test criterion. The tree is never expanded, because the tree is not trained incrementally. I collect examples of corners first, then train the tree, then emit source code and then use the tree. In principle incremental training could be done however, but I’m not sure how useful that would be.

Discussions on FAST Corner Detection...Part 1

source:
http://fastcorner.wordpress.com/2010/09/09/fast-faq/
http://www.edwardrosten.com/work/fast.html


Great many thanks to Edward Rosten for creating a forum to ask questions and promptly answering them.

The below paragraphs are copied from the forum which I thought are helpful :)

MAN says:
In your code(ex. fast_9.cpp), the decision tree has already been conerted into C-code. It is a very long string of nested if-then-else statements. How can I make it?? What’s the machine learning?? Please explain the process to me.
  • edrosten says:
    It depends if you want to create code from a tree, or make a whole new tree.
    Either way, you will want the FAST-ER source code.
    Pre-made trees which are not yet converted into code are available. Look for the file “FAST_trees.tar.bz2″ and untar it. The raw (unconverted) trees have the suffix .ftree. They contain trees in a language neutral format which should be easy to convert into source code using your language of choice. I’ve used shell scripts to perform the conversion. See fast_tree_to_CXX as an example.
    If you want to make your own trees from scratch, then you need to use the learn_fast_tree program. That takes as input a list of extracted features. You can generate all features for fast-N using fast_N_featuresto ensure your tree is exact. I have a program to extract fast-N features from images, but it isn’t currently in the distribution. Let me know if you need it and I will add it.
    Let me know if you need any more information.


    Vivek Anand says:
    You consider points which are approximately 3 pixels away from the point being tested to find out if its a feature or not. Will the radius of 3pixels work for all image sizes or we need to change it based on the image size.
    • edrosten says:
      The radius you need depends on the scale of the features, rather than the size of the image. If the features are very blurry, then you will need a bigger ring. The easiest and most efficient way to do this is to subsample the image, e.g. by taking 2×2 squares, and averaging the pixels inside to make a single output pixel.

Discussion on NARF features...

Thanks Bastian Steder for your nice replies to my pestering emails :)


Paper about NARF:
- B. Steder, R. B. Rusu, K. Konolige, and W. Burgard. 

Point Feature Extraction on 3D Range Scans Taking into Account Object Boundaries


or his PhD-Thesis:



- Bastian Steder. Feature-Based 3D Perception for Mobile Robots.



You can find those on his homepage:


The rest of the article is the reply I got from Bastian to my questions.

> I was going through the paper on NARF key points and it uses range image

> representation of pointcloud in key point extraction.

> 1. Does it mean that NARF key points can only be extracted on point
> clouds that can be converted to range images?

> 2. Can Un-organized point clouds be converted to range images?

> 3. Is the statement below true,

> "The range image that is visualized in pcl tutorial uses spherical
> projection for better view only. Essentially range image is depth image
> where pixels positions contain depth information. The range image can be
> viewed as normal planar image also but the points not in 180 field of
> view cannot be displayed."



it is correct that NARFs can only be extracted from range images. But a 
range image can be created both from organized and unorganized point 
clouds, as long as you can provide a viewpoint.
I am not sure if I understand the statement in 3. correctly, but there 
are two RangeImage classes. RangeImage uses spherical projections and 
can therefore represent full 360deg views. RangeImagePlanar uses a 
pin-hole camera model with a projection plane and can therefore only 
represent scenes with less than 180deg field of view.



>Goal: One thing I found is that NARF features are not that stable on
> Kinect data, so I was wondering if I can make them more stable. 


I noticed that too, but was not able to fix it in the original NARF method. The problem is that the borders ob objects in the Kinect-Data 'flicker' a lot and are not stable. This leads to big differences in keypoint positions between frames.
I am working on a new keypoint method that should work better in this context, but it is not finished and therefore also not published yet.


>  I guess that you had lidar point cloud in mind
> while developing NARF's.  But sometimes there is a possibility of point
> clouds from lidar like velodyne being very sparse.
> 1.So how does sparse point clouds effect the NARF's stability and  no.
> of features extracted?


Sparsity itself is typically not a problem. I used the method on Velodyne data before. The only thing you have to keep in mind is that you need to choose a large enough support size that there are actually enough points for the estimation process.


> 2. Could you give any suggestion on improving the algorithm and any
> drawbacks which you found on the current method from your experiences


As mentioned above, the method does not work so well on Kinect-like sensors, mainly because of the unstable borders. If you somehow make the border positions more stable that might help, but I do not have a good idea how to do that.


> 3. Currently I am in the beginning stage of understanding your
> algorithm. I was not able to find the math behind converting point cloud
> to range image. Could you please point where can I get theory on
> converting point cloud to range image.


Converting a point cloud into a range image is typically done using a simple z-buffer. This means that all points are converted into pixel coordinates and ranges and then every pixel is the minimum of the ranges that fall into it.


>  4.I went through the tutorial on creating range image from point cloud
> in PCL. We have to set a parameter "angular resolution". Does this
> parameter change the number the number of pixels in the range image
> generated from the point cloud ? If yes, then how can I get the range
> image with all the information I have in point cloud and not missing any
> data? Will setting a small value interpolate any data?



Yes, this conversion step does not keep the original number of pixels. This is a drawback you have to live with if you want to use this method. Range images naturally encode a certain perspective and therefore not encode 3D structures that are occluded from other structures given the perspective.
Yes, the angular resoultion parameter changes the number of pixels. The method in PCL includes some interpolation, but if the resolution of your cloud is much lower than the one of the range image you will have empty pixels in there.



> 5. One more pre-requisite is "view-point" information to generate range
> image from point cloud. If I am planning to use NARF's for pose
> estimation by feature matching, then I actually dont know the current
> "pose" or "view point" and that is what I am trying to estimate in pose
> estimation process. So does it mean that NARF's cannot be used for
> pose-estimation by feature matching. Please correct me, if I am missing
> something fundamental :(



Pose estimation is basically what I use NARFs for. :-)
The viewpoint I am talking about is the sensor position in the coordinate frame of your scan. If the scan is in the coordinate frame of the sensor (which is often the case) this viewpoint is typically just (0,0,0). If you converted the scan into the coordinate system of your robot, e.g., it is the position of the sensor on the robot.
But you can also choose another viewpoint to simulate another perspective.
I would propose to just try the method on one of your scans and visualize the resulting range image.


Hope that this article is useful :-)



Wednesday, September 25, 2013

Difference between Organized and Unorganized Point cloud

source: http://stackoverflow.com/questions/18994680/can-anyone-explain-the-difference-between-organized-and-unorganized-point-cloud

Question: What is the difference between organized point cloud and an unorganized one?
Am I right in thinking that a point cloud is a column of [x,y,z] values? Than what is the difference in the structure of the point cloud if its organized or unorganized ?
How does it affect the later processing steps like filtering, normal estimation, registration etc ?
Answer:See the notes on this page on HEIGHT and WIDTH of a point cloud: Point Cloud File Format
An organised point cloud is organized as a 2D array of points with the same properties you'd expect if the points were obtained from a projective camera, like the Kinect, DepthSense or SwissRanger. In PCL the points array of a point cloud is actually a 2D array but one of those dimensions is only used for representing organised point clouds.
In both organised and unorganised point clouds, all of X Y and Z are provided for each point, but the memory layout of organised point clouds is that of a 2D array. The memory layout of the points then is closely related to the spatial layout as represented by these XYZ values.
Algorithms that work on unorganised point clouds will generally work on organised point clouds (since the 2D array of points is packed and can be interpreted as a 1D array) but specialised algorithms can be designed to work on organized point clouds. An example is the use of the organized property of a point cloud to speed up the process of calculating normals: Integral Images Normal Estimation Tutorial


Thursday, September 12, 2013

Good Will Hunting, a must watch !!!

Yesterday night, I wanted to start my work after just seeing some movie scenes from Youtube. So i started to think of some movie names that come to my mind and it was Good Will Hunting. Having seen this movie before, i wanted to see some good scenes from this. The first link on the youtube was the scene in which , the Will Hunting(Matt Damon) will be asked by Robin Williams what he wanna do with his life ? He gives some bull shit answer by saying that he wants to become a shepherd. Robin William tells him that he doesnt even know and you have some stupid answer for every question.



This scene was so good that temped me to see the full movie. So I started to see the full movie.

I guess that this movie is based on a real life story. Matt Damon is a prodigy who is exceptionally good in math. But his childhood and many incidents from his life make him feel abandoned by the ones who should have loved him. Thus he hangs out with workers and not "intellectual"  people who work for a construction company. Robin says in the movie that he does that becoz he finds trust and loyalty in those people.

Matt Damon used to work as a janitor at MIT and a Prof. Gerald identifies this genius and wants him to go to great places and solve cutting edge problems. The problem with Matt is that he engages in lot of street fights but gets rid of them from the court arguing some previous incidents from history orcivilian rights from the constitution of United states of America. Prof. Gerald gets Matt from prison on two conditions that he ll do math and goes to a therapist.

Prof. Gerald finds Robin who is a psychologist by his profession and the whole movie is about how Robin helps Matt to make good choices in his life. 

Tuesday, September 10, 2013