« View All Projects
3d modelling off video
rouncer - Posted on February 7, 2015 at 12:38 am.
I actually got the idea, from watching Zdeneks predator tracker (he has an attractive toy himself), then thought 'what if i connect the dots together?' :)
I realize the small response I got from this, so heres my algorythm
if anyone wants to know how I do it (may change in very near future)::
It uses texture to track, so if there isnt enough texture there, itll pull too many even scores, but most things have texture, i find. It has a little trouble with corner appearance/spawning too.
1. get a binary descriptor of every pixel on the screen, like ORB or BRISK. 256 bits is what I use. (8 unsigned ints... its pretty big.) these are binary rgb compares, so its aware of colour differences.
I do it in a disc, with less density on the outside, so in the future you could add spin invarience, which I havent worked out yet.
2. get the derivative of the screen.
3.OPFLOW part- look around every pixel, and scoring using nearness of id, and amplitude of the derivative map (gets you nicer mesh) minus the average of all the scores, write this score difference on every pixel, and the position where its going to go. Minusing the average will descore the id tracks that go in too many different directions equal scored. I imagine if you look 2 frames ahead it gets rid of even more shit points.
4. non maximally suppress the score, plus threshold complete shit scores that are too low to be good for anything, and you have the new corners that appeared, any corner/tracker that was already there stays there and just keeps tracking through till it accidentally becomes something else, when its surface goes off the screen, and I havent done it yet, but you can actually choose a nonmax suppress radius using the average density of the derivative map, so corners generate denser the further they are away.
That gets you your points, then you realtime tesselate.
Then you see why its in the neural network section.
Actually getting the 3d models: you have to remember common connections (machine learning), you could take so many 2d vectors and decide apon a fitting 3d distance, using some reductive algebra, (albiet with a known perspective matrix). Then inflate the model made of springs. Some intercorrelation hierarchy similar to htm (look up numenta) is kinda like what you want.
Theres more, I wont stop until I have my robot villain, but thats enough for now. :)