Robotic Arm finding a target and avoiding obstacles (Computer Vision)
This project was developed during my studies as my final exam work.
In my diploma work I developed a semi - automated robotic system, composed of two cameras attached to a metal construction, objects and, of course, a robotic manipulator, controlled by a PC.
In the project, I use cheap (but wide angle) cameras, cheap robotic arm and an average PC, because the final idea is to create a real - time system that can work on an embedded platform. That's why the image should be with as lower resolution as possible.
Also, I try to minimize the cost just for the sake of lower cost systems :)
Summary:
The Image Processing Algorithm should recognize the object that the robotic arm needs to catch, as well as all of the obstacles present.
The Pseudo3D generation algorithm creates a pseudo 3D map based on the images that were pre-processed and morphologically processed and then segmented by the two opposing cameras.
It utilizes a 2D homographic approach based on the known location of the chess field on the edge of the metal construction and the position of the cameras, as well as the points on the chess field, detected by the two images, by using an SVD based algorithm.
Then, by utilizing a numerical algorithm for calculation of Inverse Kinematics, based on the Levenberg - Marquardt algorithm, which just tries to iteratively lower the error between the Forward Kinematic solution of the initial guess and the actual value, by reevaluating it with a Jacobian Matrix, the control algorithm should find a suitable solution for the angles of each of the robotic joints.
When a solution is generated, the robotic arm tries to evade all obstacles in order to avoid collision and reach the target. The collision avoidance algorithm just creates an intersection between the geometric model of the robot (primitives connected in the manner so it would represent the robotic arm in a space filled with zeros where there are no parts of the geometric representation of the robot and ones where there are such) and the pseudo - 3D map of the obstacles (same space as that of the robot model, but with ones where there are obstacles instead).
If the intersect of the ones of the robot and those of the obstacle (logical 'and' between the two spaces) coincide in the same point, which means if the sum of the entire resultant space is different than zero, then the geometrical model of the robot and the obstacles are on the same position. That means that if the robot moves to that position, it would collide with an object, so that point is discarded as not viable for movement and the path finder algorithm searches for another point that minimizes the distance from the previous point and the final position, with some maximal predefined step for every joint separately.
When the path is generated, another algorithm smooths out the path, by discarding some of the middle points that are not needed and then the robotic arm moves from point to point, until it reaches the target, then it grabs it and take it on the same route back, so it can toss it on its start position.
The Dynamics of the system are not controlled by any software but the basic software of the robotic arm, although some limitations on the speed, acceleration, moments and inertial moments were imposed, as well as the known trapezoid curve for the speed of the servo motors. The servo motors are controlled by a PWM signal, generated by the controller which is connected to the PC.
Almost the entire system works within Matlab. It is constructed of modular functions and it's not compiled, so an increase in speed is expected if a compiled version of the code is generated. Some parts of the control are done in a C++ program and the lowest level control is done in UsborNexus (the robotic arm sofware). There is a developed communication program between C++ programs and UsborNexus, that is available on the RobixRascal site, which I use as well. Matlab and the program written in C++ communicate by writing (Matlab side) and reading (C++ side) the needed info in/from a text file.
If you have any questions, you can post them in the comment section, or PM me.
Cheers,
Bojan
If you are bored of my project, you can check out my more musical YouTube channel:
/ boharhija
P.S. Sorry for the Non-English comments in the video :)