Hand gesture recognition embedded system can be used as an interfacing medium between the computer and human using different hand gestures in order to control the computer. In this proposed system, a real time vision based hand gesture interaction prototype which depends upon finger gestures using color markers is designed. The objective is to develop an embedded system by which one can communicate with any digital device with less hardware requirements and using an external camera to capture the gestures. To avoid the limitations of PC an embedded system consisting of Raspberry Pi which is a Linux based platform is used to identify different color markers on the fingers and when the mouse emulation is started, the software tracks those markers using the camera. The main aim is to create a framework with low cost and effective gesture interpretation system which uses computer vision to analyze different sets of gestures or actions done using the human fingers.