we are TEAM FANATICS of Hacknovate 5
Arpit Kumar
Anirudh Singh Tomar
Khushi Agarwal
Anshul Sharma
Problem Statement:
The increase of the influence of the entertainment and cinema industry, there is an exponential demand of Digital artists (VFX, 3D modelling, 3d animation etc). This comes with a need of motion capturing for making digital effects.
Motion Capturing techniques demand a lot of capital and resources in form of ‘MO-CAP’ suits, and heavy grade equipment, that beginner enthusiasts can not afford plus they are too complicated to use.
Proposed Solution:
To create a program that would track the body movements of a human and map it into a skeleton rig, that then can be used for various purposes, using just a camera.
The methodology used here is to use Computer Vision to enable the computer to read the video inputs from camera of the by uploading video file from the storage of the device manually, and Game development in Unity3D to integrate the 3D animation with the Computer Vision
Advantages:
Video Input Processing:
Utilizes OpenCV for capturing and processing video frames from a camera or device.
Integrates Mediapipe for advanced computer vision tasks such as hand tracking, pose estimation, or keypoint detection.
Unity3D Integration:
Transforms processed video input into dynamic 3D animations within Unity3D.
Establishes a communication channel between the Python application and Unity3D for seamless data transmission.
Cross-Platform Compatibility:
Ensures that the application runs seamlessly across different operating systems, enhancing accessibility and usability.
Multiple ways of input:
By Uploading Video:
A user can make animation via uploading a video from a device into the program, that will be passed through the various functions of OpenCV with the help of Mediapipe.
Mediapipe will then identify the 32 landmarks in the video that represents a human body and will pass their coordinates into a Text (.txt) file which can be used in a pre made Unity 3D project.
Live tracking and Animating via Camera:
As OpenCV provides inputs via camera of user’s device we use this abilty to live track the movements of a person’s body to extract the coordinates of the 32 mediapipe landmarks and then deploying them to Unity3D.
Future Development Considerations:
Outlines possibilities for future enhancements or features that could be integrated into the application.
Identifies areas for potential expansion, such as additional computer vision functionalities or enhanced user customization options.
Video Input Processing:
Utilizes OpenCV for capturing and processing video frames from a camera or device.
Integrates Mediapipe for advanced computer vision tasks such as hand tracking, pose estimation, or keypoint detection.
Unity3D Integration:
Transforms processed video input into dynamic 3D animations within Unity3D.
Establishes a communication channel between the Python application and Unity3D for seamless data transmission.
Cross-Platform Compatibility:
Ensures that the application runs seamlessly across different operating systems, enhancing accessibility and usability.
Multiple ways of input:
By Uploading Video:
A user can make animation via uploading a video from a device into the program, that will be passed through the various functions of OpenCV with the help of Mediapipe.
Mediapipe will then identify the 32 landmarks in the video that represents a human body and will pass their coordinates into a Text (.txt) file which can be used in a pre made Unity 3D project.
Live tracking and Animating via Camera:
As OpenCV provides inputs via camera of user’s device we use this abilty to live track the movements of a person’s body to extract the coordinates of the 32 mediapipe landmarks and then deploying them to Unity3D.
Future Development Considerations:
Outlines possibilities for future enhancements or features that could be integrated into the application.
Identifies areas for potential expansion, such as additional computer vision functionalities or enhanced user customization options.