Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New "holistic" Mediapipe tracker: Body + Face + Hands #225

Open
golanlevin opened this issue Oct 28, 2024 · 2 comments
Open

New "holistic" Mediapipe tracker: Body + Face + Hands #225

golanlevin opened this issue Oct 28, 2024 · 2 comments

Comments

@golanlevin
Copy link

Google advertises a "holistic" landmark tracker that allows simultaneous tracking of Body, Face and Hands:
https://ai.google.dev/edge/mediapipe/solutions/vision/holistic_landmarker

The task outputs a total of 543 landmarks (33 pose landmarks, 468 face landmarks, and 21 hand landmarks per hand) in real-time.

@shiffman
Copy link
Member

Thank you @golanlevin! @ziyuan-linn I wonder if this could help with some of the issues we came across related to #134 and #69?

@ziyuan-linn
Copy link
Member

This looks promising! We can technically already do this in ml5 by creating an instance of FaceMesh, BodyPose, and HandPose each. However, having a holistic tracker that encapsulates everything could be more convenient. I can see this applied to avatar/VTuber projects.

I wonder if TFJS will have an API for this tracker. Will definitely look out for this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants