-
-
Notifications
You must be signed in to change notification settings - Fork 35.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SkeletonUtils .retarget()
and .retargetClip()
error with documented SkeletonHelper
params
#25751
Comments
The retarget methods were added long time ago in context of Sea3D. When the related demos have been removed, both methods ended up without code examples. |
I'm willing to make contributions here. I'm thinking of:
|
That would be awesome! Regarding #25763, I really want to remove the |
@mattrossman Thanks for the initiative, if you download the r108 version and browse the SEA3D BVH examples it is possible to see some examples, I don't know exactly what may have changed since then but I think they will help. |
This is a good point to consider. I wonder what kind of use cases others have. Personally, my use case for retargeting is similar to the SEA3D example. My source is a Mixamo skeletal animation (without a mesh) and target is a skinned character mesh. So the most ergonomic combo for me is Skeleton source and SkinnedMesh or Skeleton target. One benefit of the Skeleton type is if users are working with a SkinnedMesh, it's trivial for them to access the Skeleton via the That being said I need to take a deeper look at the retarget implementation. I don't fully understand all the three.js/examples/jsm/utils/SkeletonUtils.js Lines 63 to 79 in a55464e
|
Notes as I try understanding the usage of the existing options for documentation and code cleanup. Options for
Options for
In the Sea3D examples code: I see usage of In practice, I can get expected retargeting results using only |
|
@mattrossman I tried the same with GLB models (soldier & xbot), but I can't get the scale working for those. |
@timbotimbo Thank you for the example! I had similar scaling difficulties when trying the soldier & xbot models. That would be a good litmus test for the retargeting implementation, to make sure it can work for those assets as expected. I have used retargeting on my own GLB characters and Mixamo animations in a different project successfully so there must be a way to get those working. @sunag Ok, I see how I don't fully understand
Therefore, I would expect the target character's proportions (defined by the positions of bones) to remain untouched by retargeting (i.e. "preserved") without needing to If however, Dalshin is the source animation, then I could see how a way to opt-in to transferring positions of bones other than the hip is valuable. However, transferring world positions 1:1 isn't what I'd want, because this would overwrite my target character's body proportions. Instead I suppose I'd want to apply the position deltas from the source animation (e.g. deltas on Dalshin's arm positions). It sounds like that'd be a responsibility of |
Here is the result I get when retargeting that pirouette BVH animation to the Soldier model: I get the same result with One difficulty in debugging this sort of thing is that Normally I'd add a Here's a copy that shows the axes for the source and target bones: You can see that the models use different bone orientations in their T-pose (the Soldier has the +Y axis pointing along the bone chain). Maybe this could contribute to the bones getting twisted up. I don't remember if this implementation retargets orientations relative to the bind pose or not. Not sure why their position and scale is being affected though. The BVH clip doesn't contain any scale tracks for other bones. Besides, I feel that by default retargeting shouldn't mess with scale or position of non-hip bones. |
@alankent My main finding after lots of trial and error was that Three's example retargeting function is very sensitive to things like differences in local bone transforms, difference in bind pose bone orientations, timing of matrix updates. I ended up writing my own retargeting logic for my project rather than try to work around it. I assume models are bound from a T-pose in my solution and rely more on world space transforms to account for rig differences. I was hoping to translate some of these findings into a new By comparison, I found Unreal Engine has a much more sophisticated animation system that provides a suite of animation primitives that make retargeting easier. I know @sketchpunklabs is doing some interesting work in a similar regard with ossos, that might be a library to check out in the meantime for your retargeting needs. |
Thanks for the reference. I am having a look at ossos now. I have some concerns around how easy it will be to merge into my existing app, but maybe that is just lack of familiarity. He has lots of YouTube videos to back it up, has IK support, and spring bones built in. So lots of nice stuff. Time to start exploring the demos that come with it! |
In case useful to anyone else on the same journey, I came across https://pixiv.github.io/three-vrm/packages/three-vrm/examples/humanoidAnimation/loadMixamoAnimation.js which loads a mixamo.com animation clip on a VRM character, doing retargeting. It is hard coded to particular bone names, but very useful as reference code. |
Hello folks, I started a thread about this in the forum before I saw this one, showing some live examples of transform issues: https://discourse.threejs.org/t/fixing-skeletonutils-retarget-and-retargetclip-functions/65149 The main problem to work out is transform handling (fixing parameters and .skeleton hack is comparatively easier). |
Description
SkeletonUtils Docs indicate that
.retarget()
and.retargetClip()
expect twoSkeletonHelper
as the source and target.When I call
retarget()
with these arguments, I get an error:The traceback points here:
three.js/examples/jsm/utils/SkeletonUtils.js
Lines 31 to 32 in a55464e
source.isObject3D
readstrue
for a SkeletonHelper, so it then tries to readsource.skeleton.bones
which is invalid for SkeletonHelper. Instead, bones are stored onSkeletonHelper.bones
.I've tried to piece together what type(s) retarget is meant to accept. Taking a look at that
getBones
function:three.js/examples/jsm/utils/SkeletonUtils.js
Lines 485 to 489 in a55464e
My guess is that retarget is written to accept:
SkinnedMesh
(according tosource.skeleton.bones
)Bone[]
(according toArray.isArray
ingetBones
)Skeleton
(according toskeleton.bones
ingetBones
)I tried passing my
SkeletonHelper.bones
array instead, but that didn't seem to perform retargeting. Then I tried passing aSkinnedMesh
and it sort of worked, but the retargeted pose was wrong. Then I tried passing aSkeleton
and that worked as expected.I notice in the Fiddle shared in #25288, they had to explicitly assign a skeleton to
skeletonHelper.skeleton
.I get the impression that
retarget()
andretargetClip()
are generally outdated, either in implementation or documentation. As @sunag mentioned in #25589 this could use an official example. I'd also like there to be some explanation of theoptions
object.Reproduction steps
helperSource
for ithelperTarget
for itSkeletonUtils.retarget(helperTarget, helperSource, {})
Code
Live example
https://jsfiddle.net/mattrossman/h2473jf0/4/
Screenshots
No response
Version
r151
Device
Desktop
Browser
Chrome
OS
MacOS
The text was updated successfully, but these errors were encountered: