Motions are static

[shunlinlu]

Hi,

Are there any clues about the reason why the motions reproduced are static? I believe I followed the instructions very well. I also mentioned the issue here. #5 (comment)

Shunlin

[GuyTevet]

Did you train from scratch or run the pre-trained model?

[shunlinlu]

I trained exactly using this command:
python -m src.train.train --clip_text_losses cosine --clip_image_losses cosine --pose_rep rot6d \ --lambda_vel 100 --lambda_rc 100 --lambda_rcxyz 100 \ --jointstype vertices --batch_size 20 --num_frames 60 --num_layers 8 \ --lr 0.0001 --glob --translation --no-vertstrans --latent_dim 512 --num_epochs 100 --snapshot 10 \ --device <GPU DEVICE ID> \ --dataset amass \ --datapath ./data/amass_db/amass_30fps_db.pt \ --folder ./exps/my-paper-model

My config:
activation: gelu
archiname: transformer
batch_size: 20
clip_image_losses:

• cosine
  clip_lambda_ce: 1.0
  clip_lambda_cosine: 1.0
  clip_lambda_mse: 1.0
  clip_lambdas:
  image:
  cosine: 1.0
  text:
  cosine: 1.0
  clip_layers: 12
  clip_text_losses:
• cosine
  clip_training: ''
  cuda: true
  datapath: ./data/amass_db/amass_30fps_db.pt
  dataset: amass
  debug: false
  device: 2
  expname: exps
  folder: ./exps/my-paper-model
  glob: true
  glob_rot:
• 3.141592653589793
• 0
• 0
  jointstype: vertices
  lambda_rc: 100.0
  lambda_rcxyz: 100.0
  lambda_vel: 100.0
  lambda_velxyz: 1.0
  lambdas:
  rc: 100.0
  rcxyz: 100.0
  vel: 100.0
  latent_dim: 512
  losses:
• rc
• rcxyz
• vel
  lr: 0.0001
  max_len: -1
  min_len: -1
  modelname: motionclip_transformer_rc_rcxyz_vel
  modeltype: motionclip
  num_epochs: 100
  num_frames: 60
  num_layers: 8
  num_seq_max: -1
  only_60_classes: false
  pose_rep: rot6d
  sampling: conseq
  sampling_step: 1
  snapshot: 10
  translation: true
  use_action_cat_as_text_labels: false
  use_only_15_classes: false
  vertstrans: false

My training log:
Epoch 1, train losses: {'rc': 0.02620076398320851, 'rcxyz': 0.01882972872912115, 'vel': 0.008510271206660758, 'image_cosine': 0.3801915671144213, 'text_cosine': 0.39459805526430647, 'mixed_without_clip': 5.354076390228574, 'mixed_clip_only': 0.7747896215745381, 'mixed_with_clip': 6.128866004754626}
Epoch 2, train losses: {'rc': 0.012372248624229715, 'rcxyz': 0.009911740284102659, 'vel': 0.0019560201451115844, 'image_cosine': 0.26682336355249087, 'text_cosine': 0.3097360760683105, 'mixed_without_clip': 2.424000906471222, 'mixed_clip_only': 0.5765594391122697, 'mixed_with_clip': 3.0005603450631337}
Epoch 3, train losses: {'rc': 0.01438126724974681, 'rcxyz': 0.011858998792290333, 'vel': 0.0012753707487949185, 'image_cosine': 0.26419087604634345, 'text_cosine': 0.3131230684499892, 'mixed_without_clip': 2.7515636812126827, 'mixed_clip_only': 0.5773139437512745, 'mixed_with_clip': 3.3288776236867146}
Epoch 4, train losses: {'rc': 0.010049479806588756, 'rcxyz': 0.008472307770705175, 'vel': 0.0009547068816443373, 'image_cosine': 0.25010608498539244, 'text_cosine': 0.3031957212894682, 'mixed_without_clip': 1.947649446839378, 'mixed_clip_only': 0.5533018069607871, 'mixed_with_clip': 2.500951250110354}
Epoch 5, train losses: {'rc': 0.012309758084279204, 'rcxyz': 0.009951381777627541, 'vel': 0.000983913770150615, 'image_cosine': 0.25864426121115686, 'text_cosine': 0.3105136335132614, 'mixed_without_clip': 2.324505362151161, 'mixed_clip_only': 0.5691578957296554, 'mixed_with_clip': 2.8936632571712373}
Epoch 6, train losses: {'rc': 0.010788808287017875, 'rcxyz': 0.00912739260377185, 'vel': 0.0008097647580259022, 'image_cosine': 0.25222905785554933, 'text_cosine': 0.3025378231964414, 'mixed_without_clip': 2.072596567112302, 'mixed_clip_only': 0.5547668808509433, 'mixed_with_clip': 2.627363448767435}
Epoch 7, train losses: {'rc': 0.010944190405545727, 'rcxyz': 0.009124071917326618, 'vel': 0.0007731874674347983, 'image_cosine': 0.2579466398509722, 'text_cosine': 0.30359214565583637, 'mixed_without_clip': 2.0841449773500837, 'mixed_clip_only': 0.5615387858852507, 'mixed_with_clip': 2.645683768533525}
Epoch 8, train losses: {'rc': 0.01002939479565987, 'rcxyz': 0.008382033540438564, 'vel': 0.0006471913365188927, 'image_cosine': 0.2473705129845748, 'text_cosine': 0.2966090800979781, 'mixed_without_clip': 1.905861966997858, 'mixed_clip_only': 0.5439795926922844, 'mixed_with_clip': 2.4498415596901424}
Epoch 9, train losses: {'rc': 0.00985884934456812, 'rcxyz': 0.008497150910146061, 'vel': 0.0005960066617089545, 'image_cosine': 0.24545926575859386, 'text_cosine': 0.29492939866724466, 'mixed_without_clip': 1.8952006936073302, 'mixed_clip_only': 0.540388665407423, 'mixed_with_clip': 2.4355893593931954}
Epoch 10, train losses: {'rc': 0.00932206856825995, 'rcxyz': 0.008018700713439594, 'vel': 0.0005471890823664291, 'image_cosine': 0.24118016013313853, 'text_cosine': 0.2919261276958481, 'mixed_without_clip': 1.78879583487435, 'mixed_clip_only': 0.5331062885267394, 'mixed_with_clip': 2.3219021224786367}
Epoch 11, train losses: {'rc': 0.010844885658413645, 'rcxyz': 0.009431668519160696, 'vel': 0.0005589814502782824, 'image_cosine': 0.2446671199467447, 'text_cosine': 0.2924394416903693, 'mixed_without_clip': 2.0835535618047865, 'mixed_clip_only': 0.5371065623230404, 'mixed_with_clip': 2.620660128669133}
Epoch 12, train losses: {'rc': 0.011309855128268874, 'rcxyz': 0.009309632769445814, 'vel': 0.0005800035251590926, 'image_cosine': 0.24999903319847017, 'text_cosine': 0.29541919288181123, 'mixed_without_clip': 2.1199491448818693, 'mixed_clip_only': 0.545418226103934, 'mixed_with_clip': 2.6653673718846034}
Epoch 13, train losses: {'rc': 0.010616542998745683, 'rcxyz': 0.00855470813666692, 'vel': 0.0005794019035492388, 'image_cosine': 0.251361709419224, 'text_cosine': 0.29537956376397423, 'mixed_without_clip': 1.975065303747616, 'mixed_clip_only': 0.5467412733842456, 'mixed_with_clip': 2.5218065743408506}
Epoch 14, train losses: {'rc': 0.010216282935635675, 'rcxyz': 0.008265728044641455, 'vel': 0.0005310366222854807, 'image_cosine': 0.2466369504256854, 'text_cosine': 0.2917394321589243, 'mixed_without_clip': 1.901304759439968, 'mixed_clip_only': 0.5383763831759256, 'mixed_with_clip': 2.4396811425685883}
Epoch 15, train losses: {'rc': 0.009418412171570318, 'rcxyz': 0.007853363052247063, 'vel': 0.0004915713894343565, 'image_cosine': 0.23818033216964632, 'text_cosine': 0.2893190181208035, 'mixed_without_clip': 1.7763346582178086, 'mixed_clip_only': 0.5274993503850604, 'mixed_with_clip': 2.303834011535796}
Epoch 16, train losses: {'rc': 0.009053210758175405, 'rcxyz': 0.007652931457148894, 'vel': 0.0004700503213190666, 'image_cosine': 0.23559115064286051, 'text_cosine': 0.2874197776591967, 'mixed_without_clip': 1.717619253387527, 'mixed_clip_only': 0.523010928479452, 'mixed_with_clip': 2.2406301803059048}
Epoch 17, train losses: {'rc': 0.008684544647038574, 'rcxyz': 0.007410312666269463, 'vel': 0.00045194468705356863, 'image_cosine': 0.23240918243924777, 'text_cosine': 0.28678242831476153, 'mixed_without_clip': 1.6546802019789106, 'mixed_clip_only': 0.5191916117119411, 'mixed_with_clip': 2.173871811798641}
Epoch 18, train losses: {'rc': 0.009129220617949844, 'rcxyz': 0.007839403362641674, 'vel': 0.0004559715309398367, 'image_cosine': 0.2361739742495711, 'text_cosine': 0.2873620876717189, 'mixed_without_clip': 1.7424595506891372, 'mixed_clip_only': 0.5235360625953901, 'mixed_with_clip': 2.2659956137339274}
Epoch 19, train losses: {'rc': 0.008977558844101926, 'rcxyz': 0.007756827787740068, 'vel': 0.0004512808089318227, 'image_cosine': 0.23406769833157934, 'text_cosine': 0.28713535839130006, 'mixed_without_clip': 1.7185667442423957, 'mixed_clip_only': 0.5212030582957797, 'mixed_with_clip': 2.239769802301649}
Epoch 20, train losses: {'rc': 0.008556980689957975, 'rcxyz': 0.007377977741120886, 'vel': 0.0004376188430118389, 'image_cosine': 0.23078059384983685, 'text_cosine': 0.28627214476702706, 'mixed_without_clip': 1.63725772821714, 'mixed_clip_only': 0.5170527391963535, 'mixed_with_clip': 2.154310468643431}
Epoch 21, train losses: {'rc': 0.008600754817829482, 'rcxyz': 0.007475834317497968, 'vel': 0.00043818613522157577, 'image_cosine': 0.23137261739798953, 'text_cosine': 0.28594269345677087, 'mixed_without_clip': 1.651477527949545, 'mixed_clip_only': 0.5173153118481711, 'mixed_with_clip': 2.168792842683338}
Epoch 22, train losses: {'rc': 0.008859123486936802, 'rcxyz': 0.007708393847237208, 'vel': 0.0004416827480391865, 'image_cosine': 0.2320342497929694, 'text_cosine': 0.2869580871292523, 'mixed_without_clip': 1.7009200116944692, 'mixed_clip_only': 0.5189923369458743, 'mixed_with_clip': 2.219912349704712}
Epoch 23, train losses: {'rc': 0.00999655429872551, 'rcxyz': 0.00844392071234151, 'vel': 0.0004818316706592986, 'image_cosine': 0.2414493879155507, 'text_cosine': 0.2889603284616319, 'mixed_without_clip': 1.8922306691843367, 'mixed_clip_only': 0.5304097163298773, 'mixed_with_clip': 2.422640383811224}
Epoch 24, train losses: {'rc': 0.008995470699174182, 'rcxyz': 0.0074786920835899695, 'vel': 0.00046201086467287193, 'image_cosine': 0.23671237412426208, 'text_cosine': 0.2863052501091881, 'mixed_without_clip': 1.6936173658995402, 'mixed_clip_only': 0.5230176252505135, 'mixed_with_clip': 2.216634991906938}
Epoch 25, train losses: {'rc': 0.008269063537452548, 'rcxyz': 0.007114326457367352, 'vel': 0.0004319489693328265, 'image_cosine': 0.22851694187238103, 'text_cosine': 0.2845576810931403, 'mixed_without_clip': 1.581533895693128, 'mixed_clip_only': 0.513074623734232, 'mixed_with_clip': 2.094608520609992}
Epoch 26, train losses: {'rc': 0.00850738804780006, 'rcxyz': 0.007393334449882368, 'vel': 0.00044336353649481364, 'image_cosine': 0.23291656424601873, 'text_cosine': 0.2850326070236781, 'mixed_without_clip': 1.6344086034430398, 'mixed_clip_only': 0.5179491717900548, 'mixed_with_clip': 2.152357772796873}
Epoch 27, train losses: {'rc': 0.008621744522898798, 'rcxyz': 0.007470166125142621, 'vel': 0.00043627465831435524, 'image_cosine': 0.22993354215508416, 'text_cosine': 0.28508054824575546, 'mixed_without_clip': 1.6528185289530528, 'mixed_clip_only': 0.5150140913942504, 'mixed_with_clip': 2.1678326181949132}
Epoch 28, train losses: {'rc': 0.008394231188244053, 'rcxyz': 0.007326283880383782, 'vel': 0.0004311498561051733, 'image_cosine': 0.22825607574648327, 'text_cosine': 0.28471503889276867, 'mixed_without_clip': 1.6151664905131808, 'mixed_clip_only': 0.5129711144263782, 'mixed_with_clip': 2.1281376043955484}
Epoch 29, train losses: {'rc': 0.008324702772393172, 'rcxyz': 0.00731181979478736, 'vel': 0.00042680368067175595, 'image_cosine': 0.2281818315268509, 'text_cosine': 0.2835797024388162, 'mixed_without_clip': 1.606332625377746, 'mixed_clip_only': 0.5117615340957566, 'mixed_with_clip': 2.118094157415723}
Epoch 30, train losses: {'rc': 0.00817214517063269, 'rcxyz': 0.007256142667768198, 'vel': 0.00042509159761793273, 'image_cosine': 0.228034327049104, 'text_cosine': 0.2830370445336614, 'mixed_without_clip': 1.585337944257827, 'mixed_clip_only': 0.5110713720321656, 'mixed_with_clip': 2.096409314303171}
Epoch 31, train losses: {'rc': 0.00795686372762753, 'rcxyz': 0.007078914664730075, 'vel': 0.0004194752502140574, 'image_cosine': 0.2263988744171839, 'text_cosine': 0.2823998045826715, 'mixed_without_clip': 1.5455253650271703, 'mixed_clip_only': 0.508798678834287, 'mixed_with_clip': 2.0543240441216364}
Epoch 32, train losses: {'rc': 0.00821673691361433, 'rcxyz': 0.0072902307984241774, 'vel': 0.00043596370958275945, 'image_cosine': 0.2303192163034091, 'text_cosine': 0.28301544844631166, 'mixed_without_clip': 1.5942931412704406, 'mixed_clip_only': 0.513334665128163, 'mixed_with_clip': 2.1076278075339303}
Epoch 33, train losses: {'rc': 0.008049134417097011, 'rcxyz': 0.007192168339887368, 'vel': 0.00042496927431091624, 'image_cosine': 0.2280375365936567, 'text_cosine': 0.2823740147172459, 'mixed_without_clip': 1.5666272033774664, 'mixed_clip_only': 0.51041155173665, 'mixed_with_clip': 2.0770387553033376}
Epoch 34, train losses: {'rc': 0.008172893374123507, 'rcxyz': 0.0073247662872918656, 'vel': 0.0004229371310516615, 'image_cosine': 0.22703422145947577, 'text_cosine': 0.28268466260698105, 'mixed_without_clip': 1.5920596804883744, 'mixed_clip_only': 0.5097188847878623, 'mixed_with_clip': 2.1017785649451}
Epoch 35, train losses: {'rc': 0.0077749546181174025, 'rcxyz': 0.006982023417136617, 'vel': 0.0004136666968522135, 'image_cosine': 0.22465445361440145, 'text_cosine': 0.28176188090490917, 'mixed_without_clip': 1.5170644742155832, 'mixed_clip_only': 0.5064163342591316, 'mixed_with_clip': 2.023480805328914}
Epoch 36, train losses: {'rc': 0.007721087000968438, 'rcxyz': 0.006944041525656801, 'vel': 0.000411024940683366, 'image_cosine': 0.22491278287673752, 'text_cosine': 0.28117282977179875, 'mixed_without_clip': 1.5076153496901195, 'mixed_clip_only': 0.5060856124711415, 'mixed_with_clip': 2.0137009639588612}
Epoch 37, train losses: {'rc': 0.007620367143924038, 'rcxyz': 0.006930062091828043, 'vel': 0.00040351909217803503, 'image_cosine': 0.2231565363468632, 'text_cosine': 0.28094067095764097, 'mixed_without_clip': 1.4953948338353444, 'mixed_clip_only': 0.5040972081205202, 'mixed_with_clip': 1.9994920382424006}
Epoch 38, train losses: {'rc': 0.007611091455466868, 'rcxyz': 0.006911257251224939, 'vel': 0.00040479569623361357, 'image_cosine': 0.2237358618350256, 'text_cosine': 0.2805084973810211, 'mixed_without_clip': 1.4927144444651075, 'mixed_clip_only': 0.5042443594762257, 'mixed_with_clip': 1.9969588038467225}
Epoch 39, train losses: {'rc': 0.007679190373026012, 'rcxyz': 0.006944918654127313, 'vel': 0.0004111849958708118, 'image_cosine': 0.2257399596273899, 'text_cosine': 0.2808597541044629, 'mixed_without_clip': 1.5035294010525657, 'mixed_clip_only': 0.5065997140275107, 'mixed_with_clip': 2.010129116357319}
Epoch 40, train losses: {'rc': 0.007611540539027561, 'rcxyz': 0.006918730034071598, 'vel': 0.0004079689814211128, 'image_cosine': 0.2246409809187291, 'text_cosine': 0.2803331791171952, 'mixed_without_clip': 1.4938239532803732, 'mixed_clip_only': 0.504974159172603, 'mixed_with_clip': 1.9987981109392075}
Epoch 41, train losses: {'rc': 0.007979391531516162, 'rcxyz': 0.007163044478234258, 'vel': 0.0004176556842543909, 'image_cosine': 0.22638721673025025, 'text_cosine': 0.2812280723972926, 'mixed_without_clip': 1.5560091685681117, 'mixed_clip_only': 0.5076152897070325, 'mixed_with_clip': 2.0636244578020912}
Epoch 42, train losses: {'rc': 0.007918199923421656, 'rcxyz': 0.00706648764141377, 'vel': 0.00041762352449863793, 'image_cosine': 0.22659562408687575, 'text_cosine': 0.28075845381097186, 'mixed_without_clip': 1.5402311104157613, 'mixed_clip_only': 0.5073540782644635, 'mixed_with_clip': 2.047585189342499}
Epoch 43, train losses: {'rc': 0.007719839558315774, 'rcxyz': 0.00695605656642112, 'vel': 0.0004098756818007493, 'image_cosine': 0.2241253022518423, 'text_cosine': 0.28081548072042917, 'mixed_without_clip': 1.5085771760770252, 'mixed_clip_only': 0.5049407832679295, 'mixed_with_clip': 2.01351795849346}
Epoch 44, train losses: {'rc': 0.007466296569281627, 'rcxyz': 0.006768664982103344, 'vel': 0.00040142996391902367, 'image_cosine': 0.22187424329301667, 'text_cosine': 0.28003144902842386, 'mixed_without_clip': 1.463639152523071, 'mixed_clip_only': 0.5019056928299722, 'mixed_with_clip': 1.965544845565917}
Epoch 45, train losses: {'rc': 0.00747469283958217, 'rcxyz': 0.006836685911744892, 'vel': 0.0004001293393634518, 'image_cosine': 0.2219928290517557, 'text_cosine': 0.2798349039422141, 'mixed_without_clip': 1.4711508091953065, 'mixed_clip_only': 0.5018277331713646, 'mixed_with_clip': 1.9729785446136716}
Epoch 46, train losses: {'rc': 0.0077142196372964435, 'rcxyz': 0.0069983563295358585, 'vel': 0.0004099648399879077, 'image_cosine': 0.22358832521334526, 'text_cosine': 0.28035758062014504, 'mixed_without_clip': 1.5122540819266486, 'mixed_clip_only': 0.5039459057270534, 'mixed_with_clip': 2.0161999875590917}
Epoch 47, train losses: {'rc': 0.007403145515034714, 'rcxyz': 0.0068018153925160215, 'vel': 0.00040026938782714765, 'image_cosine': 0.22154185428978904, 'text_cosine': 0.27973666538794834, 'mixed_without_clip': 1.4605230301618577, 'mixed_clip_only': 0.501278519961569, 'mixed_with_clip': 1.9618015505018689}
Epoch 48, train losses: {'rc': 0.007466743175043828, 'rcxyz': 0.006802624930620253, 'vel': 0.0004015669522766349, 'image_cosine': 0.2223779370269132, 'text_cosine': 0.27981482432002114, 'mixed_without_clip': 1.4670935057458423, 'mixed_clip_only': 0.5021927606728341, 'mixed_with_clip': 1.9692862645974234}
Epoch 49, train losses: {'rc': 0.00818246753834602, 'rcxyz': 0.007418685766225237, 'vel': 0.00045794508821148024, 'image_cosine': 0.23307711616868065, 'text_cosine': 0.2816716320812702, 'mixed_without_clip': 1.6059098357246036, 'mixed_clip_only': 0.5147487484983035, 'mixed_with_clip': 2.1206585847196124}
Epoch 50, train losses: {'rc': 0.007416577223459229, 'rcxyz': 0.006749508538771243, 'vel': 0.0004176121095605638, 'image_cosine': 0.224064847578605, 'text_cosine': 0.27994136618716375, 'mixed_without_clip': 1.4583697845065404, 'mixed_clip_only': 0.5040062135647214, 'mixed_with_clip': 1.9623759976455144}
Epoch 51, train losses: {'rc': 0.007289907178058038, 'rcxyz': 0.00671054737232182, 'vel': 0.0004014072291384865, 'image_cosine': 0.22096364461476842, 'text_cosine': 0.2794485228165748, 'mixed_without_clip': 1.4401861773123816, 'mixed_clip_only': 0.5004121673722116, 'mixed_with_clip': 1.9405983464112357}
Epoch 52, train losses: {'rc': 0.007402453292715585, 'rcxyz': 0.006801958516654041, 'vel': 0.0004002315890541049, 'image_cosine': 0.22091074158984517, 'text_cosine': 0.27930540279263544, 'mixed_without_clip': 1.4604643407795164, 'mixed_clip_only': 0.500216145103886, 'mixed_with_clip': 1.9606804863801077}
Epoch 53, train losses: {'rc': 0.0074421755516798135, 'rcxyz': 0.006878388452617126, 'vel': 0.0004030679936390481, 'image_cosine': 0.22129320463254337, 'text_cosine': 0.2796566750322069, 'mixed_without_clip': 1.4723631988915187, 'mixed_clip_only': 0.5009498801496294, 'mixed_with_clip': 1.9733130786154005}
Epoch 54, train losses: {'rc': 0.007956758877151601, 'rcxyz': 0.007236216869574809, 'vel': 0.0004281156349327556, 'image_cosine': 0.22504898127582337, 'text_cosine': 0.2805955768104583, 'mixed_without_clip': 1.5621091408388956, 'mixed_clip_only': 0.505644558133587, 'mixed_with_clip': 2.067753700699125}
Epoch 55, train losses: {'rc': 0.0075538561254414535, 'rcxyz': 0.006841557559424213, 'vel': 0.00043601011600999514, 'image_cosine': 0.22434007591671415, 'text_cosine': 0.28106856398166175, 'mixed_without_clip': 1.483142381338846, 'mixed_clip_only': 0.505408640607955, 'mixed_with_clip': 1.9885510203384218}
Epoch 56, train losses: {'rc': 0.007257131039024523, 'rcxyz': 0.0066512702760981425, 'vel': 0.0004029232489258339, 'image_cosine': 0.22069033163170965, 'text_cosine': 0.2793499642657855, 'mixed_without_clip': 1.4311324586470922, 'mixed_clip_only': 0.5000402955072266, 'mixed_with_clip': 1.9311727526642026}
Epoch 57, train losses: {'rc': 0.007300389005011686, 'rcxyz': 0.006746891468378996, 'vel': 0.00039915383446015534, 'image_cosine': 0.22005403240521748, 'text_cosine': 0.2790251000296502, 'mixed_without_clip': 1.4446434315234895, 'mixed_clip_only': 0.4990791322456466, 'mixed_with_clip': 1.9437225637927888}
Epoch 58, train losses: {'rc': 0.00731794128641634, 'rcxyz': 0.006736570847080282, 'vel': 0.00039871862851820574, 'image_cosine': 0.22004739901139622, 'text_cosine': 0.27915853959700415, 'mixed_without_clip': 1.445323072634046, 'mixed_clip_only': 0.4992059382654372, 'mixed_with_clip': 1.9445290113252307}
Epoch 59, train losses: {'rc': 0.007256023776865313, 'rcxyz': 0.006740004460637768, 'vel': 0.00039885901434380294, 'image_cosine': 0.22080016531168467, 'text_cosine': 0.2786795131507374, 'mixed_without_clip': 1.4394887244417554, 'mixed_clip_only': 0.49947967787110614, 'mixed_with_clip': 1.938968404039504}
Epoch 60, train losses: {'rc': 0.007420261989584163, 'rcxyz': 0.00678838316114649, 'vel': 0.0004146185524217481, 'image_cosine': 0.22333100426291663, 'text_cosine': 0.2795648439772545, 'mixed_without_clip': 1.4623263711967165, 'mixed_clip_only': 0.5028958482638238, 'mixed_with_clip': 1.9652222201937721}
Epoch 61, train losses: {'rc': 0.0072770757554468535, 'rcxyz': 0.006793073149922762, 'vel': 0.00040134259826219126, 'image_cosine': 0.22046803003029217, 'text_cosine': 0.27935435133320946, 'mixed_without_clip': 1.4471491486307175, 'mixed_clip_only': 0.4998223815882017, 'mixed_with_clip': 1.946971531141372}
Epoch 62, train losses: {'rc': 0.007588677619221724, 'rcxyz': 0.006988563589263885, 'vel': 0.0004240814801180802, 'image_cosine': 0.22197735484630343, 'text_cosine': 0.27986680387504514, 'mixed_without_clip': 1.500132266916926, 'mixed_clip_only': 0.5018441580354221, 'mixed_with_clip': 2.001976427104738}
Epoch 63, train losses: {'rc': 0.007228912255682406, 'rcxyz': 0.006649697215909818, 'vel': 0.0003994804953565888, 'image_cosine': 0.21932638624594325, 'text_cosine': 0.27887631822673103, 'mixed_without_clip': 1.427808995710479, 'mixed_clip_only': 0.4982027043662374, 'mixed_with_clip': 1.9260116994380951}
Epoch 64, train losses: {'rc': 0.007333072232422493, 'rcxyz': 0.0067282699024496925, 'vel': 0.00040642237051672833, 'image_cosine': 0.2204928212340862, 'text_cosine': 0.27949829612459454, 'mixed_without_clip': 1.446776451856371, 'mixed_clip_only': 0.4999911173232018, 'mixed_with_clip': 1.9467675685882568}
Epoch 65, train losses: {'rc': 0.0077770869748386, 'rcxyz': 0.007093168141150344, 'vel': 0.0004329386774481954, 'image_cosine': 0.22442496035780227, 'text_cosine': 0.28090902924064604, 'mixed_without_clip': 1.5303193781584028, 'mixed_clip_only': 0.5053339894092272, 'mixed_with_clip': 2.0356533668343983}
Epoch 66, train losses: {'rc': 0.00738745174732887, 'rcxyz': 0.006745697750842997, 'vel': 0.00041458095900394717, 'image_cosine': 0.22255128925991438, 'text_cosine': 0.28032098190178945, 'mixed_without_clip': 1.454773043450855, 'mixed_clip_only': 0.5028722711498775, 'mixed_with_clip': 1.9576453138911536}
Epoch 67, train losses: {'rc': 0.007152713490982673, 'rcxyz': 0.006643475731465197, 'vel': 0.0004002127271041178, 'image_cosine': 0.21941969451450166, 'text_cosine': 0.2789510694997651, 'mixed_without_clip': 1.419640196646963, 'mixed_clip_only': 0.4983707635648667, 'mixed_with_clip': 1.9180109595495558}
Epoch 68, train losses: {'rc': 0.0073954792523254, 'rcxyz': 0.006827926730352735, 'vel': 0.00040894613579554954, 'image_cosine': 0.22237069689565234, 'text_cosine': 0.2792418598419144, 'mixed_without_clip': 1.4632352139741656, 'mixed_clip_only': 0.5016125569977458, 'mixed_with_clip': 1.9648477726512485}
Epoch 69, train losses: {'rc': 0.007144211445720718, 'rcxyz': 0.0066273891817896615, 'vel': 0.00039668036831518825, 'image_cosine': 0.21883463243406917, 'text_cosine': 0.27878214794965017, 'mixed_without_clip': 1.4168280983728077, 'mixed_clip_only': 0.4976167802536298, 'mixed_with_clip': 1.9144448779878163}
Epoch 70, train losses: {'rc': 0.007490359469582992, 'rcxyz': 0.006891763423355148, 'vel': 0.0004120923795401015, 'image_cosine': 0.22204667540296677, 'text_cosine': 0.2788450240142762, 'mixed_without_clip': 1.4794215273762505, 'mixed_clip_only': 0.5008916997720325, 'mixed_with_clip': 1.9803132298446837}
Epoch 71, train losses: {'rc': 0.007476405432773015, 'rcxyz': 0.006868898675089614, 'vel': 0.0004128846877609335, 'image_cosine': 0.2225157457329924, 'text_cosine': 0.27922387683675404, 'mixed_without_clip': 1.4758188809667314, 'mixed_clip_only': 0.5017396224160043, 'mixed_with_clip': 1.977558503832136}
Epoch 72, train losses: {'rc': 0.007162328740944051, 'rcxyz': 0.0066500095925885174, 'vel': 0.0004003202500161789, 'image_cosine': 0.22012645298289873, 'text_cosine': 0.27878782668757063, 'mixed_without_clip': 1.4212658553369462, 'mixed_clip_only': 0.49891427951672723, 'mixed_with_clip': 1.9201801365330107}
Epoch 73, train losses: {'rc': 0.00706100740566081, 'rcxyz': 0.006590952006323884, 'vel': 0.00039095620280325724, 'image_cosine': 0.21798938334224716, 'text_cosine': 0.2782379927143218, 'mixed_without_clip': 1.4042915640369293, 'mixed_clip_only': 0.496227375500732, 'mixed_with_clip': 1.900518939040956}
Epoch 74, train losses: {'rc': 0.0072420354215576065, 'rcxyz': 0.006716677810745462, 'vel': 0.00039670269867952263, 'image_cosine': 0.21921856657616676, 'text_cosine': 0.27888889369510467, 'mixed_without_clip': 1.435541592325483, 'mixed_clip_only': 0.49810746021213986, 'mixed_with_clip': 1.933649052987023}
Epoch 75, train losses: {'rc': 0.007146220371156695, 'rcxyz': 0.00663957459804055, 'vel': 0.00039488928923096935, 'image_cosine': 0.21871255631484682, 'text_cosine': 0.27837672647502687, 'mixed_without_clip': 1.4180684231576466, 'mixed_clip_only': 0.4970892831446633, 'mixed_with_clip': 1.9151577082891313}
Epoch 76, train losses: {'rc': 0.0076173422713985755, 'rcxyz': 0.006954408847608618, 'vel': 0.00042723833133109904, 'image_cosine': 0.224844265658231, 'text_cosine': 0.28049283585851154, 'mixed_without_clip': 1.4998989472786586, 'mixed_clip_only': 0.5053371022144953, 'mixed_with_clip': 2.0052360495877646}
Epoch 77, train losses: {'rc': 0.007129892634250046, 'rcxyz': 0.006577974585409734, 'vel': 0.0003996664102559739, 'image_cosine': 0.21889095369075973, 'text_cosine': 0.2786445293161604, 'mixed_without_clip': 1.410753364695443, 'mixed_clip_only': 0.4975354829714412, 'mixed_with_clip': 1.9082888460348524}
Epoch 78, train losses: {'rc': 0.0070901696967508, 'rcxyz': 0.006599369398983462, 'vel': 0.00039330657124833483, 'image_cosine': 0.21744921452232768, 'text_cosine': 0.27836414402912535, 'mixed_without_clip': 1.4082845656171679, 'mixed_clip_only': 0.495813358326753, 'mixed_with_clip': 1.90409792339991}
Epoch 79, train losses: {'rc': 0.007362025735958937, 'rcxyz': 0.006794816740448513, 'vel': 0.0004095204363231518, 'image_cosine': 0.22173833254547345, 'text_cosine': 0.2794512355847964, 'mixed_without_clip': 1.4566362886201767, 'mixed_clip_only': 0.5011895678109592, 'mixed_with_clip': 1.957825856170957}
Epoch 80, train losses: {'rc': 0.007118913224373486, 'rcxyz': 0.0066168401523360185, 'vel': 0.00039439393794820054, 'image_cosine': 0.21900692487519885, 'text_cosine': 0.27830876918539166, 'mixed_without_clip': 1.4130147294865714, 'mixed_clip_only': 0.497315693705801, 'mixed_with_clip': 1.9103304249899729}
Epoch 81, train losses: {'rc': 0.0070588713619000616, 'rcxyz': 0.0065685884570629, 'vel': 0.0003908615795634909, 'image_cosine': 0.21759154890028257, 'text_cosine': 0.2779826793642271, 'mixed_without_clip': 1.4018321418572985, 'mixed_clip_only': 0.4955742285838203, 'mixed_with_clip': 1.8974063697315398}
Epoch 82, train losses: {'rc': 0.007203035644396016, 'rcxyz': 0.006719105851851285, 'vel': 0.0004118389258528912, 'image_cosine': 0.2217148348570816, 'text_cosine': 0.2789796847199637, 'mixed_without_clip': 1.4333980446770078, 'mixed_clip_only': 0.500694519044861, 'mixed_with_clip': 1.934092565661385}
Epoch 83, train losses: {'rc': 0.0070273964614089995, 'rcxyz': 0.0065724822709203835, 'vel': 0.00039403510387591453, 'image_cosine': 0.21831431460995523, 'text_cosine': 0.27808203683013005, 'mixed_without_clip': 1.3993913862913374, 'mixed_clip_only': 0.4963963519013117, 'mixed_with_clip': 1.8957877359693012}
Epoch 84, train losses: {'rc': 0.0070440348222970016, 'rcxyz': 0.006579831024698381, 'vel': 0.00038941555057323376, 'image_cosine': 0.21694600353874857, 'text_cosine': 0.2781512115446348, 'mixed_without_clip': 1.401328138463081, 'mixed_clip_only': 0.4950972150952097, 'mixed_with_clip': 1.8964253537238591}
Epoch 85, train losses: {'rc': 0.007041657043229197, 'rcxyz': 0.00659785322283232, 'vel': 0.00039052816445334993, 'image_cosine': 0.21753405634136427, 'text_cosine': 0.27806517585875495, 'mixed_without_clip': 1.4030038434361654, 'mixed_clip_only': 0.49559923157332436, 'mixed_with_clip': 1.8986030757427215}
Epoch 86, train losses: {'rc': 0.007011049719978242, 'rcxyz': 0.0065836583493307, 'vel': 0.0003857710774542601, 'image_cosine': 0.2164764380525975, 'text_cosine': 0.2778571119857213, 'mixed_without_clip': 1.3980479137765036, 'mixed_clip_only': 0.4943335494115239, 'mixed_with_clip': 1.8923814611775533}
Epoch 87, train losses: {'rc': 0.006992633222594916, 'rcxyz': 0.006540328137842672, 'vel': 0.0003838213489091556, 'image_cosine': 0.21633291450284775, 'text_cosine': 0.277524476037139, 'mixed_without_clip': 1.3916782705556778, 'mixed_clip_only': 0.4938573900432814, 'mixed_with_clip': 1.885535659771117}
Epoch 88, train losses: {'rc': 0.007067314607446038, 'rcxyz': 0.00661333292253345, 'vel': 0.00038878239170622286, 'image_cosine': 0.21733599594897693, 'text_cosine': 0.27805361085467867, 'mixed_without_clip': 1.406942990989912, 'mixed_clip_only': 0.4953896068391346, 'mixed_with_clip': 1.9023326005254473}
Epoch 89, train losses: {'rc': 0.006950985331324831, 'rcxyz': 0.006543001138959967, 'vel': 0.0003825471292541821, 'image_cosine': 0.2160999132172456, 'text_cosine': 0.27763538251793574, 'mixed_without_clip': 1.3876533597707748, 'mixed_clip_only': 0.49373529527395493, 'mixed_with_clip': 1.8813886558252668}
Epoch 90, train losses: {'rc': 0.0068956338750597625, 'rcxyz': 0.006478721687998918, 'vel': 0.0003794797100699581, 'image_cosine': 0.21553627482009313, 'text_cosine': 0.27720774556436234, 'mixed_without_clip': 1.3753835258975862, 'mixed_clip_only': 0.4927440200533186, 'mixed_with_clip': 1.868127549262274}
Epoch 91, train losses: {'rc': 0.007019160879731533, 'rcxyz': 0.006652137496331263, 'vel': 0.0003871598131173948, 'image_cosine': 0.21636752570195802, 'text_cosine': 0.27751102017031776, 'mixed_without_clip': 1.405845821234915, 'mixed_clip_only': 0.4938785452218283, 'mixed_with_clip': 1.899724366740575}
Epoch 92, train losses: {'rc': 0.006887454860880675, 'rcxyz': 0.006475694614299942, 'vel': 0.0003819710051175207, 'image_cosine': 0.21565401451218696, 'text_cosine': 0.27711454022025306, 'mixed_without_clip': 1.3745120456767461, 'mixed_clip_only': 0.4927685542239083, 'mixed_with_clip': 1.8672806000898754}
Epoch 93, train losses: {'rc': 0.00686511321895061, 'rcxyz': 0.006472892007820072, 'vel': 0.00037878693329976016, 'image_cosine': 0.2151129865930194, 'text_cosine': 0.2768947912823586, 'mixed_without_clip': 1.3716792167179168, 'mixed_clip_only': 0.49200777747328317, 'mixed_with_clip': 1.863686992630126}
Epoch 94, train losses: {'rc': 0.006899433152838832, 'rcxyz': 0.006511959626657208, 'vel': 0.0003787165966983478, 'image_cosine': 0.21533169526429402, 'text_cosine': 0.27732073305145144, 'mixed_without_clip': 1.3790109382262306, 'mixed_clip_only': 0.4926524274405979, 'mixed_with_clip': 1.8716633653829968}
Epoch 95, train losses: {'rc': 0.00685179497370319, 'rcxyz': 0.006479748458941541, 'vel': 0.00037759395922803957, 'image_cosine': 0.21493251905554817, 'text_cosine': 0.2766728976889262, 'mixed_without_clip': 1.3709137393841668, 'mixed_clip_only': 0.49160541629507426, 'mixed_with_clip': 1.8625191570274413}
Epoch 96, train losses: {'rc': 0.006844549304953525, 'rcxyz': 0.006483641901933071, 'vel': 0.0003763123749180638, 'image_cosine': 0.214866622667464, 'text_cosine': 0.2765640872574988, 'mixed_without_clip': 1.3704503610493646, 'mixed_clip_only': 0.49143071001957334, 'mixed_with_clip': 1.8618810734105489}
Epoch 97, train losses: {'rc': 0.0068600254634114365, 'rcxyz': 0.006499303364202321, 'vel': 0.0003764923758151579, 'image_cosine': 0.21478549816778728, 'text_cosine': 0.27649267330056143, 'mixed_without_clip': 1.3735821182765657, 'mixed_clip_only': 0.491278171255475, 'mixed_with_clip': 1.8648602883021037}
Epoch 98, train losses: {'rc': 0.006881102508429202, 'rcxyz': 0.006535988653056501, 'vel': 0.0003764459880166835, 'image_cosine': 0.21491587241254156, 'text_cosine': 0.2765487343072891, 'mixed_without_clip': 1.3793537129485418, 'mixed_clip_only': 0.49146460644782536, 'mixed_with_clip': 1.8708183193017567}
Epoch 99, train losses: {'rc': 0.0070977505978329905, 'rcxyz': 0.006664124725475198, 'vel': 0.0003930132248829718, 'image_cosine': 0.21763979900214409, 'text_cosine': 0.2773394354279079, 'mixed_without_clip': 1.41548885381411, 'mixed_clip_only': 0.4949792341580467, 'mixed_with_clip': 1.9104680884452092}
Epoch 100, train losses: {'rc': 0.007229151050116689, 'rcxyz': 0.006836065939546282, 'vel': 0.00040490413844559547, 'image_cosine': 0.21969662696596176, 'text_cosine': 0.2779202746493476, 'mixed_without_clip': 1.4470121117811354, 'mixed_clip_only': 0.49761690215932, 'mixed_with_clip': 1.9446290153359609}

[shunlinlu]

And I test the model you provided, it works fine.

[ColinZhou1993]

Hi,

Are there any clues about the reason why the motions reproduced are static? I believe I followed the instructions very well. I also mentioned the issue here. #5 (comment)

Shunlin

Hello, I met the same problem, have you solved it?