Merge v3.5.1 zqb branch to v3.6 (cocos#11637)

* support asset property preview (cocos#11017)

(cherry picked from commit 85769cf8f81459b5ad98ccab4138420009572a32)

* show mesh asset thumbnail (cocos#10981)

* V3.5.1 noise particle (cocos#11003)

* add cpu & gpu noise for particle

* add some params & change particle noise shader

* change shader

* add cpu noise particle

* add noise preview

* run generation when preview

* revert shaders

* remove unused statements

* add particle noise module

* change noise preview interface

* add invisible to remap

* remove get deltaTime

* set noiseModule private

* fix eslint

* feat: add noise preview (cocos#11260)

* [Optimize] asset editing undo (cocos#10779)

* edit render-pipeline

* support renderpipeline undo

* optimize material editing undo

* reset last record data

* temporary not support animation-mask undo

* support animation-mask edit undo

* animation mask remove _jointMasks

* Animation clip embedded players (cocos#11320)

* Animation clip subregions

* Log

* Test

* Eval

* Update

* Update

* No longer use animation state to implement clip motion

* Clear subregions

* Fix

* Mark subregion as editor extendable

* Animation clip subregion player path

* fix isnan(0.0) with webgl1.0 (cocos#11262)

* fix isinf(0.0)==true with webgl1.0

* code style

* fix specular params with surface toon (cocos#11265)

* code format and add sliders for parameters (cocos#11266)

* Broadcasting of animation state\'s speed

* Fix

* Rename

* Update

* Fix

* Revert "No longer use animation state to implement clip motion"

This reverts commit 20755f9.

* Revert some exotic changes

* Update docs

Co-authored-by: jk20012001 <jkregister@163.com>
Co-authored-by: 徐兵 <49358166+xubing0906@users.noreply.github.com>

* fix skinning model split (cocos#11378)

* fix skinning model split

* rename function _resizeMaxJointForDS

Co-authored-by: zhakesi <zhakesi@foxmail.com>

* [ci skip][AUTO]: Automated genbindings update: d823cbb  (cocos#11378) (cocos#11383)

Co-authored-by: cocos-robot <cocos-robot@cocos.com>

* use stop emitting instead (cocos#11381)

* add stop emitting api

* use stop emitting instead

* Update cocos/particle/particle-system.ts

Co-authored-by: Leslie Leigh (李的序) <leslieleigh@outlook.com>

Co-authored-by: Leslie Leigh (李的序) <leslieleigh@outlook.com>

* fix: imageData fault tolerance (cocos#11374)

* fix: imageData fault tolerance

* chore: when data is an empty array,stop the function

* material  undo updating effects list (cocos#11409)

* modify noise module ui (cocos#11433)

* use visible decorator to control the visibility of property (cocos#11438)

* fix joints data with value nan (cocos#11489)

* V3.5.2 CI Android NDK  set version to 21 (cocos#11448) (cocos#11492)

* Minor improvement

* Fix format

* Fix particle culler

* Improve particle culler

* Manually migrate joint data upload with texture in skinning model

* Update shaders for skinning model based on joint texture

* fis native skinning model with realtime-texture mode (#45)

Co-authored-by: zhakesi <zhakesi@foxmail.com>

* fix clangtidy (#46)

* Small tweaks

* modify code struct for skinning model (#47)

* modify code struct for skinning model

* update skinning chunk

Co-authored-by: 黄森斌 <arsen2010@126.com>
Co-authored-by: zxx43 <zxx43@hotmail.com>
Co-authored-by: 袁炜海 <35713518+dogodo-cc@users.noreply.github.com>
Co-authored-by: Leslie Leigh (李的序) <leslieleigh@outlook.com>
Co-authored-by: jk20012001 <jkregister@163.com>
Co-authored-by: 徐兵 <49358166+xubing0906@users.noreply.github.com>
Co-authored-by: zhakesi <liqiao@cocos.com>
Co-authored-by: zhakesi <zhakesi@foxmail.com>
Co-authored-by: Cocos Robot <48829427+cocos-robot@users.noreply.github.com>
Co-authored-by: cocos-robot <cocos-robot@cocos.com>
Co-authored-by: Santy-Wang <wh19930829@qq.com>
Co-authored-by: zhangxin <59186205+chichinohaha@users.noreply.github.com>
Co-authored-by: 江战 <40414978+PatriceJiang@users.noreply.github.com>

EngineErrorMap.md

@@ -1840,6 +1840,10 @@ Instancing/Batching enabled for non-baked skinning model '%s', this may result i
 ### 3937
 Previous error occurred when instantiating animation clip %s on node %s.
 
+### 3938
+
+'%s' is not found from '%s'. It's specified as the root node to play animation clip '%s'.
+
 ### 4000
 
 <!-- DEPRECATED -->
@@ -3267,4 +3271,4 @@ The shadow visible distance is so small that CSM stratification is not effective
 
 ### 15004
 
-The native folder may be generated from older versions, please refer https://docs.cocos.com/creator/manual/en/release-notes/ to upgrade.
+The native folder may be generated from older versions, please refer https://docs.cocos.com/creator/manual/en/release-notes/ to upgrade.

cocos/3d/assets/skeleton.jsb.ts

@@ -23,11 +23,6 @@
  THE SOFTWARE.
 */
 
-/**
- * @packageDocumentation
- * @module asset
- */
-
 import { ccclass, type, serializable } from 'cc.decorator';
 import { CCString } from '../../core/data/utils/attribute';
 import { Mat4 } from '../../core/math';

cocos/3d/models/skinning-model.ts

@@ -28,19 +28,26 @@ import { RenderingSubMesh } from '../../core/assets/rendering-sub-mesh';
 import { Mesh } from '../assets/mesh';
 import { Skeleton } from '../assets/skeleton';
 import { AABB } from '../../core/geometry';
-import { BufferUsageBit, MemoryUsageBit, DescriptorSet, Buffer, BufferInfo, Attribute } from '../../core/gfx';
+import { BufferUsageBit, MemoryUsageBit, DescriptorSet, Buffer, BufferInfo, Attribute, FormatFeatureBit, Format, Texture } from '../../core/gfx';
 import { Mat4, Vec3 } from '../../core/math';
-import { UBOSkinning } from '../../core/pipeline/define';
+import { UBOSkinning, UNIFORM_REALTIME_JOINT_TEXTURE_BINDING } from '../../core/pipeline/define';
 import { Node } from '../../core/scene-graph/node';
 import { ModelType } from '../../core/renderer/scene/model';
 import { uploadJointData } from '../skeletal-animation/skeletal-animation-utils';
 import { MorphModel } from './morph-model';
 import { deleteTransform, getTransform, getWorldMatrix, IJointTransform } from '../../core/animation/skeletal-animation-utils';
 import { IMacroPatch, BatchingSchemes, Pass } from '../../core/renderer';
 import { warnID } from '../../core/platform/debug';
+import { ImageAsset, Texture2D, director } from '../../core';
+import { PixelFormat } from '../../core/assets/asset-enum';
 
-const myPatches: IMacroPatch[] = [
+const uniformPatches: IMacroPatch[] = [
     { name: 'CC_USE_SKINNING', value: true },
+    { name: 'CC_USE_REAL_TIME_JOINT_TEXTURE', value: false },
+];
+const texturePatches: IMacroPatch[] = [
+    { name: 'CC_USE_SKINNING', value: true },
+    { name: 'CC_USE_REAL_TIME_JOINT_TEXTURE', value: true },
 ];
 
 function getRelevantBuffers (outIndices: number[], outBuffers: number[], jointMaps: number[][], targetJoint: number) {
@@ -73,6 +80,14 @@ const v3_2 = new Vec3();
 const m4_1 = new Mat4();
 const ab_1 = new AABB();
 
+class RealTimeJointTexture {
+    public static readonly WIDTH = 256;
+    public static readonly HEIGHT = 3;
+    public _format = PixelFormat.RGBA32F; // default use float texture
+    public _textures: Texture2D[] = [];
+    public _buffers: Float32Array[] = [];
+}
+
 /**
  * @en
  * The skinning model that is using real-time pose calculation.
@@ -84,6 +99,8 @@ export class SkinningModel extends MorphModel {
     private _dataArray: Float32Array[] = [];
     private _joints: IJointInfo[] = [];
     private _bufferIndices: number[] | null = null;
+    private _realTimeJointTexture = new RealTimeJointTexture();
+    private _realTimeTextureMode = false;
     constructor () {
         super();
         this.type = ModelType.SKINNING;
@@ -97,6 +114,12 @@ export class SkinningModel extends MorphModel {
             }
             this._buffers.length = 0;
         }
+        this._dataArray.length = 0;
+        this._realTimeJointTexture._textures.forEach((tex) => {
+            tex.destroy();
+        });
+        this._realTimeJointTexture._textures.length = 0;
+        this._realTimeJointTexture._buffers.length = 0;
         super.destroy();
     }
 
@@ -120,11 +143,14 @@ export class SkinningModel extends MorphModel {
         }
         this._bufferIndices = null; this._joints.length = 0;
         if (!skeleton || !skinningRoot || !mesh) { return; }
+        this._realTimeTextureMode = false;
+        if (UBOSkinning.JOINT_UNIFORM_CAPACITY < skeleton.joints.length) { this._realTimeTextureMode = true; }
         this.transform = skinningRoot;
         const boneSpaceBounds = mesh.getBoneSpaceBounds(skeleton);
         const jointMaps = mesh.struct.jointMaps;
         this._ensureEnoughBuffers(jointMaps && jointMaps.length || 1);
         this._bufferIndices = mesh.jointBufferIndices;
+        this._initRealTimeJointTexture();
         for (let index = 0; index < skeleton.joints.length; index++) {
             const bound = boneSpaceBounds[index];
             const target = skinningRoot.getChildByPath(skeleton.joints[index]);
@@ -185,8 +211,12 @@ export class SkinningModel extends MorphModel {
                 uploadJointData(this._dataArray[buffers[b]], indices[b] * 12, m4_1, i === 0);
             }
         }
-        for (let b = 0; b < this._buffers.length; b++) {
-            this._buffers[b].update(this._dataArray[b]);
+        if (this._realTimeTextureMode) {
+            this._updateRealTimeJointTextureBuffer();
+        } else {
+            for (let b = 0; b < this._buffers.length; b++) {
+                this._buffers[b].update(this._dataArray[b]);
+            }
         }
         return true;
     }
@@ -209,7 +239,10 @@ export class SkinningModel extends MorphModel {
     // override
     public getMacroPatches (subModelIndex: number): IMacroPatch[] | null {
         const superMacroPatches = super.getMacroPatches(subModelIndex);
-
+        let myPatches = uniformPatches;
+        if (this._realTimeTextureMode) {
+            myPatches = texturePatches;
+        }
         if (superMacroPatches) {
             return myPatches.concat(superMacroPatches);
         }
@@ -221,8 +254,13 @@ export class SkinningModel extends MorphModel {
      */
     public _updateLocalDescriptors (submodelIdx: number, descriptorSet: DescriptorSet) {
         super._updateLocalDescriptors(submodelIdx, descriptorSet);
-        const buffer = this._buffers[this._bufferIndices![submodelIdx]];
-        if (buffer) { descriptorSet.bindBuffer(UBOSkinning.BINDING, buffer); }
+        const idx = this._bufferIndices![submodelIdx];
+        if (this._realTimeTextureMode) {
+            this._bindRealTimeJointTexture(idx, descriptorSet);
+        } else {
+            const buffer = this._buffers[idx];
+            if (buffer) { descriptorSet.bindBuffer(UBOSkinning.BINDING, buffer); }
+        }
     }
 
     protected _updateInstancedAttributes (attributes: Attribute[], pass: Pass) {
@@ -234,18 +272,117 @@ export class SkinningModel extends MorphModel {
     }
 
     private _ensureEnoughBuffers (count: number) {
-        for (let i = 0; i < count; i++) {
-            if (!this._buffers[i]) {
+        if (this._buffers.length) {
+            for (let i = 0; i < this._buffers.length; i++) {
+                this._buffers[i].destroy();
+            }
+            this._buffers.length = 0;
+        }
+
+        if (this._dataArray.length) this._dataArray.length = 0;
+
+        if (!this._realTimeTextureMode) {
+            for (let i = 0; i < count; i++) {
                 this._buffers[i] = this._device.createBuffer(new BufferInfo(
                     BufferUsageBit.UNIFORM | BufferUsageBit.TRANSFER_DST,
                     MemoryUsageBit.HOST | MemoryUsageBit.DEVICE,
                     UBOSkinning.SIZE,
                     UBOSkinning.SIZE,
                 ));
+                const maxJoints = UBOSkinning.JOINT_UNIFORM_CAPACITY;
+                this._dataArray[i] = new Float32Array(12 * maxJoints);
+            }
+        } else {
+            for (let i = 0; i < count; i++) {
+                const maxJoints = RealTimeJointTexture.WIDTH;
+                this._dataArray[i] = new Float32Array(12 * maxJoints);
             }
-            if (!this._dataArray[i]) {
-                this._dataArray[i] = new Float32Array(UBOSkinning.COUNT);
+        }
+    }
+
+    private _initRealTimeJointTexture () {
+        if (this._realTimeJointTexture._textures.length) {
+            this._realTimeJointTexture._textures.forEach((tex) => {
+                tex.destroy();
+            });
+            this._realTimeJointTexture._textures.length = 0;
+        }
+        this._realTimeJointTexture._buffers.length = 0;
+        if (!this._realTimeTextureMode) return;
+
+        const gfxDevice = director.root!.device;
+        let width = RealTimeJointTexture.WIDTH;
+        const height = RealTimeJointTexture.HEIGHT;
+        const hasFeatureFloatTexture = gfxDevice.getFormatFeatures(Format.RGBA32F) & FormatFeatureBit.SAMPLED_TEXTURE;
+        if (hasFeatureFloatTexture === 0) {
+            this._realTimeJointTexture._format = PixelFormat.RGBA8888;
+            width = 4 * RealTimeJointTexture.WIDTH;
+        }
+
+        const textures = this._realTimeJointTexture._textures;
+        const buffers = this._realTimeJointTexture._buffers;
+        const pixelFormat = this._realTimeJointTexture._format;
+        for (let i = 0; i < this._dataArray.length; i++) {
+            buffers[i] = new Float32Array(4 * RealTimeJointTexture.HEIGHT * RealTimeJointTexture.WIDTH);
+            const arrayBuffer = buffers[i];
+            const updateView =  pixelFormat === PixelFormat.RGBA32F ? arrayBuffer : new Uint8Array(arrayBuffer.buffer);
+            const image = new ImageAsset({
+                width,
+                height,
+                _data: updateView,
+                _compressed: false,
+                format: pixelFormat,
+            });
+            const texture = new Texture2D();
+            texture.setFilters(Texture2D.Filter.NEAREST, Texture2D.Filter.NEAREST);
+            texture.setMipFilter(Texture2D.Filter.NONE);
+            texture.setWrapMode(Texture2D.WrapMode.CLAMP_TO_EDGE, Texture2D.WrapMode.CLAMP_TO_EDGE, Texture2D.WrapMode.CLAMP_TO_EDGE);
+            texture.image = image;
+            textures[i] = texture;
+        }
+    }
+
+    private _bindRealTimeJointTexture (idx: number, descriptorSet: DescriptorSet) {
+        if (!this._realTimeTextureMode) return;
+        const jointTexture = this._realTimeJointTexture._textures[idx];
+        if (jointTexture) {
+            const gfxTexture = jointTexture.getGFXTexture();
+            const sampler = jointTexture.getGFXSampler();
+            descriptorSet.bindTexture(UNIFORM_REALTIME_JOINT_TEXTURE_BINDING, gfxTexture!);
+            descriptorSet.bindSampler(UNIFORM_REALTIME_JOINT_TEXTURE_BINDING, sampler);
+        }
+    }
+
+    private _updateRealTimeJointTextureBuffer () {
+        if (!this._realTimeTextureMode) return;
+        const textures = this._realTimeJointTexture._textures;
+        const buffers = this._realTimeJointTexture._buffers;
+        for (let idx = 0; idx < textures.length; idx++) {
+            const arrayBuffer = buffers[idx];
+            const src = this._dataArray[idx];
+            const count = src.length / 12; // mat3x4
+            let idxSrc = 0;
+            let idxDst = 0;
+            for (let i = 0; i < count; i++) {
+                idxDst = 4 * i;
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                idxDst = 4 * (i + RealTimeJointTexture.WIDTH);
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                idxDst = 4 * (i + 2 * RealTimeJointTexture.WIDTH);
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
+                arrayBuffer[idxDst++] = src[idxSrc++];
             }
+            const pixelFormat = this._realTimeJointTexture._format;
+            const updateView = pixelFormat === PixelFormat.RGBA32F ? arrayBuffer : new Uint8Array(arrayBuffer.buffer);
+            textures[idx].uploadData(updateView);
         }
     }
 }