How to achieve PiSSA+LieRA?

[ouc-xuanyu]

Paper "Generalized Tensor-based Parameter-Efficient Fine-Tuning via Lie Group Transformations" mentions that LieRA could combine with PiSSA, In my implementation, this approach causes the value to drop (0.884 -> 0.145). I would like to know how it is specifically implemented. Does it act like dropout(x) @ (self.base_layer.weight * (lora_B.weight @ lora_A.weight )).T * scaling? where self.base_layer is q_proj/k_proj/v_proj/v_proj in dino3-s for resisc45 (remote sensing image classification)