Notes
LoRA的基本概念
LoRA的基本概念
Loading...
LoRA的基本概念
在正式介绍LoR之前, 先看一个SVD的例子
import torch
import numpy as np
_ = torch.manual_seed(0)d, k = 10, 10
# This way we can generate a rank-deficient matrix
W_rank = 2
W = torch.randn(d,W_rank) @ torch.randn(W_rank,k)
print(W)