Titlebook

最初

Opening the Black Box: Analyzing Attention Weights and Hidden States in Pre-trained Language Models problem-solving strategies. Additionally, by inspecting the attention weights layer by layer, we uncover an unconventional finding that layer 10, rather than the model’s final layer, is the optimal layer to unfreeze for the least parameter-intensive approach to fine-tune the model. We support these