Tag: flat-t5

• InstructBLIP Towards General-purpose Vision-Language Models with Instruction Tuning (17 Oct 2023)
  

  This is my reading note for InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. The paper proposes an extension of blip 2 with institution tuning. This has dramatically improved the performance to unseen tasks. The method is based on query transformer, but adding the tokens from the instruction to guide the feature extraction.