AA

A. Abdul Wahab

info

Please Note

1 records found

Master thesis (2026) - A. Abdul Wahab, A. Anand, H.S. Hung
Dense retrievers are often trained using contrastive learning to differentiate a positive document from a negative one. The harder the negative, the better the training signal and thus the better the distinction. In-batch training takes its negatives from other queries in the same batch, while cross-batch increases the negative document size by using several batches of them. These documents, however, are most of the time easy to tell apart, and larger batches come at the cost of more GPU memory. ANCE finds harder negatives by periodically re-encoding the full corpus with a new checkpoint of the model, which is expensive to do on limited hardware. This thesis investigates whether hard negatives can be found without re-encoding the whole corpus. GRASS is then proposed, which mines documents from a small cache that periodically gets updated and refreshed, and uses the model’s uncertainty in the selection of hard negatives. After fine-tuning the pretrained retriever BGE-M3, GRASS scored on par with ANCE on the 12 BRIGHT domains and higher than cross-batch and in-batch, while each mining round took about a third of the time of an ANCE round. ...