PyTerrier demonstration for vaswani

This notebook demonstrates retrieval using PyTerrier on the Vaswani corpus.

About the corpus: The Vaswani NPL corpus is a small test collection of 11,000 abstracts has been used by the Glasgow IR group for many years (created 1990). Due to its small size, it is used for many test cases used in both Terrier and PyTerrier.

In [1]:
#!pip install -q python-terrier
import pyterrier as pt
if not pt.started():
    pt.init()

from pyterrier.measures import *
dataset = pt.get_dataset('vaswani')
        

PyTerrier 0.7.0 has loaded Terrier 5.6 (built by craigmacdonald on 2021-09-17 13:27)
In [2]:
bm25_terrier_stemmed = pt.BatchRetrieve.from_dataset('vaswani', 'terrier_stemmed', wmodel='BM25')

dph_terrier_stemmed = pt.BatchRetrieve.from_dataset('vaswani', 'terrier_stemmed', wmodel='DPH')

dph_bo1_terrier_stemmed = dph_terrier_stemmed >> pt.rewrite.Bo1QueryExpansion(pt.get_dataset('vaswani').get_index('terrier_stemmed')) >> dph_terrier_stemmed
In [3]:
pt.Experiment(
    [bm25_terrier_stemmed, dph_terrier_stemmed, dph_bo1_terrier_stemmed],
    pt.get_dataset('vaswani').get_topics(),
    pt.get_dataset('vaswani').get_qrels(),
    batch_size=200,
    filter_by_qrels=True,
    eval_metrics=['map'],
    names=['bm25_terrier_stemmed', 'dph_terrier_stemmed', 'dph_bo1_terrier_stemmed'])
        

18:21:31.465 [main] WARN org.terrier.applications.batchquerying.TRECQuery - trec.encoding is not set; resorting to platform default (ISO-8859-1). Retrieval may be platform dependent. Recommend trec.encoding=UTF-8
Out[3]:
name map
0 bm25_terrier_stemmed 0.296517
1 dph_terrier_stemmed 0.283614
2 dph_bo1_terrier_stemmed 0.295955