Neural network techniques for Information Retrieval

The retrieval is being done on 2 datasets : Nepal earthquake , Italy earthquake

• Character Level embeddings
• Word and Character Level embeddings in skipgram setting
• Word embedding with attention over character embedding
• Word embedding with attention over BiLSTM character embedding

• No query expansion
• Query expansion

Mode switiching is unimplemented and for now is being done by changes to source code

• CLE : Character Level embeddings that are trained using Character Level context
• WC1 : Word and Character Level embeddings that are both combined together and trained to predict the context of the token skipgram method
• WC2 : Word and Character Level embeddings that are combined after applying attention to character sequence of the token while training is done in skipgram setting
• WC3 : Word embeddings and attention over Character level BiLSTM model for token embedding extraction while training is done in skipgram setting

The evaluation is run with ./trec eval -q -m <measure standards> <standard> <output>

The data is available / was available under FIRE2016

Mostly all utility methods are present in different files which perfectly define the use of the function. The transfer of variables was benchmarked to observe the slowdown did not exist.

• The models must have 4 specific placeholders, apart / including those made for training
• They are namely tweet_query_char_holder , tweet_query_word_holder , tweet_word_holder , tweet_char_holder
• Tweet holders take batch input for tweets which are to be evaluated
• Tweet query holder take input the tweet which is to be used as query
• There must be atleast a single tweet_similarity tensor along the model which computes the the required metric for tweet retrieval acording to which sorting shall happen
• Library does not support multiple testing metrics, but will be implemented shortly

NOTE : For access to datasets please contact prannay[dot]khosla[at]gmail[dot]com