OAK: Enriching Doc Representations using Auxiliary Knowledge for XC
The objective in eXtreme Classification (XC) is to find relevant labels for a document from an exceptionally large label space. Most XC application scenarios have rich auxiliary data associated with the input documents, e.g., frequently clicked webpages for search queries in sponsored search. Unfortunately, most of the existing XC methods do not use any auxiliary data. In this paper, we propose a novel framework, Online Auxiliary Knowledge (OAK), which harnesses auxiliary information linked to the document to improve XC accuracy. OAK stores information learnt from the auxiliary data in a knowledge bank and during a forward pass, retrieves relevant auxiliary knowledge embeddings for a given document. An enriched embedding is obtained by fusing these auxiliary knowledge embeddings with the document’s embedding, thereby enabling much more precise candidate label selection and final classification. OAK training involves three stages. (1) Training a linker module to link documents to relevant auxiliary data points. (2) Learning an embedding for documents enriched using linked auxiliary information. (3) Using the enriched document embeddings to learn the final classifiers. OAK outperforms current state-of-the-art XC methods by up to ~5% on academic datasets, and by ~3% on an auxiliary data-augmented variant of LF-ORCAS-800K dataset in Precision@1. OAK also demonstrates statistically significant improvements in sponsored search metrics when deployed on a large scale search engine.
In this video, I talk about the following: What is XC, and what was missing in XC methods? How do you use auxiliary knowledge pieces (AKPs)? How does OAK (Online Auxiliary Knowledge) work? What is OAK’s Architecture? Training the augmentation block. Inference using OAK. Results. Ablations. Performance on Tail. Results using Oracle Linker AKPs. Application to Sponsored search. Sample predictions from OAK, NGAME and GraphFormer.
For more details, please look at https://drive.google.com/file/d/12vuT...
OAK: Enriching Document Representations using Auxiliary Knowledge for Extreme Classification. Shikhar Mohan, Deepak Saini, Anshul Mittal, Sayak Ray Chowdhury, Bhawna Paliwal, Jian Jiao, Manish Gupta, Manik Varma. The 41st Intl. Conf. on Machine Learning. 2024.