Unboxing BPE, WordPiece and SentencePiece — Tokenisation is the task of splitting the text into tokens which are then converted to numbers. These numbers are in turn used by the machine learning models for further processing and training. Splitting text into tokens is not as trivial as it sounds. The simplest way we can tokenize a…

Introduction We live in a world which is more opinionated than ever. Any service that we consume leaves us either satisfied or unsatisfied. And with the advent of social media, we make our views public in no time. Vast sources of data are available in the form of reviews, customer satisfaction…

Know the flaws of BERT that gave rise to its cousins! — Introduction With the onset of the Transformer, there has been a rapid rise in language models. In 2018, BERT came and broke all records. However, shortly after BERT, a long list of its cousins were born, like RoBERTa, ALBERT, StructBERT, DistilBERT, to name a few. BERT is essentially trained to optimise…

Don’t overlook preprocessing! Tailor it according to your use case. — Introduction It is often said that data scientists spend almost 70% of their time in feature engineering. With NLP, the same is true for cleaning the text. Almost all real life NLP datasets are extremely unstructured. Sources like social media, customer reviews, even news articles or wikipedia data are exploding with…