A Beginner’s Guide to Large Language Models
By Maitreya Natu
In our ongoing series of blogs “Unravelling the AI mystery” Digitate continues to explore advances in AI and our experiences in turning AI and GenAI theory into practice. The blogs are intended to enlighten you as well as provide perspective into how Digitate solutions are built.
Please enjoy the blogs
2. Prompt Engineering – Enabling Large Language Models to Communicate With Humans
3. What are Large Language Models? Use Cases & Applications
4. Harnessing the power of word embeddings
written by different members of our top-notch team of data scientists and Digitate solution providers.
A Beginner’s Guide to Large Language Models
Natural Language Processing (NLP) influences our world in many ways. Our daily lives are permeated by applications of NLP, such as search engines, question answering, document analysis, spam filtering, customer service bots, etc. It is fascinating to study how the underlying engines running these applications work, especially how machines work with language and text.
It is numbers that a computer understands and manipulates. To represent and process language or textual information, we convert it into numbers called embeddings, which we discussed in one of our previous blog posts. Over the years, the techniques of language understanding/representation have evolved in the world of NLP. Following are some of the key areas in which NLP has evolved over time:
Statistical Measures: A baby step in this space consists of applying statistical techniques on strings. No meaning is attached to the text yet. Techniques such as set similarity and edit distance belong to this class.
Word Embeddings: This is a simple way to understand some meaning of the text. Words are represented as numbers such that these numbers capture the context in which the words are used. Techniques such as word2vec fall in this space.
Sentence Embeddings: Word embeddings are then combined to form sentence-level embeddings to define the meaning of longer sentences and paragraphs. It captures the context across sentences; however, there is no detailed contextual understanding of the language itself. Techniques such as Bag of Words fall in this space.
Language Models: These are complex models designed to understand and generate human language. They learn from a raw text corpus.
Large Language Models (LLM): If the size of the language model and training corpus is in the order of hundreds of millions, it is called a large language model. Any type of language model can be scaled up; however, most recently, the focus has been on scaling up generative models. BERT and GPT (Generative pre-trained transformer) are examples of large language models.
In this post, we will explore large language models (LLMs) with a specific focus on generative analytics. As there are many large language models, we will deep-dive with examples of the popular GPT series. This blog discusses GPT models in general. We will publish another blog with a specific focus on ChatGPT, a separate offshoot of GPT models that is specifically trained for conversations.