Natural Language Processing is the fastest-growing specialisation in AI and data science. NLP engineers are among the highest-paid data professionals, and companies are hiring rapidly for roles spanning search, conversational AI, document understanding, and LLM deployment. This guide covers the 40 most important NLP interview questions with detailed answers — from classic text preprocessing to modern transformer architectures and large language models.
Text Preprocessing and Representation
Q1. What is tokenisation in NLP and what are the different types?
Tokenisation splits raw text into units (tokens) that a model can process. Word tokenisation splits on whitespace and punctuation — simple but fails on morphologically rich languages and out-of-vocabulary words. Character tokenisation uses individual characters — no OOV problem but very long sequences. Subword tokenisation, the modern standard, splits words into frequent subword units: “unhappiness” → [“un”, “happiness”] or [“un”, “happy”, “##ness”]. Byte-Pair Encoding (BPE) — used by GPT-2, RoBERTa — iteratively merges the most frequent character pair. WordPiece — used by BERT — similar to BPE but merges based on likelihood rather than frequency. SentencePiece — language-agnostic, trains directly on raw text without pre-tokenisation. The key advantage of subword tokenisation is balancing vocabulary size (manageable) against handling rare and unknown words.
Q2. What is the difference between stemming and lemmatisation?
Both reduce words to their base form to group related words. Stemming applies heuristic rules to strip suffixes — fast but crude. “Running” → “run”, “Studies” → “studi” (not a real word). Lemmatisation uses vocabulary and morphological analysis to return the actual dictionary base form (lemma). “Running” → “run”, “Studies” → “study”, “Better” → “good”. Stemming is faster but produces non-words that hurt interpretability. Lemmatisation requires part-of-speech context (is “lying” from “lie” (recline) or “lie” (deceive)?) and is slower. For production NLP, lemmatisation is preferred when interpretability matters; stemming is acceptable for IR tasks like search where exact form matters less than matching.
Q3. What are stop words and when should you remove them?
Stop words are high-frequency, low-information words like “the”, “is”, “at”, “which”. Removing them reduces dimensionality and noise in bag-of-words models and TF-IDF. However, modern contextual models (BERT, GPT) should never have stop words removed — attention mechanisms learn which words are informative in context, and removing stop words before tokenisation destroys sentence structure. Remove stop words for: TF-IDF text classification, keyword extraction, search indexing. Keep stop words for: sentiment analysis (e.g., “not good” — “not” is critical), dependency parsing, neural models, and any task where sentence structure carries meaning.
Q4. What is TF-IDF and how does it work?
TF-IDF (Term Frequency-Inverse Document Frequency) weights words by how important they are to a specific document relative to the entire corpus. TF(term, doc) = frequency of term in that document (often normalised by document length). IDF(term) = log(total documents / documents containing term). TF-IDF = TF × IDF. A word that appears often in one document but rarely across the corpus gets a high score — it is characteristic of that document. Common words like “the” appear in nearly all documents, so IDF ≈ 0, suppressing them. TF-IDF is effective for text classification, information retrieval, and keyword extraction — but it ignores word order and semantic similarity.
Q5. What are word embeddings? Compare Word2Vec, GloVe, and FastText.
Word embeddings are dense vector representations of words in a continuous space where semantically similar words are geometrically close. They capture semantic relationships: “king” – “man” + “woman” ≈ “queen”. Word2Vec (2013, Google): predicts a word from its context (CBOW) or context from a word (Skip-gram). Trained by maximising the probability of observed context words. Produces static embeddings — each word has one vector regardless of context (“bank” in finance vs river has the same vector). GloVe (2014, Stanford): trains on global co-occurrence statistics, factorising the word-context co-occurrence matrix. Often produces better word analogies than Word2Vec. FastText (2016, Facebook): represents words as bags of character n-grams, summing n-gram vectors for the word embedding. Handles morphologically rich languages and out-of-vocabulary words naturally (“unhappiness” uses the “happy”, “un”, “ness” subword vectors).
Q6. What is the problem with static word embeddings and how do contextual embeddings fix it?
Static embeddings (Word2Vec, GloVe) assign a single fixed vector to each word regardless of context. “I went to the bank to deposit money” and “I sat on the river bank” — “bank” gets the same vector in both sentences. This is a fundamental limitation for polysemous words (words with multiple meanings). Contextual embeddings (BERT, GPT, ELMo) produce different vectors for the same word depending on its surrounding context. ELMo (2018) was the first — it used a bidirectional LSTM to generate context-dependent representations. BERT (2018) used the Transformer encoder with self-attention to produce deeply contextual representations. The same token “bank” will have a very different 768-dimensional vector in the financial sentence vs the river sentence, capturing its true meaning in context.
Classical NLP Tasks and Models
Q7. What is named entity recognition (NER) and how is it modelled?
NER identifies and classifies named entities in text — people (PER), organisations (ORG), locations (LOC), dates, monetary values, and more. Classic approach: sequence labelling with BIO (Begin, Inside, Outside) tagging scheme. “Apple announced…” → [B-ORG, O, …]. Traditional models: Conditional Random Fields (CRF) with hand-crafted features. Modern approach: fine-tune a pretrained Transformer (BERT) on labelled NER data. The token classification head outputs a probability distribution over entity labels for each token. BERT-based NER typically achieves 90%+ F1 on standard benchmarks. Challenges: nested entities (a person inside an organisation name), cross-sentence context, domain-specific entities (medical, legal), and entity boundary detection.
Q8. What is sentiment analysis and what are the different levels?
Sentiment analysis determines the emotional tone of text. Document-level: classifies the overall sentiment of a review or article as positive, negative, or neutral. Sentence-level: classifies each sentence separately. Aspect-level (ABSA): identifies sentiment toward specific aspects — “The food was great but the service was terrible” → food=positive, service=negative. Approaches: Rule-based (VADER, SentiWordNet) — uses sentiment lexicons, handles negation and intensifiers via rules, fast and interpretable. ML-based (TF-IDF + Logistic Regression) — trained on labelled data, handles domain-specific language better. Deep learning (fine-tuned BERT) — achieves state-of-the-art on most benchmarks, captures nuanced sentiment. Main challenges: sarcasm and irony, domain adaptation (financial sentiment ≠ movie sentiment), multilingual content.
Q9. What is text summarisation and what are the two main approaches?
Text summarisation condenses long documents into shorter versions retaining key information. Extractive summarisation selects and rearranges existing sentences from the document — no new text is generated. Algorithms rank sentences by importance using TF-IDF, TextRank (graph-based, similar to PageRank applied to sentences), or BERT embeddings similarity to the document centroid. Output is grammatically correct because original sentences are used unchanged. Abstractive summarisation generates new text that may not appear word-for-word in the source — like a human writing a summary. Uses encoder-decoder Transformers (T5, BART, PEGASUS). Produces more fluent, concise summaries but can introduce factual errors (hallucination). Modern production systems often use extractive first, then abstractive on the extracted content.
Q10. What is the difference between text classification and sequence labelling?
Text classification assigns a single label to an entire text or sentence — spam/not-spam, topic category, sentiment. The model reads all tokens and produces one output. Sequence labelling assigns a label to every token in the sequence — NER (each word tagged as entity type or O), part-of-speech tagging (each word tagged as NOUN, VERB, etc.), chunking. The model produces as many outputs as input tokens. The challenge in sequence labelling is that labels of adjacent tokens are not independent — in BIO tagging, an I-ORG token must follow a B-ORG or I-ORG. This dependency is handled by CRF layers on top of the neural encoder, or by the autoregressive structure of generation-based approaches.
Transformer and BERT Questions
Q11. Explain BERT’s training objectives. What is masked language modelling?
BERT is pretrained with two objectives. Masked Language Modelling (MLM): 15% of input tokens are randomly selected. Of those, 80% are replaced with [MASK], 10% with a random token, and 10% left unchanged. The model must predict the original token at masked positions. This forces BERT to learn deep bidirectional representations — it cannot just attend left-to-right. Next Sentence Prediction (NSP): given two sentences A and B, predict whether B is the actual next sentence (50%) or a random sentence (50%). This trains the model on sentence-level relationships. Note: subsequent research showed NSP provides marginal benefit — RoBERTa dropped it and performed better. The 80/10/10 masking scheme is deliberate: predicting [MASK] at inference time (when no masking occurs) would create a training-inference mismatch, so 10% random tokens and 10% unchanged tokens force the model to maintain good representations for all tokens.
Q12. What is the difference between BERT, RoBERTa, DistilBERT, and ALBERT?
BERT (2018, Google): original bidirectional Transformer pretrained on Wikipedia + BookCorpus. 110M parameters (base) or 340M (large). RoBERTa (2019, Facebook): BERT retrained with more data, longer training, larger batches, dynamic masking, and no NSP objective. Typically outperforms BERT on downstream tasks. DistilBERT (2019, Hugging Face): knowledge distillation of BERT — 40% fewer parameters, 60% faster, retains 97% of BERT’s performance. Ideal for production where inference speed matters. ALBERT (2019, Google): shares parameters across Transformer layers (factorised embedding parameterisation + cross-layer parameter sharing), dramatically reducing model size. 12M parameters for ALBERT-base vs 110M for BERT-base, with competitive performance.
Q13. How do you fine-tune a pretrained language model for a downstream task?
Fine-tuning adapts a pretrained model to a specific task with minimal architectural changes. For text classification: add a linear classification head on top of BERT’s [CLS] token representation. For NER: add a token classification head on top of all token representations. For question answering: add start/end position prediction heads. Training: use a small learning rate (2e-5 to 5e-5 for BERT — much smaller than pretraining) to avoid catastrophic forgetting. Train for 2-5 epochs — more risks overfitting. Use linear warmup + linear or cosine decay for the learning rate schedule. With limited data: freeze early layers (only fine-tune top layers and the head); use LoRA for parameter-efficient fine-tuning; apply data augmentation (backtranslation, synonym replacement).
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from transformers import TrainingArguments, Trainer
import torch
model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
def tokenize(batch):
return tokenizer(batch['text'], truncation=True, padding='max_length', max_length=128)
# Assumes dataset has 'text' and 'label' columns
tokenized = dataset.map(tokenize, batched=True)
args = TrainingArguments(
output_dir='./results',
num_train_epochs=3,
per_device_train_batch_size=16,
learning_rate=2e-5,
warmup_ratio=0.1,
evaluation_strategy='epoch',
save_strategy='epoch',
load_best_model_at_end=True,
)
trainer = Trainer(model=model, args=args,
train_dataset=tokenized['train'],
eval_dataset=tokenized['validation'])
trainer.train()
Large Language Models and Modern NLP
Q14. What is the difference between a language model and a conversational AI?
A language model is trained to predict the probability distribution over the next token given context — P(token | previous tokens). This is purely a statistical modelling objective with no explicit notion of helpfulness or instruction-following. A conversational AI is a language model that has been additionally fine-tuned to follow instructions, be helpful, avoid harmful content, and maintain dialogue coherently. This typically involves: Supervised Fine-Tuning (SFT) on curated instruction-following examples; Reward Modelling using human preference data; and Reinforcement Learning from Human Feedback (RLHF) with PPO or Direct Preference Optimisation (DPO) to align the model’s outputs with human preferences. The base GPT-4 is a language model; ChatGPT/Claude are conversational AIs built on top of language models.
Q15. What is hallucination in LLMs and what causes it?
Hallucination refers to an LLM generating plausible-sounding but factually incorrect or fabricated information — names, dates, citations, statistics, or events that do not exist. Causes: language models are trained to produce fluent, contextually coherent text, not necessarily factual text; training data contains errors, biases, and outdated information; models may “pattern-match” on superficially similar training examples rather than recalling verified facts; and autoregressive generation compounds errors (early incorrect tokens influence subsequent generation). Mitigation strategies: Retrieval-Augmented Generation (RAG) — ground responses in retrieved external documents; chain-of-thought prompting — ask the model to reason step by step; output verification — cross-check generated claims against databases; calibration — train models to express uncertainty; and Constitutional AI approaches that train models to refuse when uncertain.
Q16. What is RAG (Retrieval-Augmented Generation)?
RAG combines information retrieval with generation to produce factually grounded responses. Architecture: (1) Index — chunk your knowledge base documents, embed each chunk with a text embedding model (e.g., text-embedding-3-large), and store embeddings in a vector database (Pinecone, Chroma, Weaviate, pgvector). (2) Retrieve — when a user asks a question, embed the query with the same model and retrieve the top-k most similar document chunks using approximate nearest neighbour search. (3) Generate — prepend the retrieved chunks as context to the LLM prompt: “Answer the question based only on the following context: {chunks}. Question: {query}.” RAG dramatically reduces hallucination for knowledge-intensive tasks, allows knowledge to be updated without retraining the LLM, and enables citing specific source documents.
Q17–30 (Rapid fire NLP):
Q17. What is perplexity in language models? Perplexity = exp(-1/N × Σ log P(token_i | context)) — it measures how “surprised” the model is by the test text. Lower perplexity = better language model. A model with perplexity of 50 is on average choosing between 50 equally likely tokens at each step.
Q18. What is attention and why is it O(n²) in memory? Attention computes pairwise similarity between every pair of tokens: Q×K^T produces an n×n attention matrix. For a sequence of length n, this requires O(n²) memory and compute. For a 1000-token sequence: 1M attention scores per head. Solutions: sparse attention (only attend to nearby tokens + global tokens), Flash Attention (reorders computation to avoid materialising the full matrix in HBM), and sliding window attention (Longformer).
Q19. What is beam search in text generation? An approximate search algorithm that keeps the top-k (beam width) most probable partial sequences at each decoding step, rather than greedily choosing the single best token. Produces more coherent outputs than greedy decoding. Larger beam width = better quality but slower. Nucleus (top-p) sampling and temperature scaling are alternatives that add controlled randomness for more diverse generation.
Q20. What is the difference between extractive and generative question answering? Extractive QA finds a span within a given passage that answers the question (BERT-based, span start/end prediction). Generative QA generates an answer freely using an encoder-decoder model (T5, BART) — can synthesise information from multiple passages but may hallucinate.
Q21. What is dependency parsing? Identifying grammatical relationships between words in a sentence (subject, object, modifier). Produces a tree where each word is connected to its head. Used for information extraction, relation extraction, and as features in classical NLP pipelines. spaCy provides fast dependency parsing out of the box.
Q22. What is coreference resolution? Determining which mentions in a text refer to the same entity. “Mary went to the store. She bought milk.” — resolving that “She” refers to “Mary”. Critical for document understanding, chatbots, and reading comprehension. Modelled with mention detection + mention pair scoring.
Q23. What is zero-shot and few-shot learning in NLP? Zero-shot: a model performs a task it was not explicitly trained on by leveraging its pretraining knowledge and a natural language description of the task. Few-shot: the model is given a few examples in the prompt (in-context learning) and generalises from them. GPT-3/4 demonstrated remarkable few-shot ability without any gradient updates.
Q24. What is prompt engineering? Crafting input prompts to elicit desired behaviour from LLMs without changing model weights. Techniques: chain-of-thought (ask to reason step by step), few-shot examples, role assignment (“You are an expert statistician”), output formatting instructions, and system prompts. A well-engineered prompt can dramatically improve task performance.
Q25. What is the difference between semantic search and keyword search? Keyword search matches exact terms (inverted index, BM25). Semantic search embeds the query and documents in the same vector space and retrieves by cosine similarity — finds conceptually related content even with different vocabulary. Most production systems use hybrid search: keyword for recall, semantic for ranking.
Q26. What evaluation metrics are used for NLP tasks? Classification: Accuracy, F1, AUC-ROC. NER: entity-level F1. Summarisation: ROUGE (Recall-Oriented Understudy for Gisting Evaluation) — measures n-gram overlap between generated and reference summaries. Machine translation: BLEU score (precision of n-gram matches). Generation quality: BERTScore (embedding similarity), perplexity. Human evaluation remains gold standard for generation tasks.
Q27. What is sentence-transformers and when do you use it? A library (built on Hugging Face) that provides pretrained models specifically fine-tuned for producing semantically meaningful sentence embeddings via contrastive learning (Siamese networks). Unlike BERT’s [CLS] token, sentence-transformers produce embeddings where cosine similarity directly measures semantic similarity. Essential for semantic search, clustering, and RAG retrieval.
Q28. What is transfer learning in NLP? Using a model pretrained on a large general corpus (Wikipedia, books, web) and fine-tuning it on a smaller task-specific dataset. Dramatically reduces labelled data requirements — BERT can be fine-tuned to high accuracy on text classification with just a few hundred examples. The pretrained weights encode linguistic knowledge that transfers broadly.
Q29. What is the role of attention masks in BERT? BERT processes batches of sequences that may have different lengths. Shorter sequences are padded to match the longest. The attention mask (1 for real tokens, 0 for padding) prevents the model from attending to padding tokens, ensuring padding does not influence representations.
Q30. How does machine translation work today? Modern MT uses Transformer encoder-decoder models (the original “Attention Is All You Need” architecture). The encoder processes the source sentence; the decoder generates the target sentence token by token using cross-attention to the encoder representations. Pretrained multilingual models (mBART, NLLB-200) support 200+ language pairs from a single model.
Conclusion
NLP interviews in 2026 increasingly require knowledge of transformer architectures (BERT, GPT, T5), fine-tuning strategies (LoRA, adapters, full fine-tuning), and production concerns (RAG for hallucination reduction, embedding models for semantic search). At the same time, fundamentals — tokenisation, TF-IDF, NER, sentiment analysis — still appear in every interview. Build hands-on experience by fine-tuning a BERT model on a text classification task and building a simple RAG pipeline with LangChain — these two projects will prepare you for the practical coding rounds that now accompany almost every NLP interview.
