📋 KEY INSIGHTS
- Machine learning for drug discovery applies predictive modelling to accelerate the identification, optimisation, and development of drug candidates. The drug discovery pipeline has four stages where ML adds value: target identification (discovering biological targets whose modulation treats a disease), hit identification (finding molecules that bind the target), lead optimisation (improving binding affinity, selectivity, and drug-likeness of hit compounds), and ADMET prediction (predicting absorption, distribution, metabolism, excretion, and toxicity properties before synthesis). ML reduces the time and cost of each stage by enabling virtual screening of billions of compounds without wet-lab synthesis.
- Molecular property prediction is the core supervised learning problem in drug discovery: given a molecule represented as a SMILES string, graph, or fingerprint, predict its biological activity (binding affinity to a target protein), physicochemical properties (solubility, lipophilicity, permeability), or ADMET properties (CYP inhibition, hERG toxicity, human intestinal absorption). Graph Neural Networks have become the dominant architecture for molecular property prediction because they operate directly on the molecular graph (atoms as nodes, bonds as edges) without hand-crafted feature engineering, learning task-relevant atomic and bond representations end-to-end. The connection to our GNN guide is direct: molecular GNNs (Message Passing Neural Networks, Directed Message Passing Neural Networks, Graph Transformer) are the most practically impactful GNN application.
- Molecular fingerprints (Morgan/ECFP fingerprints, MACCS keys, RDKit descriptors) are fixed-length binary vectors encoding structural features of a molecule: ECFP (Extended Connectivity Fingerprints) encode the atomic environment at each atom up to radius r by hashing the circular neighbourhood of each atom into a bit in a fixed-length vector. ECFP fingerprints combined with gradient boosting (XGBoost, Random Forest) remain competitive with GNNs for ADMET property prediction on small datasets, and their simplicity (no message-passing training required) makes them the production baseline for most pharmaceutical companies. The connection to our gradient boosting guide is direct: XGBoost on ECFP2048 fingerprints is the go-to baseline for new ADMET modelling tasks.
- Generative models for molecular design create novel drug-like molecules with desired properties by learning the distribution of known bioactive compounds. VAEs with SMILES encoders (CVAE), graph-based VAEs (JT-VAE with junction tree decomposition), flow models, and diffusion models have all been applied to molecular generation. Reinforcement learning fine-tuning of generative models — using a reward function encoding molecular property predictions (docking score, QED drug-likeness, synthetic accessibility) to guide generation toward the desired property profile — is the standard approach for property-conditioned de novo design. REINVENT (AstraZeneca) is the most widely deployed RL-based molecular generation tool in pharmaceutical research.
- Protein structure prediction by AlphaFold2 (DeepMind, 2021) represented a paradigm shift for structure-based drug discovery: reliable predicted structures for virtually any target protein are now freely available, enabling structure-based virtual screening (docking small molecules into predicted binding pockets) without the previously mandatory and expensive experimental structure determination. AlphaFold2 uses multiple sequence alignments (MSA) as evolutionary constraints, attention-based Evoformer layers to jointly process sequence and pair representations, and an invariant point attention structure module to produce full 3D coordinates. The connection to our Transformers guide is direct: AlphaFold2 is a large Transformer operating on protein sequence and structure space, using attention mechanisms adapted for the geometric constraints of protein folding.
- The data challenges of drug discovery ML distinguish it from other ML domains: datasets are small (hundreds to low thousands of compounds per assay, compared to millions in computer vision), highly imbalanced (1-5% hit rate in high-throughput screening), and subject to activity cliffs (small structural changes causing large property changes that break smooth molecular similarity assumptions). Strategies include: multi-task learning across related assays to share statistical strength (connecting to our transfer learning guide), uncertainty quantification to flag predictions with high epistemic uncertainty (connecting to our Bayesian deep learning guide), and careful train/test splitting by scaffold (Butina clustering) rather than random to properly estimate generalisation to structurally novel compounds.
The traditional drug discovery process — identifying a drug candidate from initial target hypothesis to preclinical development — takes 5-10 years and costs hundreds of millions of dollars, with a high failure rate due to poor ADMET properties, insufficient efficacy, or unexpected toxicity discovered late in development. Machine learning accelerates this process by enabling virtual screening (scoring millions of compounds computationally before synthesis), predicting ADMET properties from molecular structure alone, and generating novel molecules with desired property profiles. This guide covers the full ML drug discovery toolkit: molecular representations (fingerprints, graph representations, SMILES), molecular property prediction with GNNs, ADMET prediction, generative models for de novo drug design, protein structure prediction (AlphaFold2), structure-based virtual screening, and the data challenges specific to pharmaceutical ML.
Molecular Representations and Graph Neural Networks
The choice of molecular representation determines what ML models can be applied and what structural information is available for learning. SMILES (Simplified Molecular Input Line Entry System) is a string representation of molecular structure — e.g. aspirin is CC(=O)Oc1ccccc1C(=O)O — enabling direct application of NLP models (Transformer on SMILES tokens, BERT pre-trained on SMILES, RNN for sequence generation). The key limitation: SMILES encodes the same molecule in multiple equivalent forms (the same graph has many valid SMILES strings), requiring canonicalisation and augmentation during training. InChI and SELFIES are alternative string representations addressing some SMILES limitations. Graph representations model molecules as undirected graphs with atoms as nodes (features: atomic number, charge, hybridisation, aromaticity, ring membership) and bonds as edges (features: bond type, ring membership, stereochemistry) — a natural input for GNNs that respects the inherent graph structure of molecules. Message Passing Neural Networks (MPNNs, Gilmer et al., 2017) are the foundational molecular GNN: in each message passing step, atom hidden states are updated by aggregating messages from neighbouring atoms and bonds: h_v^{(k+1)} = U_k(h_v^{(k)}, Σ_{u ∈ N(v)} M_k(h_v^{(k)}, h_u^{(k)}, e_{uv})), where U is the update function, M is the message function, and e_{uv} is the bond feature. After K message passing steps, a readout function (sum/mean pooling followed by an MLP) maps the atom-level representations to a molecular-level property prediction. The connection to our GNN guide is direct: MPNNs are a molecular-domain specialisation of the general MPNN/GNN framework; Directed MPNN (DMPNN) used in Chemprop improves upon undirected MPNN by separately encoding forward and reverse bond directions, reducing information leakage through the graph. The connection to our Graph Attention Networks guide is direct: Attentive FP and graph Transformer models apply attention to molecular graphs, allowing the model to focus on the most chemically relevant substructures for each property prediction task.
| Representation | Model Family | Strengths | Limitations | Key Tool |
|---|---|---|---|---|
| ECFP fingerprints | XGBoost, Random Forest | Fast, interpretable, no training | Fixed-radius, loses 3D info | RDKit, scikit-learn |
| Molecular graph (2D) | MPNN, DMPNN, GATv2 | Learned features, end-to-end | No 3D geometry | Chemprop, PyG |
| 3D conformer | SchNet, DimeNet, SE(3)-Transformer | Full 3D geometry; rotation-equivariant | Conformer generation needed | DGL-LifeSci |
| SMILES string | Transformer, RNN | Pre-trained models; generation | Non-canonical; no geometry | HuggingFace ChemBERTa |
| Protein sequence | ESM-2, ProtTrans | Evolutionary information | Sequence only (no structure) | ESM-2 (Meta AI) |
Generative Chemistry, AlphaFold2 and ADMET Prediction
ADMET (Absorption, Distribution, Metabolism, Excretion, Toxicity) prediction is the most commercially deployed ML application in drug discovery: pharmaceutical companies use ADMET prediction models to filter out compounds with poor drug-likeness before synthesis, dramatically reducing the number of wet-lab experiments required. The key ADMET endpoints: aqueous solubility (poor solubility reduces bioavailability), lipophilicity (logP — high lipophilicity causes non-specific binding and toxicity), membrane permeability (Caco-2, MDCK — predicts intestinal and blood-brain barrier penetration), CYP enzyme inhibition (predicts drug-drug interactions), hERG channel inhibition (predicts cardiotoxicity), and plasma protein binding. The ChEMBL and PubChem databases contain millions of measured ADMET values enabling training of multi-task GNNs across all ADMET endpoints simultaneously, achieving better performance on individual endpoints through shared representations than single-task models — a direct application of our transfer learning principle: tasks with related input distributions (all are molecular property prediction) share a molecular representation learned jointly. AlphaFold2 predicted structures for the entire UniProt database (~200M proteins) are freely available via the AlphaFold Protein Structure Database, fundamentally changing structure-based virtual screening: rather than requiring expensive and sometimes impossible experimental crystal structures, medicinal chemists can now dock compound libraries into AlphaFold2-predicted binding pockets for virtually any human target protein. The molecular docking tools Glide, AutoDock-Vina, and Gnina (a CNN-based scoring function) score thousands of candidate compounds per hour against a target structure, providing a computational filter that reduces wet-lab high-throughput screening costs by 10-100×. The connection to our foundation models guide is direct: ESMFold (Meta AI) uses ESM-2 protein language model representations to predict protein structure 60× faster than AlphaFold2 by eliminating the MSA computation step, enabling rapid structure prediction in tight drug discovery timelines. Generative molecular design connects directly to our diffusion models guide: DiffSBDD and TargetDiff apply 3D diffusion models to structure-based drug design, generating molecules in 3D space that are conditioned on the target protein binding pocket, producing compounds geometrically complementary to the pocket without separate docking evaluation.
✦ SUMMARIZE THIS ARTICLE WITH AI
ChatGPTGraph Neural Networks for molecular property prediction are in our GNN guide and Graph Attention Networks guide. Bayesian deep learning for uncertainty quantification in ADMET models is in our Bayesian Deep Learning guide. Diffusion models for 3D molecular generation are in our Diffusion Models guide. Foundation models (ESM-2 protein language models, ESMFold) are in our Foundation Models guide. Transfer learning and multi-task learning for ADMET multi-endpoint modelling are in our Transfer Learning guide. Data science for healthcare (clinical trial analytics, biomarker modelling) is in our Healthcare guide.
