Feature Engineering for Machine Learning – Complete Python Guide 2026

Feature engineering — transforming raw data into meaningful inputs for machine learning models — is often the single biggest lever for improving model performance. Better features beat better algorithms. This comprehensive guide covers every major technique with working Python code.

Why Feature Engineering Matters

A linear model with great features often outperforms a deep neural network with poor features. Features encode domain knowledge that models cannot learn from raw data alone. They also reduce the data needed to train a good model. The difference between a 78% and 92% AUC is usually not a better algorithm — it is a smarter representation of the input data.

Handling Numeric Features

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler

df = pd.read_csv('features.csv')

# ── Scaling ──────────────────────────────────────────────────
# StandardScaler: zero mean, unit variance (use for most models)
scaler = StandardScaler()
df['income_scaled'] = scaler.fit_transform(df[['income']])

# RobustScaler: uses median/IQR — robust to outliers
robust = RobustScaler()
df['income_robust'] = robust.fit_transform(df[['income']])

# ── Log transformation for skewed distributions ──────────────
df['log_income']   = np.log1p(df['income'])        # log(x + 1)
df['sqrt_income']  = np.sqrt(df['income'])

# ── Binning continuous to categorical ────────────────────────
df['age_group'] = pd.cut(df['age'],
                          bins=[0, 18, 35, 55, 100],
                          labels=['teen', 'young_adult', 'adult', 'senior'])

# ── Clipping outliers ────────────────────────────────────────
lower, upper = df['income'].quantile([0.01, 0.99])
df['income_clipped'] = df['income'].clip(lower, upper)

Encoding Categorical Features

from sklearn.preprocessing import LabelEncoder, OrdinalEncoder
from category_encoders import TargetEncoder, BinaryEncoder

# ── One-hot encoding (low cardinality < 10 unique values) ────
df_ohe = pd.get_dummies(df, columns=['city', 'gender'], drop_first=True)

# ── Ordinal encoding (ordered categories) ────────────────────
oe = OrdinalEncoder(categories=[['low', 'medium', 'high', 'very_high']])
df['priority_encoded'] = oe.fit_transform(df[['priority']])

# ── Target encoding (high cardinality, e.g., 100+ categories)
# Replaces category with mean of target — watch for leakage
te = TargetEncoder(cols=['product_id'])
df['product_id_te'] = te.fit_transform(df['product_id'], df['target'])

# ── Binary encoding (medium cardinality, 10-100 unique values)
be = BinaryEncoder(cols=['country'])
df_binary = be.fit_transform(df)

# ── Frequency encoding (simple, powerful for tree models) ────
freq_map = df['city'].value_counts().to_dict()
df['city_freq'] = df['city'].map(freq_map)

Interaction and Polynomial Features

from sklearn.preprocessing import PolynomialFeatures

# ── Manual interaction features (domain-driven) ──────────────
df['income_per_age']     = df['income'] / (df['age'] + 1)
df['spend_rate']         = df['total_spend'] / df['tenure_months'].clip(1)
df['revenue_per_click']  = df['revenue'] / (df['clicks'] + 1)

# ── Automated polynomial features ────────────────────────────
poly = PolynomialFeatures(degree=2, interaction_only=True,
                          include_bias=False)
numeric_cols = ['age', 'income', 'tenure']
poly_features = poly.fit_transform(df[numeric_cols])
poly_df = pd.DataFrame(poly_features,
                       columns=poly.get_feature_names_out(numeric_cols))
df = pd.concat([df, poly_df], axis=1)

Date and Time Features

df['date'] = pd.to_datetime(df['date'])

# Extract components
df['year']        = df['date'].dt.year
df['month']       = df['date'].dt.month
df['day']         = df['date'].dt.day
df['day_of_week'] = df['date'].dt.dayofweek   # 0=Monday
df['quarter']     = df['date'].dt.quarter
df['is_weekend']  = df['day_of_week'].isin([5, 6]).astype(int)
df['is_month_end'] = df['date'].dt.is_month_end.astype(int)

# Cyclical encoding for periodic features (preserves circular nature)
df['month_sin'] = np.sin(2 * np.pi * df['month'] / 12)
df['month_cos'] = np.cos(2 * np.pi * df['month'] / 12)
df['dow_sin']   = np.sin(2 * np.pi * df['day_of_week'] / 7)
df['dow_cos']   = np.cos(2 * np.pi * df['day_of_week'] / 7)

# Time since reference event
df['days_since_signup'] = (df['date'] - df['signup_date']).dt.days

Lag and Rolling Window Features (Time Series)

df = df.sort_values('date')

# Lag features — what happened N days ago
for lag in [1, 7, 14, 30]:
    df[f'sales_lag_{lag}'] = df['sales'].shift(lag)

# Rolling statistics
for window in [7, 14, 30]:
    df[f'sales_rolling_mean_{window}'] =         df['sales'].rolling(window, min_periods=1).mean()
    df[f'sales_rolling_std_{window}'] =         df['sales'].rolling(window, min_periods=1).std()
    df[f'sales_rolling_max_{window}'] =         df['sales'].rolling(window, min_periods=1).max()

# Exponential weighted mean (recent values weighted more)
df['sales_ewm_7'] = df['sales'].ewm(span=7, adjust=False).mean()

Automated Feature Generation with Featuretools

import featuretools as ft

# Define entities and relationships
es = ft.EntitySet(id='customer_data')
es.add_dataframe(dataframe=transactions_df,
                 dataframe_name='transactions',
                 index='transaction_id',
                 time_index='date')
es.add_dataframe(dataframe=customers_df,
                 dataframe_name='customers',
                 index='customer_id')
es.add_relationship('customers', 'customer_id',
                    'transactions', 'customer_id')

# Deep Feature Synthesis — generates hundreds of features automatically
feature_matrix, feature_defs = ft.dfs(
    entityset=es,
    target_dataframe_name='customers',
    max_depth=2,
    agg_primitives=['mean', 'sum', 'count', 'std', 'max', 'min'],
    trans_primitives=['day', 'month', 'year', 'is_weekend']
)
print(f'Generated {len(feature_defs)} features automatically')

Feature Selection

from sklearn.feature_selection import SelectKBest, f_classif, mutual_info_classif
from sklearn.ensemble import RandomForestClassifier

X, y = df.drop('target', axis=1), df['target']

# Mutual information — works for non-linear relationships
mi_scores = mutual_info_classif(X, y, random_state=42)
mi_df = pd.Series(mi_scores, index=X.columns).sort_values(ascending=False)
print(mi_df.head(20))

# Variance threshold — remove near-constant features
from sklearn.feature_selection import VarianceThreshold
vt = VarianceThreshold(threshold=0.01)
X_filtered = vt.fit_transform(X)

# RFECV — recursive elimination with cross-validation
from sklearn.feature_selection import RFECV
rf  = RandomForestClassifier(n_estimators=50, random_state=42)
rfe = RFECV(rf, cv=5, scoring='roc_auc', n_jobs=-1)
rfe.fit(X, y)
selected = X.columns[rfe.support_].tolist()
print(f'Selected {len(selected)} features: {selected}')

Conclusion

Feature engineering is where data science expertise turns raw tables into predictive power. Master the fundamentals — scaling, encoding, interactions, and time features — and you will consistently outperform data scientists who rely solely on model tuning. Use automated tools like featuretools to explore the feature space systematically, then apply domain knowledge to select and refine the features that make intuitive sense. The best features are those that encode real-world relationships your model would otherwise need vastly more data to discover.