Dependency conflicts are one of the most frustrating parts of Python development. A project that worked last week breaks because another project updated a shared library. Virtual environments solve this by giving each project its own isolated Python installation. This guide covers every tool you need — from the built-in venv to modern Poetry — so you never fight a dependency conflict again.
Why Virtual Environments?
Python installs packages globally by default. If Project A needs numpy 1.24 and Project B needs numpy 2.0, one of them breaks. Virtual environments create isolated directories with their own Python interpreter and package set. Each project gets exactly what it needs, and changes in one project never affect another. This is not optional best practice — it is the minimum standard for professional Python development.
Built-in venv
# Create a virtual environment
python -m venv myenv
# Activate (Windows)
myenv\Scriptsctivate
# Activate (Mac/Linux)
source myenv/bin/activate
# Your prompt now shows (myenv)
# Install packages — they go into myenv only
pip install numpy pandas scikit-learn
# Freeze dependencies for reproducibility
pip freeze > requirements.txt
# Anyone can recreate your environment exactly
pip install -r requirements.txt
# Deactivate when done
deactivate
# Delete the environment (just delete the folder)
rm -rf myenv
Conda for Data Science
Conda manages both Python versions and non-Python dependencies (C libraries, CUDA). It is the standard for data science and ML because many packages (NumPy, SciPy, PyTorch) depend on C extensions that pip sometimes installs incorrectly.
# Create environment with specific Python version
conda create -n ds-project python=3.11
# Activate
conda activate ds-project
# Install packages — conda resolves dependencies better than pip
conda install numpy pandas scikit-learn matplotlib jupyter
# Install packages not in conda with pip (inside the conda env)
pip install some-package
# Export environment
conda env export > environment.yml
# Recreate from file (cross-platform)
conda env create -f environment.yml
# List all environments
conda env list
# Remove environment
conda env remove -n ds-project
# Useful: update all packages
conda update --all
# environment.yml example
name: ds-project
channels:
- conda-forge
- defaults
dependencies:
- python=3.11
- numpy=1.26
- pandas=2.2
- scikit-learn=1.5
- matplotlib=3.9
- jupyter
- pip:
- langchain==0.2.0
- chromadb==0.5.0
Poetry – Modern Dependency Management
Poetry is the best tool for Python projects that others will install or deploy. It manages dependencies, versions, and packaging in one tool — replacing setup.py, requirements.txt, and pip-tools.
pip install poetry
# Create a new project
poetry new my-project
cd my-project
# Or initialise in an existing directory
poetry init
# Add dependencies
poetry add numpy pandas scikit-learn
poetry add --group dev pytest black ruff
# Install all dependencies
poetry install
# Activate the environment
poetry shell
# Run a command inside the environment without activating
poetry run python train.py
poetry run pytest
# Update dependencies
poetry update
# Show dependency tree
poetry show --tree
# Export to requirements.txt (for Docker or pip users)
poetry export -f requirements.txt --output requirements.txt --without-hashes
# pyproject.toml — generated by Poetry
[tool.poetry]
name = "my-ml-project"
version = "0.1.0"
description = "ML pipeline for churn prediction"
authors = ["Durgesh Kekare "]
[tool.poetry.dependencies]
python = "^3.11"
numpy = "^1.26"
pandas = "^2.2"
scikit-learn = "^1.5"
xgboost = "^2.0"
fastapi = "^0.111"
[tool.poetry.group.dev.dependencies]
pytest = "^8.0"
black = "^24.0"
ruff = "^0.4"
jupyter = "^1.0"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
pip-tools for requirements pinning
pip install pip-tools
# Create requirements.in with top-level deps only
# requirements.in:
# numpy>=1.26
# pandas>=2.0
# scikit-learn>=1.4
# Compile to pinned requirements.txt
pip-compile requirements.in
# Install pinned dependencies
pip-sync requirements.txt
# Upgrade all packages and recompile
pip-compile --upgrade requirements.in
Managing Python Versions with pyenv
# Install pyenv (Mac/Linux)
curl https://pyenv.run | bash
# List available Python versions
pyenv install --list | grep 3.11
# Install a specific version
pyenv install 3.11.9
# Set global default
pyenv global 3.11.9
# Set local version for a project (creates .python-version file)
cd my-project
pyenv local 3.10.14
# Check active version
python --version
pyenv version
Best Practices
Never install packages globally (outside a virtual environment). Always commit requirements.txt or pyproject.toml to git — never commit the environment folder itself (add it to .gitignore). Pin exact versions in production (numpy==1.26.4) and use flexible ranges in libraries (numpy>=1.26). Use conda when working with GPU packages or C extensions. Use Poetry when building packages others will install. Use plain venv when you want minimal tooling and full control.
Conclusion
The 20 minutes spent learning environment management will save you hundreds of hours of debugging mysterious import errors and version conflicts. Start with venv for simple projects, conda if you are doing GPU deep learning, and Poetry for anything you plan to share or deploy. Whatever you choose, the non-negotiable rule is always use an environment — never install to the global Python.
