How to add a new path to your PYTHONPATH to import your own python modules or packages ?
Examples of how to add a new path to your PYTHONPATH to import your own python modules or packages:
Introduction
For example, I have on my local computer a reperestory called «github_projects» (located in the following path «/Users/John/github_projects») where I stored all my own python modules that I develop:
github_projects/
project_01/
project_02/
project_03/
.
.
.
However, if I try to import a python module from another reperestory on my computer I will get the following error message:
>>> import project_01
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'project_01'
We get this error message here because python doesn't know where to find the python module project_01. A simple solution to check that is to look at sys.path:
returns for example in my case:
['', '/Users/John/anaconda3/lib/python36.zip', '/Users/John/anaconda3/lib/python3.6', '/Users/John/anaconda3/lib/python3.6/lib-dynload', '/Users/John/anaconda3/lib/python3.6/site-packages', '/Users/John/anaconda3/lib/python3.6/site-packages/aeosa']
can also check your PYTHONPATH using os:
>>> import os
>>> os.environ['PYTHONPATH']
Another solution to check your PYTHONPATH is to enter directly
in your terminal (not in your python interpreter!).
Adding a new path to your PYTHONPATH
To add a new path to your PYTHONPATH it is going to depend on the your shell (I used hereafter bash shell ). To get your shell just enter
To temporary add a new path in your PYTHONPATH:
export PYTHONPATH="/Users/John/github_projects"
and you can now start python (in the same window that you entered "export PYTHONPATH="/Users/John/github_projects") and try to import your module:
Another solution to make that more permanently just open the
file and add the following line:
export PYTHONPATH="/Users/John/github_projects"
It will then add automatically "/Users/John/github_projects" to your PYTHONPATH each time you open a new terminal window.
Reloading your own python module
Note: another interessting tool is to be able to reload your python (> 3.4) module:
import importlib
importlib.reload(module)
So you don't need to restart python each time you make some change in your python module for example "project_01". You just need to reload it:
import importlib
importlib.reload(project_01)
References
Benjamin
Greetings, I am Ben! I completed my PhD in Atmospheric Science from the University of Lille, France. Subsequently, for 12 years I was employed at NASA as a Research Scientist focusing on Earth remote sensing. Presently, I work with NOAA concentrating on satellite-based Active Fire detection. Python, Machine Learning and Open Science are special areas of interest to me.
Skills
The initialization of the sys.path module search path¶
A module search path is initialized when Python starts. This module search path may be accessed at sys.path .
The first entry in the module search path is the directory that contains the input script, if there is one. Otherwise, the first entry is the current directory, which is the case when executing the interactive shell, a -c command, or -m module.
The PYTHONPATH environment variable is often used to add directories to the search path. If this environment variable is found then the contents are added to the module search path.
PYTHONPATH will affect all installed Python versions/environments. Be wary of setting this in your shell profile or global environment variables. The site module offers more nuanced techniques as mentioned below.
The next items added are the directories containing standard Python modules as well as any extension module s that these modules depend on. Extension modules are .pyd files on Windows and .so files on other platforms. The directory with the platform-independent Python modules is called prefix . The directory with the extension modules is called exec_prefix .
The PYTHONHOME environment variable may be used to set the prefix and exec_prefix locations. Otherwise these directories are found by using the Python executable as a starting point and then looking for various ‘landmark’ files and directories. Note that any symbolic links are followed so the real Python executable location is used as the search starting point. The Python executable location is called home .
Once home is determined, the prefix directory is found by first looking for python majorversion minorversion .zip ( python311.zip ). On Windows the zip archive is searched for in home and on Unix the archive is expected to be in lib . Note that the expected zip archive location is added to the module search path even if the archive does not exist. If no archive was found, Python on Windows will continue the search for prefix by looking for Lib\os.py . Python on Unix will look for lib/python majorversion . minorversion /os.py ( lib/python3.11/os.py ). On Windows prefix and exec_prefix are the same, however on other platforms lib/python majorversion . minorversion /lib-dynload ( lib/python3.11/lib-dynload ) is searched for and used as an anchor for exec_prefix . On some platforms lib may be lib64 or another value, see sys.platlibdir and PYTHONPLATLIBDIR .
Once found, prefix and exec_prefix are available at sys.prefix and sys.exec_prefix respectively.
Finally, the site module is processed and site-packages directories are added to the module search path. A common way to customize the search path is to create sitecustomize or usercustomize modules as described in the site module documentation.
Certain command line options may further affect path calculations. See -E , -I , -s and -S for further details.
Virtual environments¶
If Python is run in a virtual environment (as described at Virtual Environments and Packages ) then prefix and exec_prefix are specific to the virtual environment.
If a pyvenv.cfg file is found alongside the main executable, or in the directory one level above the executable, the following variations apply:
- If home is an absolute path and PYTHONHOME is not set, this path is used instead of the path to the main executable when deducing prefix and exec_prefix .
_pth files¶
To completely override sys.path create a ._pth file with the same name as the shared library or executable ( python._pth or python311._pth ). The shared library path is always known on Windows, however it may not be available on other platforms. In the ._pth file specify one line for each path to add to sys.path . The file based on the shared library name overrides the one based on the executable, which allows paths to be restricted for any program loading the runtime if desired.
When the file exists, all registry and environment variables are ignored, isolated mode is enabled, and site is not imported unless one line in the file specifies import site . Blank paths and lines starting with # are ignored. Each path may be absolute or relative to the location of the file. Import statements other than to site are not permitted, and arbitrary code cannot be specified.
Note that .pth files (without leading underscore) will be processed normally by the site module when import site has been specified.
Embedded Python¶
If Python is embedded within another application Py_InitializeFromConfig() and the PyConfig structure can be used to initialize Python. The path specific details are described at Python Path Configuration . Alternatively the older Py_SetPath() can be used to bypass the initialization of the module search path.