How to expand environment variables in python as bash does?
With os.path.expandvars I can expand environment variables in a string, but with the caveat: «Malformed variable names and references to non-existing variables are left unchanged» (emphasis mine). And besides, os.path.expandvars expands escaped \$ too. I would like to expand the variables in a bash-like fashion, at least in these two points. Compare:
import os.environ import os.path os.environ['MyVar'] = 'my_var' if 'unknown' in os.environ: del os.environ['unknown'] print(os.path.expandvars("$MyVar$unknown\$MyVar")) unset unknown MyVar=my_var echo $MyVar$unknown\$MyVar 7 Answers 7
The following implementation maintain full compatibility with os.path.expandvars , yet allows a greater flexibility through optional parameters:
import os import re def expandvars(path, default=None, skip_escaped=False): """Expand environment variables of form $var and $. If parameter 'skip_escaped' is True, all escaped variable references (i.e. preceded by backslashes) are skipped. Unknown variables are set to 'default'. If 'default' is None, they are left unchanged. """ def replace_var(m): return os.environ.get(m.group(2) or m.group(1), m.group(0) if default is None else default) reVar = (r'(?]*)\>)' return re.sub(reVar, replace_var, path) Below are some invocation examples:
>>> expandvars("$SHELL$unknown\$SHELL") '/bin/bash$unknown\\/bin/bash' >>> expandvars("$SHELL$unknown\$SHELL", '') '/bin/bash\\/bin/bash' >>> expandvars("$SHELL$unknown\$SHELL", '', True) '/bin/bash\\$SHELL' I gave this a test, and it’s not 100% equivalent to what you would get from a bash expansion. In bash, «\$IGNORE» would consume the backslash and return «$IGNORE». Whereas this python implementation would leave it as «\$IGNORE». Just pointing this out in case someone is looking for 1-to-1 bash expansion
re.sub('\$[A-Za-z_][A-Za-z0-9_]*', '', os.path.expandvars(path)) The regular expression should match any valid variable name, as per this answer, and every match will be substituted with the empty string.
Edit: if you don’t want to replace escaped vars (i.e. \$VAR ), use a negative lookbehind assertion in the regex:
(which says the match should not be preceded by \ ).
Edit 2: let’s make this a function:
def expandvars2(path): return re.sub(r'(? >>> print(expandvars2('$TERM$FOO\$BAR')) xterm-256color\$BAR the variable $TERM gets expanded to its value, the nonexisting variable $FOO is expanded to the empty string, and \$BAR is not touched.
I was going to say that this would expand escaped \$ too, which I don't want, but then i realized that os.path.expandvars does the same. Maybe I have to modify the question.
I do not follow. expandvars already replaces $VAR with its value if VAR is an existing env var. the result can only contain more occurrences of $VAR if VAR is not an env variable, which you said you wanted to replace with the empty string, like bash does.
I've modified and expanded the question now, hopefully it's clearer. Your solution would remove $unknown from the result, but leave \my_var where I want $MyVar (unexpanded, because the dollar was escaped).
But now my problem is that os.path.expandvars already expands the \$MyVar I don't want expanded! (and the regexp wouldn't work if it were the backslash what's escaped, as in \\$unknown ).
which python version are you using? on my machine print(os.path.expandvars("$MyVar$unknown\$MyVar")) prints my_var$unknown\$MyVar (check my updated answer)
The alternative solution - as pointed out by @HuStmpHrrr - is that you let bash evaluate your string, so that you don't have to replicate all the wanted bash functionality in python.
Not as efficient as the other solution I gave, but it is very simple, which is also a nice feature 🙂
>>> from subprocess import check_output >>> s = '$TERM$FOO\$TERM' >>> check_output(["bash","-c","echo \"<>\"".format(s)]) b'xterm-256color$TERM\n' P.S. beware of escaping of " and \ : you may want to replace \ with \\ and " with \" in s before calling check_output
I think this is way better. Regular expressions are hard to maintain and debug. I doubt anybody would really feel the inefficiency of spawning a process calling the shell etc. How many times one would be calling (with different s !!) this? If you're calling with same s over and over, caching is the solution, not regular expressions.
Here's a solution that uses the original expandvars logic: Temporarily replace os.environ with a proxy object that makes unknown variables empty strings. Note that a defaultdict wouldn't work because os.environ
For your escape issue, you can replace r'\$' with some value that is guaranteed not to be in the string and will not be expanded, then replace it back.
class EnvironProxy(object): __slots__ = ('_original_environ',) def __init__(self): self._original_environ = os.environ def __enter__(self): self._original_environ = os.environ os.environ = self return self def __exit__(self, exc_type, exc_val, exc_tb): os.environ = self._original_environ def __getitem__(self, item): try: return self._original_environ[item] except KeyError: return '' def expandvars(path): replacer = '\0' # NUL shouldn't be in a file path anyways. while replacer in path: replacer *= 2 path = path.replace('\\$', replacer) with EnvironProxy(): return os.path.expandvars(path).replace(replacer, '$') I have run across the same issue, but I would propose a different and very simple approach.
If we look at the basic meaning of "escape character" (as they started in printer devices), the purpose is to tell the device "do something different with whatever comes next". It is a sort of clutch. In our particular case, the only problem we have is when we have the two characters '\' and '$' in a sequence.
Unfortunately, we do not have control of the standard os.path.expandvars, so that the string is passed lock, stock and barrel. What we can do, however, is to fool the function so that it fails to recognize the '$' in that case! The best way is to replace the $ with some arbitrary "entity" and then to transform it back.
def expandvars(value): """ Expand the env variables in a string, respecting the escape sequence \$ """ DOLLAR = r"\$" escaped = value.replace(r"\$", r"\%s" % DOLLAR) return os.path.expandvars(escaped).replace(DOLLAR, "$") I used the HTML entity, but any reasonably improbable sequence would do (a random sequence might be even better). We might imagine cases where this method would have an unwanted side effect, but they should be so unlikely as to be negligible.
Модуль os.path
os.path является вложенным модулем в модуль os, и реализует некоторые полезные функции для работы с путями.
os.path.abspath(path) - возвращает нормализованный абсолютный путь.
os.path.basename(path) - базовое имя пути (эквивалентно os.path.split(path)[1]).
os.path.commonprefix(list) - возвращает самый длинный префикс всех путей в списке.
os.path.dirname(path) - возвращает имя директории пути path.
os.path.exists(path) - возвращает True, если path указывает на существующий путь или дескриптор открытого файла.
os.path.expanduser(path) - заменяет ~ или ~user на домашнюю директорию пользователя.
os.path.expandvars(path) - возвращает аргумент с подставленными переменными окружения ($name или $ заменяются переменной окружения name). Несуществующие имена не заменяет. На Windows также заменяет %name%.
os.path.getatime(path) - время последнего доступа к файлу, в секундах.
os.path.getmtime(path) - время последнего изменения файла, в секундах.
os.path.getctime(path) - время создания файла (Windows), время последнего изменения файла (Unix).
os.path.getsize(path) - размер файла в байтах.
os.path.isabs(path) - является ли путь абсолютным.
os.path.isfile(path) - является ли путь файлом.
os.path.isdir(path) - является ли путь директорией.
os.path.islink(path) - является ли путь символической ссылкой.
os.path.ismount(path) - является ли путь точкой монтирования.
os.path.join(path1[, path2[, . ]]) - соединяет пути с учётом особенностей операционной системы.
os.path.normcase(path) - нормализует регистр пути (на файловых системах, не учитывающих регистр, приводит путь к нижнему регистру).
os.path.normpath(path) - нормализует путь, убирая избыточные разделители и ссылки на предыдущие директории. На Windows преобразует прямые слеши в обратные.
os.path.realpath(path) - возвращает канонический путь, убирая все символические ссылки (если они поддерживаются).
os.path.relpath(path, start=None) - вычисляет путь относительно директории start (по умолчанию - относительно текущей директории).
os.path.samefile(path1, path2) - указывают ли path1 и path2 на один и тот же файл или директорию.
os.path.sameopenfile(fp1, fp2) - указывают ли дескрипторы fp1 и fp2 на один и тот же открытый файл.
os.path.split(path) - разбивает путь на кортеж (голова, хвост), где хвост - последний компонент пути, а голова - всё остальное. Хвост никогда не начинается со слеша (если путь заканчивается слешем, то хвост пустой). Если слешей в пути нет, то пустой будет голова.
os.path.splitdrive(path) - разбивает путь на пару (привод, хвост).
os.path.splitext(path) - разбивает путь на пару (root, ext), где ext начинается с точки и содержит не более одной точки.
os.path.supports_unicode_filenames - поддерживает ли файловая система Unicode.
Для вставки кода на Python в комментарий заключайте его в теги
Python os.path.expandvars() Method
- Syntax of os.path.expandvars() Method
- Example 1: Use the os.path.expandvars() Method in Python
- Example 2: Use the os.path.expandvars() Method on Windows OS
- Example 3: Use an Unspecified Environment Variable in the os.path.expandvars() Method
Python os.path.expandvars() method is an efficient way of expanding the environment variables in a specified path. All the substrings of the form $name or $ are replaced with the set value of the environment variable name.
Syntax of os.path.expandvars() Method
Parameters
path - an address object of a file system path or a symlink. The object can either be an str or bytes.
Return
The return type of this method is a string value representing the new environment variable after expanding it, based on the input parameter.
Example 1: Use the os.path.expandvars() Method in Python
import os.path path_address = "$HOME/File.txt" new_path = os.path.expandvars(path_address) print(new_path) os.environ["HOME"] = "/home/Documents" path_address = "$HOME/User/File.txt" new_path = os.path.expandvars(path_address) print(new_path) os.environ["USER"] = "programmer" path_address = "/home/$ /File.txt" new_path = os.path.expandvars(path_address) print(new_path) //File.txt /home/Documents/User/File.txt /home/programmer/File.txt In the code above, we have replaced the values of the environment variable HOME and USER with corresponding values.
Example 2: Use the os.path.expandvars() Method on Windows OS
import os.path path_address_1 = R"% HOME %\Documents\File.txt" path_address_2 = R"C:\User\$USERNAME\HOME\File.txt" path_address_3 = R"$ \File.txt" new_path1 = os.path.expandvars(path_address_1) new_path2 = os.path.expandvars(path_address_2) new_path3 = os.path.expandvars(path_address_3) print(new_path1) print(new_path2) print(new_path3)