How to terminate a python subprocess launched with shell=True
The command keeps running in the background, so I was wondering how can I actually terminate the process. Note that when I run the command with:
p = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE) What does your cmd look like? It might contain a command which triggers several processes to be started. So it’s not clear which process you talk about.
11 Answers 11
Use a process group so as to enable sending a signal to all the process in the groups. For that, you should attach a session id to the parent process of the spawned/child processes, which is a shell in your case. This will make it the group leader of the processes. So now, when a signal is sent to the process group leader, it’s transmitted to all of the child processes of this group.
import os import signal import subprocess # The os.setsid() is passed in the argument preexec_fn so # it's run after the fork() and before exec() to run the shell. pro = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True, preexec_fn=os.setsid) os.killpg(os.getpgid(pro.pid), signal.SIGTERM) # Send the signal to all the process groups our testing sugggests that setsid != setpgid, and that os.pgkill only kills subprocesses that still have the same process group id. processes that have changed process group are not killed, even though they may still have the same session id.
I would not recommend doing os.setsid(), since it has other effects as well. Among others, it disconnects the controlling TTY and makes the new process a process group leader. See win.tue.nl/~aeb/linux/lk/lk-10.html
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True) p.kill() p.kill() ends up killing the shell process and cmd is still running.
I found a convenient fix this by:
p = subprocess.Popen("exec " + cmd, stdout=subprocess.PIPE, shell=True) This will cause cmd to inherit the shell process, instead of having the shell launch a child process, which does not get killed. p.pid will be the id of your cmd process then.
I don’t know what effect this will have on your pipe though.
Very nice solution. If your cmd happens to be a shell script wrapper for something else, do call the final binary there with exec too in order to have only one subprocess.
This is beautiful. I have been trying to figure out how to spawn and kill a subprocess per workspace on Ubuntu. This answer helped me. Wish i could upvote it more than once
@speedyrazor — Does not work on Windows10. I think os specific answers should be clearly marked as such.
If you can use psutil, then this works perfectly:
import subprocess import psutil def kill(proc_pid): process = psutil.Process(proc_pid) for proc in process.children(recursive=True): proc.kill() process.kill() proc = subprocess.Popen(["infinite_app", "param"], shell=True) try: proc.wait(timeout=3) except subprocess.TimeoutExpired: kill(proc.pid) I think get_children() should be children(). But it did not work for me on Windows, the process is still there.
@Godsmith — psutil API has changed and you’re right: children() does the same thing as get_children() used to. If it doesn’t work on Windows, then you might want to create a bug ticket in GitHub
from subprocess import Popen process = Popen(command, shell=True) Popen("TASKKILL /F /PID /T".format(pid=process.pid)) it killed the cmd.exe and the program that i gave the command for.
Or use the process name: Popen(«TASKKILL /F /IM » + process_name) , if you don’t have it, you can get it from the command parameter.
When shell=True the shell is the child process, and the commands are its children. So any SIGTERM or SIGKILL will kill the shell but not its child processes, and I don’t remember a good way to do it. The best way I can think of is to use shell=False , otherwise when you kill the parent shell process, it will leave a defunct shell process.
In order to avoid zombie processes one can terminate the process then wait and then poll the process to be sure that they are terminated, as described by «@SomeOne Maybe» in the following stackoverflow answer: stackoverflow.com/questions/2760652/…
None of these answers worked for me so Im leaving the code that did work. In my case even after killing the process with .kill() and getting a .poll() return code the process didn’t terminate.
Following the subprocess.Popen documentation:
«. in order to cleanup properly a well-behaved application should kill the child process and finish communication. «
proc = subprocess.Popen(. ) try: outs, errs = proc.communicate(timeout=15) except TimeoutExpired: proc.kill() outs, errs = proc.communicate() In my case I was missing the proc.communicate() after calling proc.kill() . This cleans the process stdin, stdout . and does terminate the process.
@mouad and Bryant answers are ok in my case if you use as you said the communicate after calling kill. Thanks.
As Sai said, the shell is the child, so signals are intercepted by it — best way I’ve found is to use shell=False and use shlex to split the command line:
if isinstance(command, unicode): cmd = command.encode('utf8') args = shlex.split(cmd) p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) Then p.kill() and p.terminate() should work how you expect.
In my case it doesn’t really help given that cmd is «cd path && zsync etc etc». So that actually makes the command to fail!
The ability to change the working directory for the child process is built-in. Just pass the cwd argument to Popen .
Send the signal to all the processes in group
self.proc = Popen(commands, stdout=PIPE, stderr=STDOUT, universal_newlines=True, preexec_fn=os.setsid) os.killpg(os.getpgid(self.proc.pid), signal.SIGHUP) os.killpg(os.getpgid(self.proc.pid), signal.SIGTERM) There is a very simple way for Python 3.5 or + (Actually tested on Python 3.8 )
import subprocess, signal, time p = subprocess.Popen(['cmd'], shell=True) time.sleep(5) #Wait 5 secs before killing p.send_signal(signal.CTRL_C_EVENT) Then, your code may crash at some point if you have a keyboard input detection, or sth like this. In this case, on the line of code/function where the error is given, just use:
try: FailingCode #here goes the code which is raising KeyboardInterrupt except KeyboardInterrupt: pass What this code is doing is just sending a » CTRL + C » signal to the running process, what will cause the process to get killed.
Note: this is Wndows-specific. There is no CTRL_C_EVENT defined in Mac or Linux implementations of signal . Some alternative code (which I have not tested) can be found here.
Solution that worked for me
if os.name == 'nt': # windows subprocess.Popen("TASKKILL /F /PID /T".format(pid=process.pid)) else: os.kill(process.pid, signal.SIGTERM) Full blown solution that will kill running process (including subtree) on timeout reached or specific conditions via a callback function. Works both on windows & Linux, from Python 2.7 up to 3.10 as of this writing.
Install with pip install command_runner
from command_runner import command_runner # Kills ping after 2 seconds exit_code, output = command_runner('ping 127.0.0.1', shell=True, timeout=2) Example for specific condition: Here we’ll stop ping if current system time seconds digit is > 5
from time import time from command_runner import command_runner def my_condition(): # Arbitrary condition for demo return True if int(str(int(time()))[-1]) > 5 # Calls my_condition() every second (check_interval) and kills ping if my_condition() returns True exit_code, output = command_runner('ping 127.0.0.1', shell=True, stop_on=my_condition, check_interval=1) How to Terminate Python Subprocess
Python allows you to run system commands using subprocesses. They allow you to spawn new processes, connect to their input/output/error and get their return codes. Subprocesses are very useful to run shell commands from Python. But sometimes you may need to terminate a running subprocess. In this article, we will learn how to terminate python subprocess.
For this purpose, we will create a process group that allows you to send signals to all processes in process group. To do this, we will attach a session id to the parent process of the shell process (subprocess). This will make it the leader of the group of processes. Thereafter, when you send a signal to the process group leader it will be sent to all processes in the group.
How to Terminate Python Subprocess
Here is a python command generally used to create subprocess.
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
Instead of creating the subprocess using above command, we slightly modify to be it as following.
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True, preexec_fn=os.setsid)
In the above command, the os.setsid() is passed in the argument preexec_fn so it is run after the fork() but before exec() to run the shell.
Once you create subprocess with the above command, you can always refer to it using its id attribute as p.id and send a SIGTERM signal to it.
Here is the full code to terminate a subprocess created in python.
import os import signal import subprocess pro = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True, preexec_fn=os.setsid) os.killpg(os.getpgid(pro.pid), signal.SIGTERM)
In the above command, we use killpg command to send the terminate signal to all the process groups.
In this article, we have learnt how to terminate python subprocess using process group. You can modify these commands as per your requirement.