Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I have some Python code that I want to debug with perf. For that purpose I want to use subprocess. The following command returns instruction-related information of a process until the command is exited via Ctrl^C.

perf stat -p <my_pid>
Now, I want to run this inside a Python code in background, until some point where I want to be able to terminate its operation and print the commands output. To show what I mean:
x = subprocess.call(["perf","stat","-p",str(GetMyProcessID())])
.. CODE TO DEBUG ..
print x   # I want to terminate subprocess here and output 'x'
Now, I want to determine what to do at the line of 'print x' to terminate the process and check the output.
Any idea/help is appreciated.
Cheers and thanks in advance,
Use subprocess.Popen to run perf. Then, use pipe.communicate() to send input and get the process's output.
After you've done, call pipe.terminate() to terminate the process.
For example:
pipe = subprocess.Popen(["perf","stat","-p",str(GetMyProcessID())], stdout=PIPE)
pipe.terminate()
stdout, stderr = pipe.communicate()
print stdout
                Hello, I want to send pipe = subprocess.Popen(["perf","stat","-p",str(GetMyProcessID())]) and output before terminated. I dont have to send any input command but Popen. Now the program gets stuck at  stdout, stderr = pipe.communicate(None) line. What am I supposed to do? Thanks
– mozcelikors
                Jan 15, 2017 at 10:22
                @mozcelikors I should have used stdout=PIPE as a parameter to subproces.Popen. I update the answer.
– Shmuel H.
                Jan 15, 2017 at 16:17
                but still the pipe.communicate is blocking. I need something nonblocking. Or simply I want to get the output when pipe is terminated. So it should run when I call it, but the output should return as I terminate it.
– mozcelikors
                Jan 15, 2017 at 18:16
                I just tried it, then the output is not there since the pipe had been terminated. I just read that Popen.communicate does not return until the process has terminated.
– mozcelikors
                Jan 15, 2017 at 18:22
First: I advise against calling perf from within your python process (as you see in the complexity of the task below), but instead use is from the command line:
sudo perf stat -- python test.py
If you really want to call perf from within python then you'll face some tricky problems:
to terminate perf and make it output the gathered performance stats you need to send it the SIGINT signal (try it out with sudo perf stat -p mypid: ctrl-\ will print nothing whereas ctrl-c will)
you need to capture stderr as perf sends its output to stderr (at least in my version)
you need to use fork() with one process sending SIGINT and the other reading it's output while the process dies. Without forks it won't work because after you SIGINTed the perf process you cannot read from stdin any more as the process is already gone, and when you read from stdin first you won't get any output until perf is correctly terminated.
That means you'd end up with this python program:
import subprocess
import os
import signal
import time
perf = subprocess.Popen(['perf', 'stat',  '-p', str(os.getpid())], stderr=subprocess.PIPE)
# <-- your code goes here
if os.fork() == 0:
    # child
    time.sleep(1)  # wait until parent runs `stderr.read()`
    perf.send_signal(signal.SIGINT)
    exit(0)
# parent
print("got perf stats>>{}<<".format(perf.stderr.read().decode("utf-8")))
The time.sleep(1) bit is ugly, what it does it that it will but I guess it will do the trick for 99% of the cases. It has almost no influence on the perf data, the only influence it has is on the "total runtime" (*xx seconds time elapsed)
                This seems like its really close to what I would want. One question though: Since perf takes stats dynamically, any delay in the system such as time.sleep increases the number of instructions (therefore,error) in the output. Is there any way to do this without sleep?
– mozcelikors
                Jan 16, 2017 at 9:45
                Another thing is that I have perf output but with a Fatal IO error 11. Should I expect this error ?
– mozcelikors
                Jan 16, 2017 at 9:46
                @mozcelikors yes, you could do so with threads, I'll have a look later because I'm curious. Btw: doing a sleep has hardly any influence on the perf data, the only influence it has is on the "total runtime" *xx seconds time elapsed)
– hansaplast
                Jan 16, 2017 at 11:03
                @mozcelikors with threads it would get a lot more complex, plus you still had the problem of synchronizing the two threads. I would go with the sleep solution, as it does not pollute your stats (you could try with sleep(0.1))
– hansaplast
                Jan 16, 2017 at 15:58
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.