How to find the statistics and execution times of the code in python
So i need something like this when i run the above code, what i am trying is to write this functionality of printing statistics inside the file test.py instead of running the file with command python -m cProfile test.py from terminal.
At least i want to find the statistics and execution time of the function calculation() when the file runs because in real the function calculation has big functionality that performs some operation.
You probably need profiling (this is a keyword for further research). Also look at the docs and this question.
@chris: Thats what my doubt actually, i dont know why its given down vote, i had doubt about this and approached SO
3 Answers 3
It appears what you are asking is how to the programmatic interface for the timeit module. That is documented here. You’ll need a sample set to calculate statistics, such min, max, average, etc. from, which means running calculate numerous times through the repeat method of the Timeit class included in the timeit module.
timer = timeit.Timer(calculation) results = timer.timeit(10000) Actually we can find execution time by passing the function calucation() inside Timer/timeit right?. Fine but what about statistics like i need all the above result info as i mentioned in the above edited code. That is if we run the file as «python -m cProfile test.py» we willl get the entire output as above(how many function calls etc.,), but i don’t want to run the file with this command, instead when i run the file as «python test.py», it should display all the result as above, that is we need to implement the cprofile functionality inside the test.py file
Yes to your question. In my example above results contains a list of all 10000 timings. The first column in the printed output is the total calls, which is 10000 in my example since that is what was passed in. That second column is total time, which would be the sum of the values in the list, and so on.
I think you are asking how to use cProfile from within the code. It actually turns out to be quite easy. cProfile.Profile has two undocumented methods, enable and disable , which can be used to start and stop the profiler. This means you can easily create a context manager or decorator. The following recipe implements both of these in one object, and includes a way of processing and printing the output with the pstat module.
import cProfile, pstats, functools class profile: def __init__(self, filename=None): """ If defined, output is written to *filename*, otherwise it is processed using *pstats* and printed to STDOUT. """ self._filename = filename self._prof = cProfile.Profile() def __enter__(self): self._prof.enable() def __exit__(self, exc_type, exc_value, traceback): self._prof.disable() if self._filename is not None: self._prof.dump_stats(self._filename) else: stats = pstats.Stats(self._prof) self.print_stats(stats) def print_stats(self, stats): """ This method processes the stats and prints them. """ stats.strip_dirs().sort_stats('cumulative').print_stats(20) def __call__(self, func): self._func = func @functools.wraps(func) def wrapper(*args, **kwargs): with self: return func(*args, **kwargs) return wrapper @profile() def calculation(): a = 2 b = 3 res = a + b return res calculation() with profile('output_file.pstat'): calculation() You can change print_stats as necessary to show the output you want.
How long does my Python application take to run?
Basically, I want to be able to output the elapsed time that the application is running for. I think I need to use some sort of timeit function but I’m not sure which one. I’m looking for something like the following.
START MY TIMER code for application more code more code etc STOP MY TIMER 8 Answers 8
The simplest way to do it is to put:
import time start_time = time.time() print "My program took", time.time() - start_time, "to run" To get the best result on different platforms:
from timeit import default_timer as timer # START MY TIMER start = timer() code for application more code more code etc # STOP MY TIMER elapsed_time = timer() - start # in seconds timer() is a time.perf_counter() in Python 3.3 and time.clock()/time.time() in older versions on Windows/other platforms correspondingly.
If you’re running Mac OS X or Linux, just use the time utility:
$ time python script.py real 0m0.043s user 0m0.027s sys 0m0.013s If not, use the time module:
import time start_time = time.time() # . print time.time() - start_time, 's' You could use the system command time :
foo@foo:~# time python test.py hello world! real 0m0.015s user 0m0.008s sys 0m0.007s I like this, except this will include any «overhead» for imports and initializations you wouldn’t want to test, in the case of timing a snippet.
Take system current time at start of your program, and at the end again take the end time of your program, finally subtract the two time.
Below code depicts how we can perform the above said operation:
import time start = time.time() cnt = 1 for fun in range(10**11): if (fun % 10**9 == 0): print(cnt, fun) cnt += 1 end = time.time() print('Time taken for fun program: ', end - start) Hope this is clear for ever.
Couldn’t you just get the current system time at the start and at the end, then subtract the start time from the final time?
import time as t def TimeTakenDecorator(func): def wraper(*args,**kwargs): start = t.time() func(*args,**kwargs) end = t.time() print('Time taken for fun program: ', end - start) return wraper @TimeTakenDecorator def hello(s): for _ in range(100000000): pass print(s) hello("test") Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
I would use Rad’s answer with the decorator, and the main thing I would change is to make sure to use a monotonic clock, so instead of t.time() use t.monotonic() , or for greater precision t.monotonic_ns() . But this requires that you use python v3.3 or higher, otherwise you may try this ctypes variant of getting a monotonic clock.
Return the value (in fractional seconds) of a monotonic clock, i.e. a clock that cannot go backwards. The clock is not affected by system clock updates. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.
Use monotonic_ns() to avoid the precision loss caused by the float type.
New in version 3.3.
Changed in version 3.5: The function is now always available and always system-wide.
Changed in version 3.10: On macOS, the function is now system-wide.
Similar to monotonic(), but return time as nanoseconds.
New in version 3.7.
>>> import time as t >>> t.time() 1666507313.6913335 >>> t.monotonic_ns() 2346741390948 >>> t.monotonic_ns() 2350236290783 >>> t.monotonic() 2353.851540444 >>> t.monotonic_ns() 2356366587038 >>> t.get_clock_info('time') namespace(implementation='clock_gettime(CLOCK_REALTIME)', monotonic=False, adjustable=True, resolution=1e-09) >>> t.get_clock_info('monotonic') namespace(implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, adjustable=False, resolution=1e-09) >>> >>> t.monotonic() 3475.071162687 >>> t.monotonic_ns()/10**9 3475.877335106