Immediate calculation of CPU statistics

[giampaolo]

From akanas...@gmail.com on September 17, 2009 08:49:05

What steps will reproduce the problem?  
1. Create object Process
2. immediately (in the same time) call get_cpu_percentage() function
After it You will get result (0.0 0.0). It is because internal logic of
this class.

What i want is ability to get this info immediately after creation object
Process. In my last program i needed to do some additional work to get all
my object Process exist all time i need this info.

Main idea is to store variables like _last_sys_time in external storage
indexed by PID. When object of class Process is initialized it gets this
info from this storage. And this object will be able to return statistics
immediately after creation. 
There is a problem with storing variables like _last_sys_time in global
storage. I don't know when it is possible to do it. I had wanted to use
procedure __del__ of class but i read that this procedure can be never called.

The second variant i can offer is sigle instance of object Process for each
PID. When user wants to get new object for the same PID he will get only
link to existing object Process. This way we get ability to calculate
statistics any time.

Original issue: http://code.google.com/p/psutil/issues/detail?id=65

[giampaolo]

From jlo...@gmail.com on February 01, 2010 08:49:26

> Main idea is to store variables like _last_sys_time in external storage
> indexed by PID. When object of class Process is initialized it gets this
> info from this storage. And this object will be able to return statistics
> immediately after creation.

The Process class initializes the per-process CPU time stats when the object is first
created. The only other way to do this would be to create a table of process times at
the import of the entire module. Even so, using a separate storage class only helps
if you aren't creating Process objects right away, e.g. 

    import time
    import psutil

    time.sleep(1)
    myProc = psutil.Process(1234)
    print myProc.get_cpu_percent()

In other words, you still need to wait a given amount of time before you can
calculate CPU percentage. The only thing you gain by a separate storage class is that
it "starts the clock" at the import of the module rather than the creation of the
object. This might be a slight help in some use cases, but I'm not sure that it would
be worth the amount of effort required to create and maintain a separate table of
process times.

The main limiting factor is the resolution we're getting on the various CPU and
system time measurements, making it very hard to calculate a meaningful value for
smaller time periods. For example: 

    >>> for i in range(0,10):
    ...     print psutil.cpu_times()
    ... 
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13
    system=327237.97; idle=2680105.06; user=332732.58; nice=2559.13

Note that even in a 10-iteration loop, the times aren't incremented because the
resolution isn't high enough. The same thing happens even with a 100 iterations on my
machine. If we can get higher resolution times for CPU times and system time we could
potentially calculate the CPU time much sooner, perhaps even enough to make it work
immediately after instantiation.

Summary: Immediate calculation of CPU statistics
Labels: -Type-Defect Type-Enhancement OpSys-All

[giampaolo]

From akanas...@gmail.com on February 01, 2010 09:21:40

i know about time i should wait for proper calculation.

i wrote program for monitoring of CPU and memory usage by whole system of programs. 
Main difficulty was the fact that some programs can start childs and exit. So every 
step i needed to go throw the list of current running processes and get info from 
every found. So it was quite difficult to have objects of class Process exist between 
steps of my program. I resolved it. But one global storage for statistics will help 
much in situations like this.

[giampaolo]

From g.rodola on June 21, 2010 13:03:56

As Jay stated, maintaining a separate process table wouldn't really solve your problem since you would still have to wait some time.
Plus, that would mean starting a poller in a thread or a separate process which 
keeps checking for new and "gone" processes which is something I don't particularly like.
Closing this out as won't fix.

Status: WontFix

[giampaolo]

From akanas...@gmail.com on June 21, 2010 15:45:42

ok. as you wish :)
i really implemented method you described in my tool. and all was the same you said. After all it works good.

[giampaolo]

From g.rodola on June 21, 2010 18:36:53

If that's open source I'd like to take a look at the code, just out of personal curiosity.
Would it be possible?

[giampaolo]

From akanas...@gmail.com on June 22, 2010 02:28:46

this is internal tool for testing purposes in our company. it doesn't contain 
any copyrights so i think you can use this code for free. i would be happy if your package contain some parts of this code.

and i hope this code will not make you laught :) i'm not as good in Python.

Attachment: README.txt start_monitor.py

[giampaolo]

From g.rodola on June 22, 2010 03:16:11

Thanks.

> and i hope this code will not make you laught :) i'm not as good in Python.

It doesn't look bad to me.

[giampaolo]

From g.rodola on June 09, 2011 15:33:50

Labels: -OpSys-All