High cpu in Client when looping through list - VisualCron - Forum

Community forum

Jon Tofte-Hansen
2018-01-22T16:44:38Z
Hi

I have a two task loop in a job iterating through a list of ids. The first task reads in a database and the second task writes the output to a file. This works fine but I have noticed that an open Client - remote or not (showing or not showing the beautiful sight of a loop task processing thousands of files) - uses about 50% of the Client hosts cpu while the loop runs!

Now, I may not be running the latest OS but I find it strange, that the hit is that hard.

Client host:
CPU: Virtual Intel Xeon E5-2620 2GHz*2
RAM: 8GB
Host os: Windows Server 2008 R2 Datacenter SP1 64-bit - fully updated

Are anyone else experiencing the same? Maybe VisualCron should tune the clients resource use a bit?
Support
2018-01-24T14:37:59Z
Many servers do not have the graphic cards for the X number of changes that occur in realtime. It is recommended that you connect remotely from your desktop machine OR turn of the realtime progress update in the Client on your server.
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-25T09:23:31Z
Thank you for answering this.

Sometimes it is desirable to run the client either from the server host or from a terminal host and not your local workstation.

I went to [File]/[Settings]/[Client settings]/[Grid]/[Main grid updates] and unticked "Update Task/Job progress in real time" and tested a loop (a read file task after a list files task). The progress were not shown now and the execution time was not updated now (not good), but the CPU still went up to 50%. I ticked "Delay/Cache update" (3 seconds) and ran the test job again. When the list files task finished the client froze with black screen and I had to kill the process!

I setup a VisualCron client on my local workstation and tested the loop. On that machine the cpu only went up to 40% which is still very high for a rather simple graphical rendering. But I must say that most tasks are much quicker on a local installation of the client.

Comment: When creating a new server connection in the client you have two credential options: AD or internal. In our case the VisualCron server is in another AD than my workstation. I miss an option to use my credentials on that AD to logon to the server.
Support
2018-01-25T09:58:14Z
Can you create a sample Job using the Wait Task that reproduces this?

How many times do you loop?
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-25T10:11:21Z
I tried the task [Interactivity]/[Wait] in the loop without any delay values. I now run from my local workstation. The result is the same. Rather high cpu usage for a simple graphical task. I iterate through a list of 1000 files.

Forgot to post my version: Client and Server 8.3.2.
Support
2018-01-25T10:14:19Z
Originally Posted by: Jon Tofte-Hansen 

I tried the task [Interactivity]/[Wait] in the loop without any delay values. I now run from my local workstation. The result is the same. Rather high cpu usage for a simple graphical task. I iterate through a list of 1000 files.

Forgot to post my version: Client and Server 8.3.2.



There are a lot of graphical events. A workaround is to add a 0,3 second wait in the loop. Execution time will be longer but it will be less updates each second. We will see if we can optimize this.
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-25T10:20:57Z
Hmm. The "Sleep/Wait in each iteration" only have seconds as shortest time, and I cannot set the value to 0,3 (it becomes 0). 0.3 becomes 3 seconds.

Thank you for looking into this. The issue is obviously not critical, but still a nuisance.
Support
2018-01-25T10:22:21Z
Yes, I ws suggesting a Task. But if you use 1 second - how does it affect the CPU usage?
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-25T10:28:11Z
This of course eliminates the high cpu usage.
Support
2018-01-25T10:43:38Z
Originally Posted by: Jon Tofte-Hansen 

This of course eliminates the high cpu usage.



Do you have Extended debugging on in Server settings? If yes, turn it off and turn the wait off and let us know if it gets better.



Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Support
2018-01-25T11:00:08Z
After some investigation it seems that the fact that we log to file in two places causes more CPU usage. One of the places can currently be turned off if you turn of Extended debugging so you should see result. The other cannot in current version. However, in this build you can:

https://www.visualcron.c....aspx?g=posts&t=7786 

I suggest you download and install it.
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-25T12:11:19Z
Thank you.

I do not use "Extended debug logging". It is unticked in [Server]/[Main settings]/[Settings]/[Log settings]/[Settings].

As stated it is not very important, so I will wait for the final release.
Support
2018-01-25T13:26:41Z
Originally Posted by: Jon Tofte-Hansen 

Thank you.

I do not use "Extended debug logging". It is unticked in [Server]/[Main settings]/[Settings]/[Log settings]/[Settings].

As stated it is not very important, so I will wait for the final release.



It would be interesting though if it solves your problem before releasing. Because when we turned it off we noticed a difference.


Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-26T09:44:28Z
I have tested 8.3.3.

I have created a job with a list file task and a read file task with a loop iterating the list of 1000 files.

Client on virtual test host: 40-50% CPU use.
Client on local workstation: ~25% CPU use.
Support
2018-01-26T09:54:57Z
Originally Posted by: Jon Tofte-Hansen 

I have tested 8.3.3.

I have created a job with a list file task and a read file task with a loop iterating the list of 1000 files.

Client on virtual test host: 40-50% CPU use.
Client on local workstation: ~25% CPU use.



And you turned off Extended debugging?
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-26T09:56:59Z
Support
2018-01-26T10:09:46Z
Originally Posted by: Support 

Originally Posted by: Jon Tofte-Hansen 

I have tested 8.3.3.

I have created a job with a list file task and a read file task with a loop iterating the list of 1000 files.

Client on virtual test host: 40-50% CPU use.
Client on local workstation: ~25% CPU use.



And you turned off Extended debugging?



Please test the same with Wait Task in loop.


Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-26T10:14:25Z
Same result: A lot of cpu spent.
Support
2018-01-26T10:26:19Z
Strange. Are you sure nothing else is running?

See video of my test:
https://www.screencast.com/t/FR8jy3Ht 
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-26T11:56:00Z
1: I open the client on the test host (wich also runs the server). There are no other programs running on that machine.
2: I open the client on the local workstations. Only standard office programs in the background not doing anything.
3: I run a job with one Wait task set to wait 1 millisecond with a for loop set to 1000 iterations
4: I monitor the cpu use on the server and on the workstation simultaniously (task maneger uses a little but apart from that the cpu% is 0 on both before and after the job runs)

Cpu: server ~40%, workstation ~30%.

The graphic adapters are Intel HD Graphics 5500 on the workstations and Microsoft Virtual Machine Bus Video Device on the test server.

Is it because the client needs an independant graphic adapter to behave nicely?
Support
2018-01-26T18:02:50Z
Originally Posted by: Jon Tofte-Hansen 

1: I open the client on the test host (wich also runs the server). There are no other programs running on that machine.
2: I open the client on the local workstations. Only standard office programs in the background not doing anything.
3: I run a job with one Wait task set to wait 1 millisecond with a for loop set to 1000 iterations
4: I monitor the cpu use on the server and on the workstation simultaniously (task maneger uses a little but apart from that the cpu% is 0 on both before and after the job runs)

Cpu: server ~40%, workstation ~30%.

The graphic adapters are Intel HD Graphics 5500 on the workstations and Microsoft Virtual Machine Bus Video Device on the test server.

Is it because the client needs an independant graphic adapter to behave nicely?



I am not sure why it affects your system more. But, would you think it would be a good solution if we offered milliseconds for break between each iteration in the Loop settings?
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-29T07:26:59Z
Hmm, that solution didn't help when using the Wait task. And as stated before, the issue is not a showstopper. Thank you for looking into it though.
Support
2018-01-30T10:21:16Z
Originally Posted by: Jon Tofte-Hansen 

Hmm, that solution didn't help when using the Wait task. And as stated before, the issue is not a showstopper. Thank you for looking into it though.




But you wrote:

Quote:

This of course eliminates the high cpu usage.



Also, it would be more effective as no Task needs to be launched - it will just be internal wait in the loop.
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Jon Tofte-Hansen
2018-01-30T11:08:33Z
Originally Posted by: Support 

Yes, I ws suggesting a Task. But if you use 1 second - how does it affect the CPU usage?



You wrote 1 second, which already exist in the loop definition. So if the delay is 1 second the cpu use is low (of course). If the delay is 1 millisecond the cpu use is still high.
Support
2018-01-30T11:19:33Z
Originally Posted by: Jon Tofte-Hansen 

Originally Posted by: Support 

Yes, I ws suggesting a Task. But if you use 1 second - how does it affect the CPU usage?



You wrote 1 second, which already exist in the loop definition. So if the delay is 1 second the cpu use is low (of course). If the delay is 1 millisecond the cpu use is still high.



Yes, I was thinking some inbetween. 1 millisecond is not much but maybe 50-100 is enough to lower.
Henrik
Support
http://www.visualcron.com 
Please like  VisualCron on facebook!
Scroll to Top