|
From: | Groom, George |
Subject: | RE: [EXTERNAL] Re: Linux kernel question, pre 2.6.23 vs later kernel versions |
Date: | Mon, 22 Oct 2018 16:54:49 +0000 |
MCVE Extract the 2 attachments in the same directory. Execute milti-processor-examples shell script. Needs sudo access to pull machine profile. Change path to linux dictionary in script as needed. Script is straight forward as it forks “grep ‘string’ dict” processes for all words in word_list. Actual output from my 2 HP systems are below. Note execution time differences between later and older kernels. Runtimes are quicker on an older machine with older kernel (process scheduling?). Note runtimes of xargs and for-loop are
much quicker! Thank you for your time, George Groom Comparing runtime of serial and 3 parallel processing methods...parallel, xargs and a for-loop Run mode: cat Kernel 3.10.0-693.17.1.el7.x86_64 Machine profile HTT (Multi-threading) Version: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz Core Count: 22 Core Enabled: 22 Thread Count: 44 Multi-Core Hardware Thread HTT (Multi-threading) Version: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz Core Count: 22 Core Enabled: 22 Thread Count: 44 Multi-Core Hardware Thread Product Name: Synergy 480 Gen9 Product Name: Synergy 480 Gen9 Compute Module String 2: Product ID: 732350-B21 serially... in this mode processing takes too long...skipping parallel... real 0m36.438s user 2m40.102s sys 0m29.745s xargs... real 0m28.718s user 2m26.768s sys 0m13.546s for-loop... real 0m29.858s user 2m25.937s sys 0m9.881s Comparing runtime of serial and 3 parallel processing methods...parallel, xargs and a for-loop Run mode: cat Kernel 2.6.18-238.el5 Machine profile Product Name: ProLiant DL360 G7 HTT (Multi-threading) Version: Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
Core Count: 6 Core Enabled: 6 Thread Count: 12 HTT (Multi-threading) Version: Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
Core Count: 6 Core Enabled: 6 Thread Count: 12 String 2: Product ID: 579237-B21 serially... in this mode processing takes too long...skipping parallel... real 0m23.732s user 1m33.362s sys 0m34.111s xargs... real 0m4.950s user 1m16.905s sys 0m16.935s for-loop... real 0m3.667s user 1m2.713s sys 0m12.475s -----Original Message----- On Wed, Oct 17, 2018 at 10:25 PM Groom, George <George_Groom@comcast.com> wrote: > I understand that kernel process scheduling changed at 2.6.23… run queue based to Completely Fair Scheduling (CFS). Has anyone experienced longer parallel script execute times on systems with later versions of the Linux kernel? > > I’ve noticed longer execute times where I’m generating several thousands of processes on later kernel machines. CFS seems to throttle my processes within a few seconds and then control execution until script completion. Are there
known workarounds for this behavior? Depending on application needs the older method might be preferable? I cannot reproduce this. Make an MCVE. /Ole |
multi-processor-examples
Description: multi-processor-examples
word_list
Description: word_list
[Prev in Thread] | Current Thread | [Next in Thread] |