memory limits
2 posters
Page 1 of 1
memory limits
Hi,
I find it confusing that not all memory limits use the same unit.
E.g. $STAREXEC_MAX_MEM is specified in MB (see [1])
while the "maximum memory" when creating a job (see help text
when hoovering with the mouse over the field) are specified in GB.
I have started a job with 128.0 "maximum memory" and the
$STAREXEC_MAX_MEM variable ineed returned the correct
value 131072.
Harald.
[1] https://wiki.uiowa.edu/display/stardev/User+Guide#UserGuide-SpecialVariables
I find it confusing that not all memory limits use the same unit.
E.g. $STAREXEC_MAX_MEM is specified in MB (see [1])
while the "maximum memory" when creating a job (see help text
when hoovering with the mouse over the field) are specified in GB.
I have started a job with 128.0 "maximum memory" and the
$STAREXEC_MAX_MEM variable ineed returned the correct
value 131072.
Harald.
[1] https://wiki.uiowa.edu/display/stardev/User+Guide#UserGuide-SpecialVariables
hzankl- Posts : 19
Join date : 2014-05-14
Re: memory limits
Harald,
Your point is well taken. I am sorry for the discrepancy. We can fix that. I guess the least invasive fix would be to change "maximum memory" when creating a job. Otherwise people might have to change their solvers if we change the unit for $STAREXEC_MAX_MEM.
Aaron
Your point is well taken. I am sorry for the discrepancy. We can fix that. I guess the least invasive fix would be to change "maximum memory" when creating a job. Otherwise people might have to change their solvers if we change the unit for $STAREXEC_MAX_MEM.
Aaron
Re: memory limits
Fine with me.
I am not sure if the help text of "maximum memory" is correct, saying
"the maximum of this value and half the available memory on the nodes will be used.
I read the explanation as max(X,Y/2) where X is the value specified and Y is the memory of the node.
So Y = 128 or Y = 256, right?
But I get different results concerning "memory out" when using X = 1.0 and X = 8.0.
So I did some experiments and think there are two bugs:
(1) min(X,Y/2) is used computed as memory limit (although max(X,Y/2) is claimed)
(2) the computed value is not passed to runsolver but X is passed
Details:
Test 1: I have chosen 12800GB as "maximum memory" and indeed runsolver is called with this limit (in MB).
> 06/12/14 03:05:12 PM CDT: node memory in megabytes = 129022
> truncating max memory from requested 13107200 to 64511 (half max memory on the node)
> 06/12/14 03:05:12 PM CDT: we will run the job pair with max memory in megabytes = 64511
...
> /export/starexec/sandbox/solver/bin/runsolver --timestamp --add-eof --cores 0-3 -C 240 -W 60 -M 13107200
Test 2: I have chosen 1GB as "maximum memory" and indeed runsolver is calles with this limit (in MB).
06/12/14 03:25:22 PM CDT: we will run the job pair with max memory in megabytes = 1024
06/12/14 03:25:22 PM CDT: runsolver is /export/starexec/sandbox/solver/bin/runsolver
06/12/14 03:25:22 PM CDT: solver is /export/starexec/sandbox/solver/bin/starexec_run_test
06/12/14 03:25:22 PM CDT: benchmark is /export/starexec/sandbox/benchmark/theBenchmark
06/12/14 03:25:22 PM CDT: prolog ends
/export/starexec/sandbox/solver/bin
/export/starexec/sandbox/solver/bin/runsolver --timestamp --add-eof --cores 0-3 -C 240 -W 60 -M 1024
Harald.
PS: Sorry for my many edits on this post but it took me some time to figure out and distinguish the two bugs.
I am not sure if the help text of "maximum memory" is correct, saying
"the maximum of this value and half the available memory on the nodes will be used.
I read the explanation as max(X,Y/2) where X is the value specified and Y is the memory of the node.
So Y = 128 or Y = 256, right?
But I get different results concerning "memory out" when using X = 1.0 and X = 8.0.
So I did some experiments and think there are two bugs:
(1) min(X,Y/2) is used computed as memory limit (although max(X,Y/2) is claimed)
(2) the computed value is not passed to runsolver but X is passed
Details:
Test 1: I have chosen 12800GB as "maximum memory" and indeed runsolver is called with this limit (in MB).
> 06/12/14 03:05:12 PM CDT: node memory in megabytes = 129022
> truncating max memory from requested 13107200 to 64511 (half max memory on the node)
> 06/12/14 03:05:12 PM CDT: we will run the job pair with max memory in megabytes = 64511
...
> /export/starexec/sandbox/solver/bin/runsolver --timestamp --add-eof --cores 0-3 -C 240 -W 60 -M 13107200
Test 2: I have chosen 1GB as "maximum memory" and indeed runsolver is calles with this limit (in MB).
06/12/14 03:25:22 PM CDT: we will run the job pair with max memory in megabytes = 1024
06/12/14 03:25:22 PM CDT: runsolver is /export/starexec/sandbox/solver/bin/runsolver
06/12/14 03:25:22 PM CDT: solver is /export/starexec/sandbox/solver/bin/starexec_run_test
06/12/14 03:25:22 PM CDT: benchmark is /export/starexec/sandbox/benchmark/theBenchmark
06/12/14 03:25:22 PM CDT: prolog ends
/export/starexec/sandbox/solver/bin
/export/starexec/sandbox/solver/bin/runsolver --timestamp --add-eof --cores 0-3 -C 240 -W 60 -M 1024
Harald.
PS: Sorry for my many edits on this post but it took me some time to figure out and distinguish the two bugs.
hzankl- Posts : 19
Join date : 2014-05-14
Re: memory limits
Hi, Harald.
Thanks for reporting these problems. Sorry about that. We will fix these this week.
Aaron
Thanks for reporting these problems. Sorry about that. We will fix these this week.
Aaron
Similar topics
» Disk space limits
» Max memory for jobs?
» Memory usage in CSV file
» Time and memory not stored in .csv
» Max memory for jobs?
» Memory usage in CSV file
» Time and memory not stored in .csv
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum
|
|