install more recent JRE on nodes

At least one participant for Termination needs java >= 1.7.
The nodes currently have 1.6; but features from 1.7 libs are needed.

I built this prover's upload zip file so that it contains the JRE it wants,
and this sort-of-works (see separate message about maxmem),
but it feels super-slow, I guess because for running each single (!) benchmark,
(the contents of) this whole zip file (60 MB) is copied around.

I am not sure if we will be able to update the JRE right now, since StarExec itself is written in Java and hence we have to be at least nominally careful about upgrading.

About copying big solvers around: we are caching solvers on nodes, so at some point the overhead should disappear.

> we are caching solvers on nodes, so at some point the overhead should disappear.

interesting, how can I test this? (how to pin a job on a small set of nodes? - that's where "worker queues" would help, but currently there's only "all.q"?)

Soon we will start creating some queues for testing for all summer 2014 events, so maybe then you could confirm this.

Admin wrote:I am not sure if we will be able to update the JRE right now, since StarExec itself is written in Java and hence we have to be at least nominally careful about upgrading.

I wonder, would it be possible to install several JREs with different versions in parallel? Then StarExec could still run on a Java 6 Virtual Machine, whereas solvers that require Java 7 could be run on a Java 7 Virtual Machine. This may possibly require adapting a solver start script to use e.g. "/opt/java7-jre/bin/java" instead of just "java", but such absolute paths could be publicly announced so that solver authors could adapt their scripts. As far as I can say, this would probably be more elegant than making a solver ship its own JRE for Java 7.

This seems like a reasonable suggestion. I will ask our IT folks about this.

Any updates on this from the IT dept? It turns out that several participants of TermComp use Java 7 in their tool chain -- so it would really be convenient to have also a Java 7 installation in parallel to the Java 6 installation used for the StarExec platform itself. Thanks!

Thanks for the ping about this. I just emailed the IT folks to ask about this again (I think the discussion of Java 7 in particular got lost in lots of other communication with them), and will hopefully have an answer within a day.
Aaron

Just an update about this: we and our IT staff have agreed to install Java 7 on StarExec. The first step is we have pushed it out to our development cluster, and are testing it now. In fact, we may have encountered an issue already there, with interactions with tomcat (heavily Java, of course). Hopefully we will resolve these soon and be able to put it on StarExec. I will keep you posted.

Thanks! While offering Java 7 just as an alternative for the participants should already suffice for this year's Olympic Games (i.e., it should not really be necessary to move also Tomcat to Java 7), it may indeed make sense to move also the web server infrastructure to Java 7. According to http://www.oracle.com/technetwork/java/eol-135779.html Oracle discontinued public updates for Java 6 already in 2013, and thus possible issues in Java 6 that may become known now might not be fixed by a vendor update.

No doubt you are right about eventually moving tomcat over to Java 7, although we will wait until after the competitions are done to try this. About installing Java 7 alongside Java 6, we did run into a snag on our dev cluster, because compiling the web app, our ant build seemed to pick up Java 7, and then we could not load the class files with tomcat under Java 6. So we have to sort that out before we can get Java 7 installed alongside Java 6 on StarExec.
Aaron

One way around "accidentally" using Java 7 for compilation and generating class files incompatible with Java 6 could be to install just the Java 7 Runtime Environment, but not the corresponding Java 7 Development Kit. Then the only Java compiler present on the system would be the Java 6 compiler.

(One could also sift through all the environment variables, e.g. for $PATH, set in the build scripts and build.xml files, or alternatively set the flags for the Java compiler's output format. However, the above "low tech solution" may be the easiest to implement -- if there is no Java 7 compiler present on the machine in the first place, it cannot accidentally be invoked for compiling StarExec either.)

This is a good idea. Actually, we realized that we don't even need Java 7 on the head node, since we never run a solver on our head node. We only would need to put Java 7 on the compute nodes. So that should not interfere with our tomcat installation at all. I'll keep you posted on this. Hopefully we will have Java 7 on the compute nodes by early next week (just a guess).

Ok, we have installed Java 7 on nodes 181 and 182 in the testjavaseven.q queue, which anyone should be able to run jobs on.  Can interested users please test their Java 7 solvers out on these and let me know if it's working? Then we can roll out to the other nodes.

As far as finding Java 7 from a starexec_run script, here is some advice from our IT folks:

I have installed oracle java 1.7 (along with environment modules) on n181 and n182. If they wand to hard code the paths I suppose they could, but its probably cleaner to use environment modules. They would just need to use "module load java/oracle-1.7" to load them if its possible (that way we can change them as needed)

********************************
default java environment (as root)
********************************
[root@n182 ~]# which java
/usr/local/bin/java
[root@n182 ~]# java -version
java version "1.6.0_41"
Java(TM) SE Runtime Environment (build 1.6.0_41-b02)
Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed mode)
[root@n182 ~]# echo $JAVA_HOME


[root@n182 ~]# module avail

------------------------------------------------------------------------------- /usr/share/Modules/modulefiles -------------------------------------------------------------------------------
dot             java/oracle-1.7 module-git      module-info     modules         null            use.own

-------------------------------------------------------------------------------------- /etc/modulefiles --------------------------------------------------------------------------------------
compat-openmpi-psm-x86_64 compat-openmpi-x86_64

********************************
Java 1.7 loaded as an environment module
********************************
[root@n182 ~]# module load java/oracle-1.7
[root@n182 ~]# which java
/usr/lib/jvm/java-1.7.0-oracle-1.7.0.55.x86_64/jre/bin/java
[root@n182 ~]# java -version
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)
[root@n182 ~]# echo $JAVA_HOME
/usr/lib/jvm/java-1.7.0-oracle-1.7.0.55.x86_64/jre
[root@n182 ~]# module unload java/oracle-1.7
[root@n182 ~]# echo $JAVA_HOME

Hi,

AProVE seems to work fine with the Java 7 installation.

Best regards
Florian

Thanks for the confirmation, Florian. We are planning to install Java 7 on the other compute nodes once SMT-COMP is finished (and before the next round of competitions, including Termination).
Aaron

@Aaron - did you roll out Java 7?

Johannes,

It is on the two nodes in testjavaseven.q, and we will roll it out to the rest soon -- maybe tomorrow?

Aaron

Just confirming: we are indeed planning to roll out Java 7, as well as Mono, and Python 2.7 with numpy and scipy tomorrow. I will also announce this on my blog.
Aaron

We have patched all the compute nodes with Java 7 as of this morning. Just FYI.
Aaron