Nightly Build

From ScorecWiki

Jump to: navigation, search


Contents

Overview

SCOREC's nightly automatic build framework. Every night a script is run on designated computers which builds software according to a configuration file and runs any test scripts found in user's home directories. The system can accommodate nearly any build system but works especially well with gnu autoconf and automake. A separate configuration for each build architecture and/or compiler is required but user tests are run on all architectures.

Registering a New Package with Nightly Build

To register a new package with the nightly build system you will need to send an email to help@scorec.rpi.edu with the following information:

For each desired build architecture (currently the system is setup on x86 based Linux and Solaris):

  • The name you want your package to be known by
  • A list of email addresses which should receive an email should the build fail
  • The address of the svn repository where the package can be obtained
  • The commands necessary to build the package: either options for autoconf/automake or a script/list of commands which can build the package. Note that this can include any tests that are architecture specific that should be run at build time.
  • A list of environment variables required to build the package (for example CXX=/usr/bin/customcompiler.bin)
  • A list of user names who will be submitting custom tests. Please state if any of these users are new to the nightly build system so that they can be added to the nightlybuild group
  • A list of external libraries which are not already found on a default SCOREC workstation install and preferably a script to build them and/or a path to the prebuilt versions you want to use for each architecture

Locating the Freshly Built Software

Assuming all went well your package should be built by the next morning. You can locate the build directory in ~nightlybuild/builds/[package name]/[architecture]. You should also find a log file named [architecture].log in ~nightlybuild/builds/[package name] containing the complete output of the build process (what you would see had you checked out and built the package manually on the command line).

If the build fails then you will receive an email with the following message:

 Build of <PROJECT-NAME> on <ARCH-OS> failed with error: customBuild command failed, see /users/nightlybuild/builds/<PROJECT-NAME>/<ARCH-OS>.log for details

Setting up User Tests

To run a user test you will need to be listed in the configuration file of the package for which you want to write tests, a member of the nb group and have a "nightlybuild" directory containing a "tests" directory with read and write access for the nb group in your home directory. This should have been setup for you when you registered your package; if not please email help@scorec.rpi.edu

Assuming that is all set you need to create a directory in ~/nightlybuild/tests named exactly what you specified as your package name when you registered it. You can then write shell scripts (for bash) to do your tests and place them in this directory. The nightlybuild system will execute anything in this directory that is marked executable (chmod g+x file.sh can be used to mark the file executable)

A sample script might look like the following:

 #!/bin/bash

 ~nightlybuild/builds/GNUsample/x86_linux/GNUsample/src/grep --version 

If you are building for more than one architecture you can use ~nightlybuild/bin/getarch to determine which architecture the test is currently being run on. The output of the test script will be saved in a log file named [test name]-[architecture].log in ~/nightlybuild/tests/[package name]/ Note that the first line of the above example is required for all scripts. Generally, all your test script will need to be is a list of commands preceded with the build path as you would run them on the command line. If you need more information about scripting look at http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html or email help.scorec.rpi.edu

Tips

Now that you have your package registered with nightlybuild and tests setup it is important to check the log files produced by both the build and your user tests from time to time. If for some reason a test or build fails but doesn't return an error code you won't receive an email!

It may be easier to work from the directory where your software is built. For example:

 #!/bin/bash
 arch=`$~nightlybuild/bin/getarch`
 cd ~nightlybuild/builds/[package]/$arch
 ./test.bin && ./test2.sh && ./test3.pl


If your project uses the SCOREC build tools or any other external libraries which aren't included in the standard SCOREC install (anything which isn't found on the SCOREC workstations) you will need to either have pre-built versions someplace accessible by the "nightlybuild" user and the "nb" group or have nightlybuild build them as part of you custom script or as separate projects. Your custom build script can include calls to any utility available on SCOREC workstations (for example you could use svn to checkout a library before building your project). If you need help with scripting please contact help@scorec.rpi.edu with as much information as possible.

If you are using a custom build script which doesn't exit with a failure code (non-zero value) you won't get emails when your build fails! If you are using bash you can check the return status of individual calls by checking the $? variable.

Nightlybuild now has web space. If you want nightlybuild to post documentation for your project on the web (at build time) please state this in your registration request and include the commands to generate and copy it to an arbitrary path.

Personal tools