A worldwide e-Infrastructure for NMR and structural biology

Grid deployment scripts (for site admin)

Deploying a software on the grid can be time consuming. We wrote a chain of scripts that should facilitate the deployment of a software across multiple sites. You can downlad an archive containing a collection of scripts at the bottom of the page (deploy.tgz)

This collection of script is aimed at facilitating grid software deployment.

1.check-status

FIrst you should go in 1.check-status and run the numbered script.

1.generate-jdls.sh
will generate a jdl file per computing element that are currently advertized (found with lcg-infosites
2.submit-jdls.sh
will submit a job per ce. The job is really small (see FILES/mdirectory) and give some general info about the software directory hierarchie etc...
3.check-status.sh
Check the status of the jobs.
4.output-if-success.sh
Get the output if the job is succesfull
5.output-if-exit-code-zero.sh
get the output if the job has exit different than 0.

Once you run script 2. you should wait a while (24h?) to give the time for the jobs to finish. You can check with 3. You can use multipe time 4. and 5. as if the job is already cleared, it won't get the output anymore.

Only the ce with succesfull jobs will be used later on for grid deployment.

2.deploy

Then you should go into 2.deploy dir. For each software you want to deploy, you should

a.Add a soft link of all the files in DEPLOY-TEMPLATE/ dir in the SOFTWARE/target-software/ you want to deploy

Example
[christophe@ui-enmr gmx4.5.3.rtc]$ for i in ../../DEPLOY-TEMPLATE/*.sh; do ln -s $i ./; done

b. Create correctly the file SOFTWARE/target-software/FILES/deploy.sh and SOFTWARE/target-software/FILES/test.sh
For example SOFTWARE/gmx4.5.3.rtc/FILES/deploy.sh

c. Create correctly the tag file SOFTWARE/target-software/FILES/tag
For example SOFTWARE/gmx4.5.3.rtc/FILES/tag

d. call in the correct order the scripts linked in 1. More details bellow.

0.list-tag.sh
It list all ce and show if the target tag is already presents

1.generate-jdls.sh
generate 2 jdls (test and deploy) per targets. The targets are the one for succesfull jobs during the check-status phase (in the  1.check-status/OUT/ dir)

2.submit-one-deploy-jdl.sh
This script will ask you on which ce you want to deploy. It will display the output of the 1.check-status/OUT/ directory for visual inspection. Be carfull, some ces actually mount multiple time the same SOFT directory, so if you deploy simultanesouly the same software to 2 ces that mount the same SOFT dir, you might run into trouble.

3.check-status-deploy.sh
run this command to see the status of the deploy jobs

4.get-output-deploy-if-success.sh
Run this job to get the output of the deploy job (can be run multiple time)

4-bis.get-output-deploy-if-fail.sh
Same if the job has exit != 0

5.submit-test-jdl.sh
For succesfull deploy jobs, you can submit a test job. It will ask you for visual inspection of the output done at script 4.

6.check-status-test.sh
you can chekc the status of the test jobs

7.get-output-test-if-success.sh
You can get the output of the test jobs

7-bis.get-output-test-if-fail.sh
you can get the output of jobs with exit code != 0

8.check-test-output-and-tag.sh
This script will display for every succesfull job the output, and ask you if you want to tag the corresponding ce.
 

AttachmentSize
deploy.tgz6.16 KB
0
Your rating: None

Cite WeNMR

 
Usage of the WeNMR portals should be acknowledged in any publication:
 
"The WeNMR project (European FP7 e-Infrastructure grant, contract no. 261572www.wenmr.eu), supported by the European Grid Initiative (EGI) through the national GRID Initiatives of Belgium, France, Italy, Germany, the Netherlands, Poland, Portugal, Spain, UK, South Africa, Malaysia, Taiwan, the Latin America GRID infrastructure via the Gisela project, the International Desktop Grid Federation (IDGF) with its volunteers and the US Open Science Grid (OSG) are acknowledged for the use of web portals, computing and storage facilities. "
 
And the following article describing the WeNMR portals should be cited:
Wassenaar et al. (2012). WeNMR: Structural Biology on the Grid.J. Grid. Comp., 10:743-767.

EGI-approved

The WeNMR Virtual Research Community has been the first to be officially recognized by the EGI.

European Union

WeNMR is an e-Infrastructure project funded under the 7th framework of the EU. Contract no. 261572