The Grid Community Toolkit documentation was taken from the Globus Toolkit 6.0 documentation. As a result, there may be inaccuracies and outdated information. Please report any problems to the Grid Community Forums as GitHub issues.
This is a quickstart that shows a full installation of the Toolkit on two Fedora Linux machines, named
donkey. It shows the installation of prereqs, installation of the toolkit, creation of certificates, and configuration of services. It is designed to supplement the main admin guide, Scenarios are shown for running GridFTP and GRAM5 services, and using GridFTP and GRAM clients.
Where there is a command to be typed, it will be preceded by one of the following prompts:
Run this command as the
rootsuper-user, on the
donkeyhosts respectively. You might have to use a command like
sudo(8) to start a root shell before executing the command.
Run this command as the
myproxyuser, on the
elephanthost. This user is created automatically when the
myproxy-serverpackage is installed.
Run this command as the normal user account you are intending to interact with your GCT sevices, on the
donkeyhosts. In this document, we use the
quseraccout for this, but if you have another user, you can use it for that purpose.
Commands themselves will be typeset as
-with-arguments, and responses to the commands like this
Response Text. If there is some portion of a command which should be
replaced by value, such as a version number, it will be typeset like
Finally, in some cases you will be prompted for a passphrase. When that
occurs, the entry of the passphrase will be indicated by
even though nothing will be printed to the screen.
We distribute the Grid Community Toolkit 6.2 as a set of RPM and Debian packages
for Linux systems, as an installable package for Mac OS X, as a .zip
file for Windows and Cygwin, as well as a source installer which can be
used on other operating systems. In this quickstart, we will be
installing RPM packages. Thus, it is a prerequisite for following this
quickstart that you are running a distribution for which we provide
RPMs. If you are running a supported Debian or Ubuntu system, the
process is very similar, but you’ll need to use the
similar tools to install the packages. For the source installer, there
is more work involved, and you’ll need to consult the full installation
First, we will to set up our system to use the Globus package repository. This repository contains the GCT software packages, signed by our build manager. We provide RPM and Debian packages that contain a source configuration file and the public key which can be used to verify the packages. If your distribution has GCT 6.2 packages within its repository, you can skip to the next section.
The globus toolkit package repo RPM can be downloaded from the repo RPM package on globus.org.
To install binary RPMs, download the globus-toolkit-repo package from the link above and install it with the command:
root# rpm -hUv globus-toolkit-repo-latest.noarch.rpm
The globus toolkit package repo Debian file can be downloaded from the repo Debian package on globus.org.
To install Debian or Ubuntu package, download the globus-toolkit-repo package from the link above and install it with the command:
root# dpkg -i globus-toolkit-repo_latest_all.deb
Once you’ve installed the Globus repository package, you can use your
operating system’s packaging tools:
install the Globus components.
For operating systems based on RHEL (such as Red Hat Enterprise Linux, CentOS, and Scientific Linux), the compatible EPEL repository must be enabled before installing myproxy. For OS versions 5.x, install the EPEL 5 package, and for OS version 6.x, use 6 package. For information about installing these, see the EPEL FAQ. This step is not needed for Fedora, Debian, or Ubuntu systems.
For SUSE Linux Enterprise Server systems which will be using globus-connect-server, a newer version of apache2 must be installed in order for myproxy-oauth to work. This is available by adding the Apache2 and Apache2 Modules for SLES11 repositories from opensuse.org. These can be installed by running these commands:
root# zypper ar http://download.opensuse.org/repositories/Apache/SLE_11_SP3/Apache.repo root# zypper ar http://download.opensuse.org/repositories/Apache:/Modules/Apache_SLE_11_SP3/Apache:Modules.repo root# rpm --import http://download.opensuse.org/repositories/Apache/SLE_11_SP3/repodata/repomd.xml.key root# rpm --import http://download.opensuse.org/repositories/Apache:/Modules/Apache_SLE_11_SP3/repodata/repomd.xml.key
Setting up the first machine (GridFTP, GRAM, and MyProxy services)
Installing the Toolkit
root# yum install globus-gridftp globus-gram5 globus-gsi myproxy \ myproxy-server myproxy-admin
This will install the GridFTP, GRAM, and MyProxy services, as well as set up a basic SimpleCA so that you can issue security credentials for users to run the GCT services.
For Debian and Ubuntu systems, use
Setting up security on your first machine
The Grid Community Toolkit uses X.509 certificates and proxy certificates to authenticate and authorize grid users. For this quickstart, we use the GCT SimpleCA tools to manage our own Certificate Authority, so that we don’t need to rely on any external entitty to authorize our grid users.
In many deployment scenarios, certificates for both services and users are obtained through one or more third party CAs. In such scenarios, it is unnecessary to use SimpleCA or MyProxy to issue certificates. Since this quickstart is intended to describe a simple, standalone deployment scenario, we describe how to use these tools to issue your own certificates.
globus-simple-ca package is installed, it will
automatically create a new Certificate Authority and deploy its public
certificate into the globus trusted certificate directory. It will also
create a host certificate and key, so that the GCT services will be
able to run.
We’ll also need to copy the host certificate and key into place so that the myproxy service can use it as well.
root# install -o myproxy -m 644 \ /etc/grid-security/hostcert.pem \ /etc/grid-security/myproxy/hostcert.pem root# install -o myproxy -m 600 \ /etc/grid-security/hostkey.pem \ /etc/grid-security/myproxy/hostkey.pem
Creating a MyProxy Server
We are going to create a MyProxy server on elephant, following the
This will be used to store our user’s certificates. In order to enable
myproxy to use the SimpleCA, modify the
file, by uncommenting every line in the section file, by uncommenting
every line in the section
Complete Sample Policy #1 such that
section looks like this myproxy
# # Complete Sample Policy #1 - Credential Repository # # The following lines define a sample policy that enables all # myproxy-server credential repository features. # See below for more examples. accepted_credentials "*" authorized_retrievers "*" default_retrievers "*" authorized_renewers "*" default_renewers "none" authorized_key_retrievers "*" default_key_retrievers "none" trusted_retrievers "*" default_trusted_retrievers "none" cert_dir /etc/grid-security/certificates
We’ll next add the myproxy user to the simpleca group so that the myproxy server can create certificates.
root# usermod -a -G simpleca myproxy
Start the myproxy server:
root# service myproxy-server start Starting myproxy-server (via systemctl): [ OK ]
For Debian and Ubuntu systems, use the
Check that it is running:
root# service myproxy-server status myproxy-server.service - LSB: Startup the MyProxy server daemon Loaded: loaded (/etc/rc.d/init.d/myproxy-server) Active: active (running) since Fri, 02 Nov 2012 09:07:51 -0400; 1min 20s ago Process: 1205 ExecStart=/etc/rc.d/init.d/myproxy-server start (code=exited, status=0/SUCCESS) CGroup: name=systemd:/system/myproxy-server.service └ 1214 /usr/sbin/myproxy-server -s /var/lib/myproxy Nov 02 09:07:51 elephant.globus.org runuser: pam_unix(runuser:session):... Nov 02 09:07:51 elephant.globus.org myproxy-server: myproxy-server v5.9... Nov 02 09:07:51 elephant.globus.org myproxy-server: reading configurati... Nov 02 09:07:51 elephant.globus.org myproxy-server: usage_stats: initia... Nov 02 09:07:51 elephant.globus.org myproxy-server: Socket bound to 0.0... Nov 02 09:07:51 elephant.globus.org myproxy-server: Starting myproxy-se... Nov 02 09:07:51 elephant.globus.org runuser: pam_unix(runuser:session):... Nov 02 09:07:51 elephant.globus.org myproxy-server: Starting myproxy-se...
The important thing to see in the above is that the process is in the
active (running) state. [NOTE]
For other Linux distributions which are not using systemd, the output will be different. You should still see some information indicating the service is running.
As a final sanity check, we’ll make sure the myproxy TCP port
is in use via the netstat command:
root# netstat -an | grep 7512 tcp 0 0 0.0.0.0:7512 0.0.0.0:* LISTEN
We’ll need to specify a full name and a login name for the user we’ll
create credentials for. We’ll be using the
QuickStart User as the
user’s name and
quser as user’s account name. You can use this
as well if you first create a quser unix account. Otherwise, you can use
another local user account. Run the
command as the
myproxy user to create the credentials. You’ll be
prompted for a passphrase, which must be at least 6 characters long, to
encrypt the private key for the user. You must communicate this
passphrase to the user who will be accessing this credential. He can use
myproxy-change-passphrase command to change the passphrase.
The command to create the myproxy credential for the user is
root# su - -s /bin/sh myproxy myproxy% PATH=$PATH:/usr/sbin myproxy% myproxy-admin-adduser -c "QuickStart User" -l quser Legacy library getopts.pl will be removed from the Perl core distribution in the next major release. Please install it from the CPAN distribution Perl4::CoreLibs. It is being used at /usr/sbin/myproxy-admin-adduser, line 42. Enter PEM pass phrase: ****** Verifying - Enter PEM pass phrase:****** The new signed certificate is at: /var/lib/globus/simple_ca/newcerts/02.pem using storage directory /var/lib/myproxy Credential stored successfully Certificate subject is: /O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/OU=local/CN=QuickStart User
Finally, we’ll create a grid map file entry for this credential, so that
the holder of that credential can use it to access globus services.
We’ll use the
program for this. We need to use the exact string from the output above
as the parameter to the -dn command-line option, and the local account
name of user to authorize as the parameter to the -ln command-line
root# grid-mapfile-add-entry -dn \ "/O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/OU=local/CN=QuickStart User" \ -ln quser Modifying /etc/grid-security/grid-mapfile ... /etc/grid-security/grid-mapfile does not exist... Attempting to create /etc/grid-security/grid-mapfile New entry: "/O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/OU=local/CN=QuickStart User" quser (1) entry added
Setting up GridFTP
Now that we have our host and user credentials in place, we can start a globus service. This set up comes from the GridFTP Admin Guide.
Start the GridFTP server:
root# service globus-gridftp-server start Started GridFTP Server [ OK ]
Check that the GridFTP server is running and listening on the gridftp port:
root# service globus-gridftp-server status GridFTP Server Running (pid=20087) root# netstat -an | grep 2811 tcp 0 0 0.0.0.0:2811 0.0.0.0:* LISTEN
Now the GridFTP server is waiting for a request, so we’ll generate a
proxy from the myproxy service by using
myproxy-logon and then
copy a file from the GridFTP server with the
command. We’ll use the passphrase used to create the myproxy credential
quser% myproxy-logon -s elephant Enter MyProxy pass phrase: ****** A credential has been received for user quser in /tmp/x509up_u1001 quser% globus-url-copy gsiftp://elephant.globus.org/etc/group \ file:///tmp/quser.test.copy quser% diff /tmp/quser.test.copy /etc/group
At this point, we’ve configured the myproxy and GridFTP services and verified that we can create a security credential and transfer a file. If you had trouble, check the security troubleshooting section in the Security Admin Guide. Now we can move on to setting up GRAM5 resource management.
Setting up GRAM5
Now that we have security and GridFTP set up, we can set up GRAM for resource management. There are several different Local Resource Managers (LRMs) that one could configure GRAM to use, but this guide will explain the simple case of setting up a "fork" jobmanager, without auditing. For details on all other configuration options, and for reference, you can see the GRAM5 Admin Guide. The GRAM service will use the same host credential as the GridFTP service, and is configured by default to use the fork manager, so all we need to do now is start the service.
Start the GRAM gatekeeper:
root# service globus-gatekeeper start Started globus-gatekeeper [ OK ]
We can now verify that the service is running and listening on the GRAM5 port:
root# service globus-gatekeeper status globus-gatekeeper is running (pid=20199) root# netstat -an | grep 2119 tcp6 0 0 :::2119 :::* LISTEN
The gatekeeper is set up to run, and is ready to authorize job submissions and pass them on to the fork job manager. We can now run a couple of test jobs:
quser% myproxy-logon -s elephant Enter MyProxy pass phrase: ****** A credential has been received for user quser in /tmp/x509up_u1001. quser% globus-job-run elephant /bin/hostname elephant.globus.org quser% globus-job-run elephant /usr/bin/whoami quser
Setting up your second machine
Alas, it’s not much of a grid with just one machine. So let’s start up on another machine and add it to this little test grid.
Setting up your second machine: Prereqs
Setting up your second machine: Installation
Install packages as before:
[email protected]# yum install globus-gridftp myproxy globus-gram5
Setting up your second machine: Security
Now let’s get security set up on the second machine. We’re going to trust the original simpleCA to this new machine; there’s no need to create a new one. First, we’ll bootstrap trust of the SimpleCA running on elephant:
[email protected]# myproxy-get-trustroots -b -s elephant Bootstrapping MyProxy server root of trust. New trusted MyProxy server: /O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/CN=host/elephant.globus.org New trusted CA (e3d1c34d.0): /O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/CN=Globus Simple CA Trust roots have been installed in /etc/grid-security/certificates/.
This allows clients and services on
donkey to trust certificates
which are signed by the CA on
elephant machine. If we weren’t
going to run any GCT services on
donkey, then we could stop
here. Users on
donkey could acquire credentials using the
myproxy-logon command and perform file transfers and execute
jobs using the
commands. However, we’ll continue to configure the GridFTP and GRAM5
donkey as well.
We’re going to create the host certificate for donkey, but we create it
on elephant, so that we don’t have to copy the certificate request
between machines. The
myproxy-admin-addservice command will
prompt for a passphrase for this credential. We will use this passphrase
to retrieve the credential on donkey.
myproxy% myproxy-admin-addservice -c "donkey.globus.org" -l donkey Legacy library getopts.pl will be removed from the Perl core distribution in the next major release. Please install it from the CPAN distribution Perl4::CoreLibs. It is being used at /sbin/myproxy-admin-addservice, line 42. Enter PEM pass phrase:****** Verifying - Enter PEM pass phrase:****** The new signed certificate is at: /var/lib/globus/simple_ca/newcerts/03.pem using storage directory /var/lib/myproxy Credential stored successfully Certificate subject is: /O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/OU=local/CN=donkey.globus.org
Next we’ll retrieve the credential on donkey as the root user.
[email protected]# myproxy-retrieve -s elephant -k donkey.globus.org -l donkey Enter MyProxy pass phrase: ****** Credentials for quser have been stored in /etc/grid-security/hostcert.pem and /etc/grid-security/hostkey.pem.
At this point, we no longer need to have
elephant's myproxy server, so we’ll delete it.
[email protected]# myproxy-destroy -s elephant -k donkey.globus.org -l donkey MyProxy credential 'donkey.globus.org' for user donkey was successfully removed.
And as a final setup, we’ll add quser’s credential to the grid-mapfile
donkey, so that the
quser account can access services
there as well.
[email protected]# grid-mapfile-add-entry -dn \ "/O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/OU=local/CN=QuickStart User" \ -ln quser Modifying /etc/grid-security/grid-mapfile ... /etc/grid-security/grid-mapfile does not exist... Attempting to create /etc/grid-security/grid-mapfile New entry: "/O=Grid/OU=GlobusTest/OU=simpleCA-elephant.globus.org/OU=local/CN=QuickStart User" quser (1) entry added
At this point, we have set up security on donkey to trust the CA on elephant. We have created a host certificate for donkey so that we can run GCT services on donkey, and we have enabled the quser account to use services on donkey. The last thing to do is to turn on the GCT services on donkey.
Setting up your second machine: GridFTP
GridFTP set up on the second machine is identical to the first. I’ll just list the commands here; see Setting up GridFTP for additional information.
[email protected]# service globus-gridftp-server start Started GridFTP Server [ OK ]
Now we can test it.
First, we’ll retrieve a proxy credential from the myproxy server so that
the user on donkey can interact with the GCT services. Here we’ll use
the same passphrase as we used to create the
[email protected]% myproxy-logon -s elephant Enter MyProxy pass phrase: ****** A credential has been received for user quser in /tmp/x509up_u1001.
Next we’ll transfer a file between the gridftp servers on donkey and elephant:
[email protected]% globus-url-copy gsiftp://elephant.globus.org/etc/group \ gsiftp://donkey.globus.org/tmp/from-elephant
That was a slightly more complicated test than we ran on elephant earlier. In this case, we did a third-party transfer between two GridFTP servers. It worked, so I have the local and remote security configured correctly.
If you run into problems, perhaps you have a firewall between the two machines? GridFTP needs to communicate on data ports, not just port 2811. The error for this condition looks like:
error: globus_ftp_client: the server responded with an error 500 500-Command failed. : callback failed. 500-globus_xio: Unable to connect to 188.8.131.52:42777 500-globus_xio: System error in connect: No route to host 500-globus_xio: A system call failed: No route to host 500 End.
You can set up a range of ports to be open on the firewall and configure GridFTP to use them. See the GridFTP admin firewall doc.
Setting up your second machine: GRAM5
Now we can submit a staging job. This job will copy the
/bin/echo program from donkey to a file called
Then it runs it with some arguments, and captures the stderr/stdout.
Finally, it will clean up the . Then it runs it with some arguments, and
captures the stderr/stdout. Finally, it will clean up the
file when execution is done. file when execution is done.
[email protected]% globus-job-run elephant \ -x '(file_stage_in=(gsiftp://donkey.globus.org/bin/echo /tmp/echo)) \ (file_clean_up=/tmp/echo)' /bin/ls -l /tmp/echo -rw-r--r-- 1 quser quser 27120 Nov 2 09:56 /tmp/echo
This example staged in a file, had an executable act on that file, and cleaned up the file afterward.
You can get other examples of GRAM files from GRAM usage scenarios.
Hopefully this guide has been helpful in familiarizing you with some of the administration tasks and tools to use the Grid Community Toolkit. If you’ve reached this point successfully, you should have enough knowledge to enable additional hosts to use your grid by repeating the tasks in Setting up your second machine. Also, by repeating the tasks in User Credentials and User Authorization you can enable additional users to access your compute and data resources.