Features new in GT 6.0:

Features that continue to be supported from previous versions

Other Supported Features

Deprecated Features

While the above list includes platforms on which we have tested GridFTP, it does not imply support for a specific platform. However, we are interested in hearing reports of success or bug reports on any platform.

Protocol changes since GT 5.2

API changes since GT 5.2

Exception changes since GT 5.2

Schema changes since GT 5.2

GridFTP depends on the following GCT components:

GridFTP depends on the following 3rd party software:

The following list describes the process between the service listening for connection and an exchange of data taking place:

There are basically four important components of the exchange:

Last Update: August 2005

Working with Los Alamos National Laboratory and the High Performance Storage System (HPSS) collaboration (http://www.hpss-collaboration.org), we have written a Data Storage Interface (DSI) for read/write access to HPSS. This DSI would allow an existing application that uses a GridFTP compliant client to utilize an HPSS data resources.

This DSI is currently in testing. Due to changes in the HPSS security mechanisms, it requires HPSS 6.2 or later, which is due to be released in Q4 2005. Distribution for the DSI has not been worked out yet, but it will probably be available from both Globus and the HPSS collaboration. While this code will be open source, it requires underlying HPSS libraries which are NOT open source (proprietary).

Last Update: August 2005

Working with the SRB team at the San Diego Supercomputing Center, we have written a Data Storage Interface (DSI) for read/write access to data in the Storage Resource Broker (SRB) (http://www.npaci.edu/DICE/SRB). This DSI will enable GridFTP compliant clients to read and write data to an SRB server, similar in functionality to the sput/sget commands.

This DSI is currently in testing and is not yet publicly available, but will be available from both the SRB web site (here) and the Globus web site (here). It will also be included in the next stable release of the toolkit. We are working on performance tests, but early results indicate that for wide area network (WAN) transfers, the performance is comparable.

When might you want to use this functionality:

The Globus FTP Client library provides a convenient way of accessing files on remote FTP servers. In addition to supporting the basic FTP protocol, the FTP Client library supports several security and performance extensions to make FTP more suitable for Grid applications. These extensions are described in the GridFTP Protocol document.

In addition to protocol support for grid applications, the FTP Client library provides a plugin architecture for installing application or grid-specific fault recovery and performance tuning algorithms within the library. Application writers may then target their code toward the FTP Client library and, by simply enabling the appropriate plugins, easily tune their application to run it on a different grid.

All applications which use the Globus FTP Client API must include the header file globus_ftp_client.h and activate the and activate the GLOBUS_FTP_CLIENT_MODULE.

To use the Globus FTP Client API, one must create an FTP Client handle. This structure contains:

The specifics of the connection caching and plugins are found in the "http://toolkit.globus.org/api/c-globus-6.0/group%5f%5fglobus%5f%5fftp%5f%5fclient%5f%5fhandleattr.html[Handle Attributes]" section of the API documentation.

Once the handle is created, one may begin transferring files or doing other FTP operations by calling the functions in the "http://toolkit.globus.org/api/c-globus-6.0/group%5f%5fglobus%5f%5fftp%5f%5fclient%5f%5foperations.html[FTP Operations]" section of the API documentation. In addition to whole-file transfers, the API supports partial file transfers, restarting transfers from a known point, and various FTP directory management commands. All FTP operations may have a set of attributes, defined in the operationattr section, associated with them to tune various FTP parameters. The data structures and functions needed to restart a file transfer are described in the "http://toolkit.globus.org/api/c-globus-6.0/group%5f%5fglobus%5f%5fftp%5f%5fclient%5f%5frestart%5f%5fmarker.html[Restart Markers]" section of the API documentation. For operations which require the user to send to or receive data from an FTP server they must call the functions described in the "globus_ftp_client_data" section of the manual.

The globus_ftp_control library provides low-level services needed to implement FTP clients and servers. The API provided is protocol specific. The data transfer portion of this API provides support for the standard data methods described in the FTP Specification as well as extensions for parallel, striped, and partial data transfer.

For information on the internationalization API, see C Common Libraries API.

globus-url-copy - globus-url-copy

globus-url-copy [options] SOURCE-URL DESTINATION-URL

The globus-url-copy program is a command line tool for multi-protocol data movement. It supports gsiftp:// (GridFTP), ftp://, http://, https://, sshftp:// and file:/// protocol specifiers in the URL.

Copyright © 1999-2014 University of Chicago

globus-gridftp-server - The Globus GridFTP server daemon

globus-gridftp-server OPTIONS

The globus-gridftp-server program is a ftp server with support for GridFTP protocol extensions, including strong authentication, parallel data transfers, and parallel data layouts.

The list below contains the command-line options for the server, and also the name of the configuration file entry that implements that option. Note that any boolean option can be negated on the command line by preceding the specified option with -no- or -n. example: -no-cas or -nf.

globus-gridftp-server-setup-chroot - Set up a chroot for the Globus GridFTP server

globus-gridftp-server-setup-chroot [-h] [-c CERT-DIR] -r NEW-CHROOT

The globus-gridftp-server-setup-chroot program creates a chroot directory tree that can be used for the globus-gridftp-server(8). This chroot contains a copy of essential POSIX devices in dev; hosts, group, passwd, and grid-security configuration files in etc; and a temporary file directory in tmp.

The -c CERT-DIR option copies certificate files from a different directory than the default [/etc/grid-security/certificates].

Copyright © 1999-2015 University of Chicago

globus-gridftp-server(8)

Grid Community Toolkit does not provide a client with Graphical User Interface (GUI) but Globus Service provides a web GUI for GridFTP data movement. It has the following features: * With a one-click Globus Connect Personal installed on your local system, you can browse the local file system and transfer files and directories between the local system and remote GridFTP servers and between two remote GridFTP servers (third-party transfers). * Supports file system operations such as creating, deleting and renaming files and directories.

Supported Platforms: * Windows * Linux * Mac

[overview]

The configuration interface for GridFTP is the admin tool, , which can be used with a configuration file and/or run-time options.

The configuration file for the GridFTP server is read from the following locations, in the given order. Only the first file found will be loaded:

Options are one per line, with the format:

If the value contains spaces, they should be enclosed in double-quotes ("). Flags or boolean options should only have a value of 0 or 1. Blank lines and lines beginning with # are ignored.

For example:

For complete command documentation including all options, see .

This page includes information about general configuration of the GridFTP server. Security options are discussed here, and more advanced configuration is described here.

The following describes a typical GridFTP configuration of the front end (control channel) and back end (data channels). For other alternatives that provide greater levels of security, see Advanced Configuration.

By default, the data channel and control channel are separate socket connections within the same process. The client sends a command and waits to finish before issuing the next command. This is good for a single host, traditional-type user. If you have a single host and you want an ultra-reliable and light weight file transfer service, this is a good choice. This configuration is also good for testing purposes.

If the GridFTP server is behind a firewall:

If the GridFTP client is behind a firewall:

Additional information on Grid Community Toolkit Firewall Requirements is available here.

There are many security options in GridFTP ranging from no security to higher security via GSI .

The following is a quick guide to running the server and using the client:

Look through the list of options for globus-gridftp-server:

Start the server in anonymous mode (discussed more fully here):

where:

Run a two party transfer with client:

Run 3rd party transfer:

Experiment with -dbg, and -vb options for debugging and checking the performance of your setup:

where:

Ctrl-c - Kill the server.

To enable file sharing using Globus Sharing, you have to add the Globus Sharing CA certificates to your trusted certificates directory (/etc/grid-security/certificates) and use -sharing-dn option in the server as follows:

and use -sharing-rp option to restrict the file paths allowed for sharing:

The GridFTP server or client libraries do not read any environment variable directly, but the security and networking related variables described below may be useful.

Verify that you can establish a control channel connection and that the server has started successfully by telnetting to the port on which the server is running:

If you see anything other than a 220 banner such as the one above, the server has not started correctly.

Verify that there are no configuration files being unexpectedly loaded from /etc/grid-security/gridftp.conf or $GLOBUS_LOCATION/etc/gridftp.conf. If those files exist, and you did not intend for them to be used, rename them to .save, or specify -c none on the command line and try again.

If you can log into the machine where the server is, try running the server from the command line with only the -s option:

The server will print the port it is listening on:

Now try and telnet to that port. If you still do not get the banner listed above, something is preventing the socket connection. Check firewalls, tcp-wrapper, etc.

If you now get a correct banner, add -p 2811 (you will have to disable (x)inetd on port 2811 if you are using them or you will get port already in use):

Now telnet to port 2811. If this does not work, something is blocking port 2811. Check firewalls, tcp-wrapper, etc.

If this works correctly then re-enable your normal server, but remove all options but -i, -s, or -S.

Now telnet to port 2811. If this does not work, something is wrong with your service configuration. Check /etc/services and (x)inetd config, have (x)inetd restarted, etc.

If this works, begin adding options back one at a time, verifying that you can telnet to the server after each option is added. Continue this till you find the problem or get all the options you want.

At this point, you can establish a control connection. Now try running globus-url-copy.

Once you’ve verified that you can establish a control connection, try to make a transfer using globus-url-copy.

If you are doing a client/server transfer (one of your URLs has file: in it) then try: in it) then try:

This will run until you control-c the transfer. If that works, reverse the direction:

Again, this will run until you control-c the transfer.

If you are doing a third party transfer, run this command:

Again, this will run until you control-c the transfer.

If the above transfers work, try your transfer again. If it fails, you likely have some sort of file permissions problem, typo in a file name, etc.

If the server has started correctly, and your problem is with a security failure or gridmap lookup failure, verify that you have security configured properly here.

If the server is running and your client successfully authenticates but has a problem at some other time during the session, please ask for help on [email protected]. When you send mail or submit bugs, please always include as much of the following information as possible:

If you run GridFTP servers via Xinetd and notice high latency for connections and/or transfers, check if /etc/xinetd.conf or the gsiftp service configuration inside or the gsiftp service configuration inside /etc/xinetd.d is set to log USERID as follows: is set to log USERID as follows:

Such a configuration tells Xinetd to log the remote user using the method defined in RFC 1413, which causes an ident client to attempt to query the machine that the connection is coming from before the service will run. Even when this succeeds, the response can’t be trusted, and more often than not it is rejected or simply dropped (which results in the longest delays) by the remote firewall.

Latency can be reduced by making sure Xinetd does not log the USERID.

A user may have multiple credentials issued by different organizations that know nothing about the other. If a user tries to perform a 3rd party FTP transfer between these organizations' servers using DCAU (Data Channel Authentication), it will fail because DCAU uses the user’s credentials and each side does not have the CA certificate that issued the other side’s user credential. Thus, users are unlikely to be able to perform secure 3rd party transfers in large federated environments. We present the DCSC command as a way to enable DCAU in this scenario even if one side is a legacy server that knows nothing about DCSC. FTP servers that use SSH for user authentication also benefit from DCSC since it provides a common, interoperable context for DCAU.

The current DCAU protocol uses an SSL context that contains the logged in user’s credential. If two servers have different user credentials and do not have each others' CA certificates, a client can not perform a secure 3rd party transfer between them. If one of the servers supports DCSC, a client can tell it to both send and accept the user credential used by the other server, thus enabling DCAU where it previously was not possible. If both servers support DCSC, clients that desire higher security may specify a random, self signed certificate as the DCAU context.

The general format for DCSC is:

Where context type is a case insensitive string and the blob is a string composed of only printable ASCII (32-126) characters, such as base64 encoding would produce. This document specifies the "P" and "D" context types.

A server that supports DCSC SHOULD include a line in its FEAT output so clients can discover the feature. The format of the FEAT line is:

For example, "DCSC P,D" means a server supports the "P" and "D" DCSC types.

The allowed return codes for DCSC P and DCSC D are:

The "P" context type (short for proxy/PEM) is:

base64 encoded blob:

The PEM certificate(s) and key are already mostly in base64 encoding (the new line characters aren’t) and thus encoding them again in base64 wastes some space. However, this simplifies client and server implementation.

A DCSC "P" command will overwrite any previous request. A server SHOULD accept a DCSC "P" blob of at least 10KB in size.

The "D" context type stands for "default context". The command "DCSC D" will revert the context to whatever it was immediately after login.

In legacy mode (only one server supports DCSC), the server supporting DCSC holds two short term credentials - one from a CA it knows about, and one from a CA of the other party. A compromised server could thus leak both short term user credentials. This might be mitigated somewhat by using an independent proxy for DCSC. If both servers support DCSC, a random, self signed certificate SHOULD be used instead of a user credential for DCAU.

Clients SHOULD send the DCSC command over an encrypted control channel.

The set of interface functions that define the DSI can be found in globus_gridftp_server.h.

All type definitions starting with globus_gfs_storage_*() are part of the DSI interface.

An API is provided to the DSI author to assist in implementation. The most interesting parts of this API provide functions that abstract away the details of sending data across the data channel. The DSI author is not expected to know the intimate details of the data channel protocols involved in a GridFTP transfer. Instead this API provides functions for reading and writing data to and from the net.

FIXME - link to api doc

The following is a brief description of part of the DSI implementation process.

An FTP session is defined from the time a client is authorized to use the server until the time the connection is disconnected (disconnect can happen due to the client sending QUIT, error, or timeout, etc). In the lifetime of the session, the client issues various commands to the FTP server. Some of these commands require access to the storage system, and thus require action by the DSI. Whenever such a command is received, the server calls out to the appropriate DSI interface function requesting that the specific operation be performed.

The server passes a

data type as a parameter to all DSI request functions. When the DSI is finished performing that operation, it calls a corresponding

function, passing it this globus_gfs_operation_t structure (and whatever other data is needed for any given operation). This lets the server know that the operation is completed and it can respond to the client appropriately.

As an example we will look at how a simple unix file system DSI would implement the stat function.

The DSI’s function signature for stat is:

When it is called, the DSI is expected to:

This is obviously a very basic example but it should serve for the purposes of understanding.

Every DSI must register itself with the Globus extensions module properly. This can be a tedious task yet must be done properly. For this reason, we created a distribution that provides a skeleton DSI upon which a developer can build.

The distribution includes a script to generate C stubs for a DSI with all of the proper shared library hooks and names needed to work with the globus-gridftp-server. The DSI implementor must fill in the stubbed-out functions with the necessary code specific to their needs.

This command will generate the c source file. "dsi name" is the string that will be associated with the DSI. It must be unique to your GCT installation. To load it into the server use the '-dsi ' option to the server.

This will compile the DSI and create the dynamically loadable library. To include additional compile dependencies or libraries, open Makefile and add them to the appropriate MACRO line. and add them to the appropriate MACRO line.

This will copy the library to $GLOBUS_LOCATION/lib, thereby making it ready for use., thereby making it ready for use.

The purpose of this work is to efficiently transfer a single data set to many locations. Our goal is to use all available network cycles, effectively moving the total amount of data (destinations * file size) at network speeds. Ideally, we would like to transfer to all destinations in the same time it takes to transfer to a single destination.

Distributing the data to many endpoints is not difficult, and has been a feature of (a popular GridFTP client) for some time. It would be quite simple to have the client loop over the destination set and send to each destination in series. Of course, this would be quite slow and the transfer time would scale linearly with the data set.

In our architecture, destination servers are configured so that they can forward packets along to other endpoints. Thus, the GridFTP servers act as both servers receiving data and clients sending data. The server is set up as a tree such that the first destination writes the data to disk and then forwards it on to N more hosts (by default N is 2). This process is repeated until all destinations have received the data.

For further explanation, the architecture can be thought of as a directed graph. Each destination is a vertex with N edges connecting it to N other vertices. A spanning tree is formed connecting all vertices. The degree of any one vertex is a client -side configuration option.

The following image illustrates this:

In the image, data blocks are purple and are sent first from the client to a root destination. The root destination then forwards it on to two more servers.

The figure in the previous section shows 4 colored boxes. Orange represents client logic, blue is server logic, and as stated above, purple is for data block. The final box type is yellow and it is used to show globus XIO drivers.

More information on Globus XIO can be found here. For our purposes we can think of each XIO driver as a modular protocol interpreter that can be plugged in to an IO stack without involving the application using it. In this way, we can add functionality to an existing application without disturbing its tested code base.

Because the uses Globus XIO for all of its IO, we are able to forward data at the block level. We achieve this by allowing the client to add a new XIO driver, the gridftp_multicast driver, to the GridFTP server’s disk stack. Because of the modular driver abstraction that Globus XIO provides as the GridFTP server writes data blocks to its file system, the data blocks are first passed through the gridftp_multicast driver. As the gridftp_multicast driver passes the data block on to be written to disk, it also forwards the block on to other GridFTP servers in the tree.

Using this approach to add the multicast functionality is minimally invasive to the tested and robust GridFTP server and is entirely modular. The driver is written to a well defined and clean abstraction. Enabling this feature is a simple matter of inserting the driver in the disk stack and passing the driver stack the destination list.

In addition to allowing for multicast, the gridftp_multicast driver and this architecture allow us to create a network overlay where many GridFTP servers act as routers forwarding packets along to each other until they get to the final destination where there are written to disk. The advantage of this type of system is actively researched by Phoebus.

The gridftp_multicast driver can be configured to only forward data along to the next server, and to not write it to disk. Furthermore, it can be told to only forward to a single endpoint. When configured in this way, we achieve the network overlay described above.

The following images illustrate this. In the first image we show the standard case where data is sent from a client to a server through the Internet. The routing is done by the Internet outside of the clients control.

In the next image we show how the gridftp_multicast driver can route data through the network via GridFTP servers. This allows the user to have greater control over the network path which the data takes.

To show the effectiveness of this architecture we ran experiments on the UC TeraGrid. We show some of those results here.

The additions to the protocol are exceptionally minor. Every server in the tree (except for leaf nodes) becomes a client to another server, but that client speaks the standard GridFTP protocol. The only change needed is a command to add the driver to the file system stack, and that command has existed in the GridFTP server for some time.

The command is:

The second parameter to the site command is a comma-separated list of driver names optionally followed by a colon (:) and a set of driver-specific URL-encoded options. From left to right, the driver names form a stack from bottom to top.

Adding the gridftp_multicast driver to this list will enable the multicast functionality. The set of options are the same as those specified in the previous section. The only difference is that each url in the urls= options must be url encoded.

The broadcast functionality can be used with globus-url-copy. We added the following option:

The file must contain a line separated list of destination urls. For example:

The source url is specified on the command line as always. A single destination url may also be specified on the command line in addition to the urls in the file. An example globus-url-copy command is: