You will need an account with access to the GridFTP log files which will be used for uploading these logs to the log collection server. The easiest way to do this is simply to configure your "globus" user (sometimes called "globust" or "gridftp") to perform the uploads, but you may choose to use a Unix Group setup or some other mechanism to allow a specific user account to access the logs for upload. Once you have decided on a user account to perform the griftp transfers, make sure that that account is available at Wrangler at TACC as well. If this is not the case, please create an account for the same user through the XSEDE User Portal, requesting access to the Wrangler resource. Also, finalize with Wrangler admins on where the files should be transferred to on Wrangler.
Setting up Log Rotation
Most systems will use the "logrotate" software tool to manage the rotation of system logs, and this tool can easily be extended to support rotation of the GridFTP transfer logs. You will need to know the location and name of these transfer logs, and you will need to decide on the frequency of log rotation. You may need to consult your system-specific documentation for specifics on how to configure certain options for log rotation. On CentOS/RedHat-based systems, you can create a file named /etc/logrotate.d/gridftp with the following contents to configure daily log rotation (this example assumes you are following the examples from the XSEDE GridFTP installation guide. Replace /var/log/gridftp/ with the actual path to your GridFTP transfer logs if you are using a different location):
/var/log/gridftp/*.log {
daily
missingok
compress
rotate 60
notifempty
create 644 root root
prerotate
sudo -u {transfers_usersname} {path_to_transfer_script}
endscript
postrotate
systemctl restart globus-gridftp-server.service
endscript
}
Here, transfers_username is the XSEDE account that is used to transfer files from your site to TACC’s Wrangler. The {path_to_transfer_script} is the script that will be used to transfer scripts using gridftp from your site to Wrangler. The script is outlined in the section below.
Setting up Daily log uploads
Regardless of whether you are using log rotation, SPs should upload logs for each of their GridFTP server instances on a regular (monthly is prefered) basis to ensure that the XSEDE centralized transfer information service has fresh data. Assuming that your server is not configured to rotate logs on a monthly basis, you can use the following script in combination with cron to accomplish this monthly upload. This is also the script that is referenced in the prerotate command above to transfer files while using logrotate.
cat /nics/c/home/xsede/nics_dtn/transfergridftplog.sh
#!/bin/bash
# specify where the user certificate will be stored
export X509_USER_PROXY=/nics/c/home/xsede/nics_dtn/x509up_u94
# get the logfile name
logfile=$(ls /var/log/gridftp | grep 'datamover[0-9].log$')
# get current date to append to filename that is transferred
datename=`date +%F`
# transfer log file to Wrangler location using gridftp
globus-url-copy /var/log/gridftp/$logfile gsiftp://gridftp.wrangler.tacc.utexas.edu/data/projects/xsede/gridftp-stats/nics/$logfile"_"$datename