Child pages
  • XSEDE Gateways Tutorial I: How to Create a Science Gateway Application
Skip to end of metadata
Go to start of metadata

The first XSEDE Gateways tutorial on "How to Create a Science Gateway Application" was posted to the Cornell Virtual Workshop in Spring of 2017. 

https://cvw.cac.cornell.edu/gateadmin/

Background

XSEDE CEE proposes to create a tutorial for members of the XSEDE community to help use, administer, and maintain their own science gateway. The current recommended development tool is Apache Airavata. 

Existing tutorials and documentation exist for Airavata, but there is a need for an XSEDE-specific tutorial to help our community integrate gateways into our advanced computing infrastructure. 

We are working with Marlon Pierce and his group to identify the appropriate target audience(s) for our tutorial(s). 

 

Update - November 9:

(from Marlon Pierce) For read-only admin access, you should use the PGA at https://testdrive.airavata.org.  Eroma can set you up with this. For the next step of the tutorial, I suggest you next look into some light customization of the PGA front end such as https://github.com/SciGaP/phasta-portal-theme or https://github.com/SciGaP/seagrid-website-theme.

From there, you could look more in depth at the PHP code of the PGA client itself. However, especially if you like Python, it may be more interesting to use a Jupyter notebook as the front end for a tutorial. This would should how to use the Airavata API in a “bare-bones” fashion without diving into the PGA code directly.


Update - December 8:

I proceeded to go through the tutorial for installation of a local PGA client using a hosted Airavata server. Eroma provide the following links for the most current instructions.

  1. Tutorial for a gateway user - https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Quick-Start+Tutorials or https://cwiki.apache.org/confluence/display/AIRAVATA/Tutorial+04+-+PHP+Reference+Gateway+for+Airavata+-+End-User+Guide

  2. Tutorial for a gateway admin - https://cwiki.apache.org/confluence/display/AIRAVATA/Tutorial+05+-+PHP+Reference+Gateway+for+Airavata+-+Gateway+Admin+Guide

  3. To host your own gateway, we have instructions and its for centOS

    1. Hosting Airavata - http://airavata.readthedocs.io/en/latest/Airavata-Installation/

    2. Hosting PGA - http://airavata.readthedocs.io/en/latest/PGA-Installation/

    3. For configurations of PGA -  http://airavata.readthedocs.io/en/latest/Gateway-Configurations/

I went through the tutorials in item 1 in early November.  I have also gone through the gateways admin tutorials in item 2. The process was very straightforward. I would not recommend any significant changes. We will not be focusing on this aspect of gateways for our first tutorial. 
I completed installation of the PGA in item 3 in Mac OS X, but Eroma recommends using CentOS.

Update - January 9:

I am attempting to install the Airavata server 'locally' using my Cloudera VM in Virtual Box, but I am having difficulties with the RabbitMQ server. Eroma and I are working to resolve these. 
By going through the tutorial on how to install a local PGA with a remotely-hosted Airavata server, I am still a bit unclear what I actually accomplished in terms of wanting to host my own gateway.
There is a significant amount of existing training material for gateways. Some of the outdated material should probably be removed. This is perhaps the best argument in favor of creating an XSEDE-specific tutorial. 

Update - February 17:

Brief Overview of a Science Gateway (based on conversation with Marlon Pierce, Eroma Abeysinghe, Suresh Marru)

Airavata is the middleware which uses an API.

The PGA is the reference implementation of the API. They mentioned that they need to beef it up to me more like a full-fledged gateway like CIPRES.

The PGA installation is still fairly complicated. There is some effort they need to wrap up over the next few months easily installable and configurable PGA's.

Hosting option is where they host the PGA. 

Another idea for an intermediate or advanced tutorial is to look at the API directly with Jupyter notebooks. 

One approach would be to help the audience become familiar with how to use a Gateway with the goal of providing insight into what is even possible. 

The gateway should be very intuitive, it should be designed to help guide the user to the things they need rather than simply providing step-by-step instructions. The same should also apply to a gateway administrator. 

For hosted gateways there is a lightweight version of the PGA and the gui look-and-feel options for themes can be updated through a github post. 

 

Update - April 11:

Thanks to the hard work and commitment from Marcus and Eroma, we managed to get a fully functional Airavata server with a PGA working on my local machine. This has been beneficial for all of us. Even though the Gateways team is planning to role out a packaged installation using Ansible which should greatly simplify installation, the process of working out the glitches has helped me get a better idea of the key configuration files and parameters relevant to a successful installation. 

The next step is content design and development for the first tutorial (see the First Tutorial section below). Some relatively mature documentation already exists for basic gateway admin functions:

http://airavata.readthedocs.io/en/latest/Admin-Tutorials/

Some of the material will be adapted to the selected XSEDE tutorial delivery system chosen by the gateways team (Note: a decision on the desired platform should be forthcoming). It will also include a series of short screencasts for the more challenging instructions. It may also include tutorials on things such as how to apply themes.

 

Update - June 13:

Thanks to the hard work and persistence of Eroma Abeysinghe and Supun Nakandala, we managed to successfully configure, submit, and run a Gaussian job on Comet from a PGA installed on a local machine. This is the culmination of a few months of iterative troubleshooting and confirms that the PGA works as expected. I will proceed to generate tutorial content for a brief prototype tutorial on PGA Administrator Tasks. This tutorial will be based on existing material already developed by the Airavata team but should include additional content and details not currently contained in existing material. 

 

Update - August 9:

The first version of an XSEDE Gateways Tutorial is complete and available for review at:

http://education.sdsc.edu/xsede2/gateways_tutorial/

This tutorial corresponds to the "First Tutorial" described below. 

We are working on creating an introductory screencast of the Local Echo example from the Airavata documentation. There is currently a bug which needs to be resolved before we can proceed further. 

 

Update - October 30:

The first draft of the tutorial has been reviewed and suggested updates and modifications have been made, including more information regarding multiple gateway login options and the new Sharing feature. 

A five-question quiz has also been added. A summative Exercise is currently being developed. Eroma Abeysinth and Jeff Sale have agreed to move forward with the second tutorial (see below). 


Summary of XSEDE Science Gateways Tutorials

First Tutorial

The best option for the introductory course may be something like the following scenario, e.g. here's a gateway, it doesn't do anything yet, so let's make it do something such as configuring resources, changing the look-and-feel, add a trivial application then a non-trivial application. We should provide a couple of applications, such as a Hello World and something more advanced such as a Gaussian application, maybe a Gaussian Hello World and another a little more advanced. These examples already exist and could probably be used as a starting point. 

A user can come and use the PGA as it is now, there is an allocation available to try things out, but that's not what we're thinking for a tutorial. Another way to characterize it would be something like this:

Here's a user who says ' I want to be a PGA gateway administrator. I want a gateway and it's ok if you guys host it but now I need to configure it. I have a couple of applications, Hello World and something non-trivial like Gaussian'. This may not be a valid assumption but it's a point of reference. 

The following links point to existing training material which might serve as the basis for the first XSEDE science gateways tutorial:

http://airavata.readthedocs.io/en/latest/Admin-Tutorials/

or

https://cwiki.apache.org/confluence/display/AIRAVATA/Tutorial+05+-+PHP+Reference+Gateway+for+Airavata+-+Gateway+Admin+Guide

Future Tutorials Under Consideration

Tutorial: "I want to be a PGA or SeaGRID user."

https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Quick-Start+Tutorials or https://cwiki.apache.org/confluence/display/AIRAVATA/Tutorial+04+-+PHP+Reference+Gateway+for+Airavata+-+End-User+Guide

Tutorial: "I want to have my own custom gateway based on the PGA. "

  1. To host your own gateway, we have instructions and its for centOS

    1. Hosting Airavata - http://airavata.readthedocs.io/en/latest/Airavata-Installation/

    2. Hosting PGA - http://airavata.readthedocs.io/en/latest/PGA-Installation/

    3. For configurations of PGA -  http://airavata.readthedocs.io/en/latest/Gateway-Configurations/

Fourth tutorial

I want to replace the PGA, so give me a Jupyter notebook version of the tutorial to make the replacement. 

NSF Abstract for the OGCE SciGaP.

https://www.nsf.gov/awardsearch/showAward?AWD_ID=1339774

2013 NSF proposal.


CIPRES uses the Workbench Framework.

The Science Gateways Research Center

https://sgrc.iu.edu/#home

Official Indiana University hosted site for gateway research.


The Science Gateways Group

https://rt.uits.iu.edu/visualization/gateways/


SciGaP Web Site

https://www.scigap.org

Official site for Science Gateways Platform as a Service.


SEAGrid

https://www.seagrid.org/

Science and Engineering Applications Grid supporting over 600 researchers resulting in over 150 publications.


Airavata Github

https://github.com/apache/airavata-php-gateway


Existing Instructional Resources


Latest Greatest Airavata, PGA Documentation

http://airavata.readthedocs.io/en/latest/


XSEDE 2016 Workshop Tutorial

https://scigap.atlassian.net/wiki/display/SEAGrid/Tutorials


[Outdated] Apache Airavata Documentation on Confluence

https://cwiki.apache.org/confluence/display/AIRAVATA/Apache+Airavata+Overview



  • No labels