XSEDE Home
Space shortcuts
Child pages
  • XSEDE Gateways Tutorial I: How to Create a Science Gateway Application

Versions Compared

Old Version 20

changes.mady.by.user Jeff Sale

Saved on

compared with

New Version Current

changes.mady.by.user Jeff Sale

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The first XSEDE Gateways tutorlal on Gateways Application Creation tutorial on "How to Create a Science Gateway Application" was posted to the Cornell Virtual Workshop in Spring of 2017. 

...

Summary of XSEDE Science Gateways Tutorials

First Tutorial

The best option for the introductory course may be something like the following scenario, e.g. here's a gateway, it doesn't do anything yet, so let's make it do something such as configuring resources, changing the look-and-feel, add a trivial application then a non-trivial application. We should provide a couple of applications, such as a Hello World and something more advanced such as a Gaussian application, maybe a Gaussian Hello World and another a little more advanced. These examples already exist and could probably be used as a starting point. 

...

https://cwiki.apache.org/confluence/display/AIRAVATA/Tutorial+05+-+PHP+Reference+Gateway+for+Airavata+-+Gateway+Admin+Guide

XSEDE Gateways Tutorial II: "How to Create a Hosted Gateway"

Rationale

The gateways community is realizing that not many users want to install their own gateways, only the larger projects are considering this. There are numerous smaller projects coming on board XSEDE who have research or community code they want to make available to their community and they will get their own XSEDE startup allocation and then maybe they will get an XRAC or encourage their users to get their own allocations and expand accordingly. 

There are a growing number of service providers who do not want to operate their own gateway. This has been a trend in the SG community including many of the popular frameworks such as Galaxy, Airavata, Agave, Hubzero. The focus should be on what the community needs rather than what we think they need. 
Where would training help?
Training would help by providing a decision-making process to help guide the user to the appropriate gateway development scenarios for their specific needs. It won't focus on a particular software but rather on the process of identifying the framework and the implementation strategy.
Globus may serve as an example. Early on, users needed to install and run gridftp and work mainly at the command line. Now they work mainly through Globus Online. There is a shift from installing and running one's own software packages to using services hosted and maintained by others. 
Our focus is on how to create a new gateway (as opposed to how to use a gateway), but it involves a somewhat different approach. Google may serve as another example. Users can create and host their own web sites and host their own email servers, etc. but Google helps make it alot easier by providing those services. Our users will be learning whether they want to host their own gateway or whether they want a hosted gateway, and if it's the latter then we help guide them down that path. If it's the former, then we point them to the existing CVW tutorial for gateway admins. 
A growing number of XSEDE researchers have either developed or are using the same software and would benefit by gaining access to it via a community gateway. The general scenario is "I have this code and I want to build a gateway for it". What do I need to do? To do it themselves can be daunting. We propose providing a tutorial involving a step-by-step process for working with gateways service providers to simplify and expedite the process. 
Training, Not Documentation
We classify this as a training effort rather than a documentation effort. There already is alot of documentation but it is not well-organized. We propose adding some order to the chaos. Susan expressed her interest in screencasts for some of the more complex steps in the process. We also felt that it may help to use something similar to a case study approach by identify some exemplar gateways which embody the type of gateway development scenarios we plan to cover. 
There may be only 4 or 5 popular gateway frameworks and services. We want to focus on creating a tutorial which encompasses these most popular options while still keeping the tutorial as general as possible. Jetstream can hopefully serve as a way of making gateway hosting and administration somewhat of a 'cookie-cutter' approach. There may also be existing XSEDE users who could help by working with us to provide insights into the process as they went through it or are going through it. Suresh singled out dReg and Simvascular as examples of 'typical' gateways hosted for the community. 
Update, May 7, 2018:
Suresh, Susan, and Jeff met to discuss first steps towards this tutorial. The growing demand for hosted gateways (as opposed to installing a standalone gateway) suggests the need for a different tutorial designed to help them with this process. Here are notes from that meeting:
 

Suresh helped lay out a sequence of steps users might typically follow in order to implement their own hosted gateway. Some resources already exist for this process but they are lacking sufficient details and are somewhat disconnected. The primary goal of this tutorial would be to integrate these resources into a coherent whole. Screencasts may play a bigger role in this tutorial. Options for hosting include Jetstream which is nice because tutorials already exist for obtaining a Jetstream allocation and getting started with Jetstream. There may be some existing gateways which could serve as exemplars or used for simple case studies. 

 

The gateways community is realizing that not many users want to install their own gateways, only the larger projects are considering this. There are lots of smaller projects coming on board XSEDE who have research or community code they want to make available to their community and they will get their own XSEDE startup allocation and then maybe they will get an XRAC or encourage their users to get their own allocations and expand accordingly. 

 

There are a growing number of service providers who do not want to operate their own gateway. This has been a trend in the SG community including many of the popular frameworks such as Galaxy, Airavata, Agave, Hubzero. The focus should be on what the community needs rather than what we think they need. 

Where would training help? In providing a decision-making process to help guide the user to the appropriate gateway development scenarios for their specific needs. It won't focus on a particular software but rather on the process of identifying the framework and the implementation strategy. 

Suresh used Globus as an example. Early on, users needed to install and run gridftp and work mainly at the command line. Now they work mainly through Globus Online. There is a shift from installing and running one's own software packages to using services hosted and maintained by others. 

We are still talking about how to create a new gateway, but it involves a different approach. Suresh used Google as another example. Users can create and host their own web sites and host their own email servers, etc. but Google helps make it alot easier by providing those services. Our users will be learning whether they want to host their own gateway or whether they want a hosted gateway, and if it's the latter then we help guide them down that path. If it's the former, then we point them to the existing CVW tutorial for gateway admins. 

At this point Suresh shared his screen and walked Susan through the same series of pages on the XSEDE web site which you and I walked through last week, including Simvascular. He also mentioned a scenario involving Cornell faculty Charles Danko who has software and wants to avoid making everyone download the software by offering a gateway. 

The general scenario is "I have this code and I want to build a gateway for it". What do I need to do? To do it themselves can be daunting. We propose providing a tutorial involving a step-by-step process for working with gateways service providers to simplify and expedite the process. 

We classify this as a training effort rather than a documentation effort. There already is alot of documentation but it is not well-organized. We propose adding some order to the chaos. Susan expressed her interest in screencasts for some of the more complex steps in the process. We also felt that it may help to use something similar to a case study approach by identify some exemplar gateways which embody the type of gateway development scenarios we plan to cover. 

There may be only 4 or 5 popular gateway frameworks and services. We want to focus on creating a tutorial which encompasses these most popular options while still keeping the tutorial as general as possible. 

Jetstream serves as a way of making gateway hosting and administration somewhat of a 'cookie-cutter' approach. 

Suresh felt there may be existing XSEDE users who could help by working with us to provide insights into the process as they went through it or are going through it. Suresh singled out dReg and Simvascular as examples of 'typical' gateways hosted for the community. 

We discussed more specifics and set priorities during our bi-weekly meeting on May 17 2018.
Update - June 29, 2018:
We are working on a flow chart which delineates the key steps in the decision-making process for deciding which path to choose for hosting a gateway.
Below is phase 1 of the chart (click to enlarge):
Image Removed

 

 

Future Tutorials Under Consideration

Tutorial: "I want to be a PGA or SeaGRID user."

https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Quick-Start+Tutorials or https://cwiki.apache.org/confluence/display/AIRAVATA/Tutorial+04+-+PHP+Reference+Gateway+for+Airavata+-+End-User+Guide

Tutorial: "I want to have my own custom gateway based on the PGA. "

  1. To host your own gateway, we have instructions and its for centOS

    1. Hosting Airavata - http://airavata.readthedocs.io/en/latest/Airavata-Installation/

    2. Hosting PGA - http://airavata.readthedocs.io/en/latest/PGA-Installation/

    3. For configurations of PGA -  http://airavata.readthedocs.io/en/latest/Gateway-Configurations/

Fourth tutorial

I want to replace the PGA, so give me a Jupyter notebook version of the tutorial to make the replacement. 

NSF Abstract for the OGCE SciGaP.

...

https://github.com/apache/airavata-php-gateway


Existing Instructional Resources


Latest Greatest Airavata, PGA Documentation

...