Child pages
  • WBS 2.3.2 Jetstream2 - XSEDE Technical Integration 2021-04-23 Meeting
Skip to end of metadata
Go to start of metadata



Agenda, Discussion, and Action Items

Agenda & Discussion


Discuss Jetstream2 technical integration with XSEDE.

Related Information

Jetstream 1 integrations

  1. RDR description
  2. Atmosphere XSEDE Web SSO (Globus Auth; login using linked identities; XSEDE recommended look and feel)
  3. Publishing and discovery in the Research Software Portal
    1. Cloud images
    2. Atmosphere user interface
  4. AMIE
  5. Cluster toolkit images (XCRI)
  6. Virtual cluster recipes (XCRI)
  7. Virtual cluster container / tutorial (XCRI)

Jetstream 1 did not integrate

  1. Horizon web app Auth integration
  2. Globus endpoints to POSIX or OpenStack Ceph storage; perhaps with High-Assurance
    1. User storage mountable from VMs
    2. VM image storage
  3. IPF published system utilization, running VMs
  4. Discoverable APIs in the Research Software Portal


Jetstream 2 characteristics

  • 4 different types of hardware each of which is a homogeneous resource
    • a compute + storage resource resource pair; two other unique compute resources
  • VMs will read-only mount a file-system of shared software
    • Two shared software binary platforms
  • Much smaller set of VM images
  • Identity and access management
    • The user identity catalog will only have the XSEDE username & their e-mail
    • Will use XSEDE project codes directly
    • Keystone based application credentials
  • OpenStatck CLIs
  • Web heavy user interfaces
    • All OAuth based using XSEDE web SSO
    • Horizon, Exosphere, and CACAO (rewrite of Atmosphere)
  • Storage
    • No data transfer nodes for the system
    • All storage in the VMs
    • Manilla available for project space
    • IU and Globus team discussing the GCS S3 storage adapter
    • Ceph cluster will be more exposed than in Jetstream 1
    • Manila to create shares and mount using CephFS
  • Globus Plus license for users
    • Users can provision their own endpoints in VMs
  • Internal Kubernetes pushbutton service, nothing XSEDE specific

Action Items

Complete by production:

  • IU team will register 4 hardware resources in RDR, and let Tabitha and JP know if new fields would be useful
  • IU team will install IPF to publish shared software LMOD modules
  • RACD with IU help will register Jetstream 2/OpenStack APIs in XSEDE's API registry
  • XCRI will provide scripts and documentation to easily provision GCS endpoints in VMs
  • RACD to convert Jetstream 1 cloud image publishing to Jetstream 2
  • RACD and IU to discuss and implement cloud utilization publishing, which may be easy to collect but more complicated to present in the XUP


Low priority that might be addressed later:

  • Whether to register regional clouds in RDR (Jeremy, Tabitha, JP)
  • Making TeraForm recipes discoverable in Information Services