Written by Susan Mehringer, Jay Alameda, Doug James, and the XSEDE Training team
Last updated 22 Feb 2017
Training Material Review
The XSEDE Training team, with input and help from other groups, has developed a procedure to conduct peer reviews of the training materials posted on the XSEDE User Portal (XUP). These reviews help inform roadmaps and other XSEDE-level training initiatives, and provide valuable feedback to authors to help them improve their training materials. Initially all existing materials will be reviewed, then update reviews will be performed on a rolling basis. New materials will be reviewed as they are added. Each resource will be reviewed by two people. If the reviewers rate the material ‘recommended’ or ‘highly recommended’, that designation will be posted on the course list. Review forms will be archived, but not public. Materials that are not recommended may be updated and re-submitted for review.
Reviewers will primarily be recruited from ECSS, facilitated by Jay Alameda. Reviewers from the Campus Champions group will be invited to participate by Susan Mehringer. Volunteers from other groups are also welcome. See section below for details.
- ECSS Staff
- Campus Champions
We expect that these reviews will require a commitment of time and effort roughly comparable to a peer review of a technical paper. Under normal circumstances we will ask reviewers to complete assigned reviews within one month.
Content to be reviewed
The master course list of training materials (documentation forthcoming) to be reviewed can be viewed on the XUP as the Course Catalog at:
https://portal.xsede.org/training/course-catalog; a "recommended" column will be added:
- Last review date or NULL (if never recommended)
- Recommended or NULL (if never recommended or did not qualify)
From this list, schedule notionally three materials for review per month. Priority for review is:
- New materials
- Materials with highest usage
- Date of last review
- Other appropriate factors
Based on the list and priorities listed above, select next set of materials for review, to be added to the review table (add link) in Confluence. The table consists of material title and URL. The title will be linked to JIRA, creating the review task. This table will serve as a simple overview of the materials currently in review, and their status in in JIRA. Two JIRA tasks will be created per resource to be reviewed.
- Training Lead: complete XSEDE Training Resource Description form , export to PDF.
- Training Lead: add a comment to each task including the completed PDF (previous step) and a link to XSEDE Training Resource Review form
- ESTEO/Training Leads: Identify reviewers, and assign the JIRA tasks. Tasks for non-staff volunteers will be assigned to a staff watcher.
- Reviewer: complete XSEDE Training Resource Review form; results are auto-archived in google drive
- Training Lead will anonymize the review form results, and provide a PDF of feedback to Point of Contact (e.g. Workshop lead or online training lead)
- Training Lead will update the master course list with Review date if Recommended, or NULL.
Campus Champion and Other Volunteers
Campus Champions are invited to contribute to material review on a volunteer basis. The review process would benefit from a larger pool of reviewers and the expertise, and it is hoped that Campus Champions would benefit from greater awareness and understanding of the training materials available.
- To describe the process, the Training Lead will periodically attend a Campus Champions monthly call and send an email to the list.
- Once per month, an email soliciting volunteers will be sent to the list citing the ~3 resources to be reviewed. (Select by first response, or ??)
- Volunteers who are not XSEDE staff will not be directly assigned in JIRA; rather, an XSEDE staff member will be assigned to track progress.