As described in the XSEDE Revision Narrative submitted to the NSF at the XSEDE program's start:
There is a widely recognized need for an effective mechanism for disseminating the CI expertise resident in high-end application support staff out to the scientific community where it can take root. The CI Fellows Program will expand expertise at a campus level by sending 8 campus champions per summer to an ECSS site to work side-by-side with ECSS mentors on real world, high priority projects. Fellows will develop expertise that will enable them to teach training classes related to their newly acquired skills, and will present their experiences at a symposium before returning to their home institutions. There, working with the scientific communities using XD resources, they will serve as important conduits for collaboration and in-situ training for the community at large. Exit surveys will be used to continually improve the program.
The program has evolved at the request of Campus Champions since the initial description above to include a time commitment that can be distributed throughout the year rather than concentrated in the summer. Supported by a stipend, Campus Champion Fellows will spend 400 hours per year working on projects with ECSS staff. 8 Fellows will be selected per year, 2 per quarter.
- ECSS identifies candidate projects from current XSEDE awards
- ECSS gets buy-in from project PI
- ECSS sends list of available projects to XUP team
- projects are posted with application deadlines and promoted
- Campus Champions complete online application
- contact info (from XUP)
- description of current skills
- description of skills they'd like to acquire
- describe how these skills enhance their value as a Champion, how they would share resulting expertise with those on their campus
- description of plan for accommodating the 400-hour time commitment, including description of support from their administration
- CC and ECSS staff meet to make matches
- Selection group includes Kay Hunt, Laura McGinnis, Scott Lathrop, Campus Champions leadership team, ECSS management, ECSS staff whose projects are up for consideration
- Review selections with PIs, select different project or different Fellows as needed
- Paperwork executed between project/NCSA and Fellow
- Program evaluator sends out a â€œpreâ€? survey once the Fellows are selected to determine their expectations for the program, etc prior to them getting involved
- Fellow makes travel arrangements to visit ECSS mentor
- ECSS mentor and Fellow develop fellowship workplan
- Fellow executes project
- ECSS staff mentor provides monthly feedback to Fellow
- Fellows coordinator meets monthly with Fellows
- Fellows coordinator meets monthly with ECSS management and mentors
- Evaluator conducts interviews of both the Fellows and their matches to see how the program is evolving
- Fellows provide quarterly progress reports
- Fellow prepares a final report summarizing activities and reflecting on overall experience
- Fellow presents at symposium at XSEDE conference
- Evaluator conducts a "post" survey with Fellows to determine whether or not program is â€œsuccessfulâ€? in meeting targeted goals, objectives, and deliverables
- Evaluation and followup press
Campus Champions Fellows Program Preliminary Evaluation Outline The evaluation will be designed to answer 4 questions:
- Implementation: Is the program being implemented on schedule and as planned?
- Effectiveness: Are key components of the fellows program (e.g. needs assessment, recruitment, training, Campus Champions, ECSS staff, training materials, skill acquisition) operating effectively across various types of institutional partners? How might they be improved? What factors are associated with effective operation?
- Impact: What outcomes are associated with participation in the CC fellows program? To what extent is the program disseminating the cyberinfrastructure expertise resident in high-end application support staff to the broader scientific community? What is the value-added of participation in the program at the individual and institutional levels? How does impact vary across user groups (e.g. faculty, AUSS staff, fellows, Campus Champions; racial and ethnic subgroups)?
- Institutionalization: How and to what extent are elements of the CC fellows program becoming institutionalized at the local, regional, and national level? What opportunities and barriers exist? What factors are associated with institutionalization?
Methods to employ in response to evaluation questions:
Implementation: Using simple monthly reporting software and follow-up interviews with key implementers (at all partner institutions); evaluators will routinely monitor program implementation and report on progress, challenges and slippage at each management committee meeting. When implementation problems or slippage occur, the management committee will determine strategies for overcoming barriers and remaining on track.
Effectiveness: With input of project leadership, the external evaluation will establish metrics and data collection strategies for each of the program objectives.
Impact: The external evaluation will implement a web-based survey system for carefully tracking all program participants over time. The system will capture entry characteristics, program participation, subsequent use of program materials and services; application of knowledge and skills gained, research, educational, and career outcomes. We will examine the value-added of program participation by comparing key outcomes (e.g. diversity, research, curriculum development, publications, presentations, awards, retention, continued education, satisfaction, and within the timeline of funding time to degree and initial employment) with baseline data from each institution.
Institutionalization: Through annual surveys, interviews with institutional leadership and key stakeholders, review of curriculum and program development, and other means, the evaluation will also examine the institutional changes that occur as a result of CC fellows program including changes in fellow and faculty knowledge and skills, curriculum reform, and institutionalization of training and other program elements, etc. We will also attempt to identify barriers to implementation so that they might be addressed by university and project leadership.