Child pages
  • Copy of XSEDE Project-wide KPIs & Metrics
Skip to end of metadata
Go to start of metadata

This page takes a few seconds for the tables to load. Thank you for your patience!

More Information: What is a KPI - KPI Guidelines - Annual KPI Reviews 

1    Executive Summary

Computing across all fields of scholarship is becoming ubiquitous: digital technologies underpin, accelerate, and enable new, even transformational, research in all domains. Researchers continue to integrate an increasingly diverse set of distributed resources and instruments directly into their research and educational pursuits. Access to an array of integrated and well-supported high-end digital services is critical for the advancement of knowledge. XSEDE (the Extreme Science and Engineering Discovery Environment) is a socio-technical platform that integrates and coordinates advanced digital services within the national ecosystem to support contemporary science. This ecosystem involves a highly distributed, yet integrated and coordinated, assemblage of software, supercomputers, visualization systems, storage systems, networks, portals and gateways, collections of data, instruments and personnel with specific expertise. XSEDE fulfills the need for an advanced digital services ecosystem distributed beyond the scope of a single institution and provides a long-term platform to empower modern science and engineering research and education. As a significant contributor to this ecosystem, driven by the needs of the open research community, XSEDE substantially enhances the productivity of a growing community of scholars, researchers, and engineers. XSEDE federates with other high-end facilities and campus-based resources, serving as the foundation for a national e-science infrastructure with tremendous potential for enabling new advancements in research and education. Our vision is a world of digitally-enabled scholars, researchers, and engineers participating in multidisciplinary collaborations while seamlessly accessing computing resources and sharing data to tackle society’s grand challenges.

Researchers use advanced digital resources and services every day to expand their understanding of our world. More pointedly, research now requires more than just supercomputers, and XSEDE represents a step toward a more comprehensive and cohesive set of advanced digital services through our mission: to substantially enhance the productivity of a growing community of scholars, researchers, and engineers through access to advanced digital services that support open research; and to coordinate and add significant value to the leading cyberinfrastructure resources funded by the NSF and other agencies

XSEDE has developed its strategic goals in a manner consistent with NSF’s strategic plan, Investing in Science, Engineering, and Education for the Nation's Future: NSF Strategic Plan for 2014 - 2018,1 NSF’s strategies stated broadly in the Cyberinfrastructure Framework for 21st Century Science and Engineering,2 vision document, and the more specifically relevant Advanced Computing Infrastructure: Vision and Strategic Plan3 document.

 

1.1       Strategic Goals

To support our mission and to guide the project’s activities toward the realization of our vision, three strategic goals are defined: 

Deepen and Extend Use: XSEDE will deepen the use—make more effective use—of the advanced digital services ecosystem by existing scholars, researchers, and engineers, and extend the use to new communities. We will contribute to preparation—workforce development—of the current and next generation of scholars, researchers, and engineers in the use of advanced digital services via training, education, and outreach; and we will raise the general awareness of the value of advanced digital services

Advance the Ecosystem: Exploiting its internal efforts and drawing on those of others, XSEDE will advance the broader ecosystem of advanced digital services by creating an open and evolving e-infrastructure, and by enhancing the array of technical expertise and support services offered. 

Sustain the Ecosystem: XSEDE will sustain the advanced digital services ecosystem by ensuring and maintaining a reliable, efficient, and secure infrastructure, and providing excellent user support services. XSEDE will further operate an effective, productive, and innovative virtual organization

The strategic goals of XSEDE cover a considerable scope. To assure we are delivering our mission and to assess progress toward our vision, we have identified key metrics to measure our progress toward meeting each sub-goal. These key performance indicators (KPIs) are a high-level encapsulation of our project metrics that measure how well we are meeting each sub-goal. Planning is driven by our vision, mission, goals, and these metrics— which are in turn rooted in the needs and requirements of the communities we serve. 

The key concept is not that the KPIs themselves must have a direct causal effect on eventual outcomes, or measure eventual outcomes or long-term impacts, but rather that the KPIs are chosen so that actions and decisions which move the metrics in the desired direction also move the organization in the direction of the desired outcomes and goals. 

Table 1-1 below shows the project’s three strategic goals and associated sub-goals along with the KPIs used to measure progress toward achieving those goals.

Table 1-1: Summary of key performance indicators (KPIs) for XSEDE.

Strategic Goals

Sub-goals

KPIs

Deepen and Extend Use

Deepen use (existing communities)

  • Number of completed ECSS projects
  • Average ECSS impact rating
  • Average satisfaction with ECSS support

Extend use (new communities)

  • Number of new users from underrepresented communities and non-traditional disciplines of XSEDE resources and services
  • Number of sustained users from underrepresented communities and non-traditional disciplines of XSEDE resources and services

Prepare the current and next generation

  • Number of attendees in synchronous and asynchronous training
  • Average impact assessment of training for attendees registered through the XSEDE User Portal

Raise awareness of the value of advanced digital services

  • Number of pageviews to the XSEDE website
  • Number of pageviews to the XSEDE User Portal
  • Number of social media impressions
  • Number of media hits

Advance the Ecosystem

Create an open and evolving e-infrastructure

  • Number of new capabilities made available for production deployment
  • Average satisfaction rating of XCI services

Enhance the array of technical expertise and support services

  • Average rating of staff regarding how well-prepared they feel to perform their jobs

Sustain the Ecosystem

Provide reliable, efficient, and secure infrastructure

  • Average composite availability of core services
  • Hours of downtime with direct user impacts from an XSEDE security incident

Provide excellent user support

  • Mean time to ticket resolution
  • Average user satisfaction ratings for allocations and other support services

Operate an effective and productive virtual organization

  • Percentage of recommendations addressed by relevant project areas

Operate an innovative virtual organization

  • Number of staff publications
  • Number of key improvements addressed from systematic evaluation
  • Number of key improvements addressed from external sources
  • Ratio of proactive to reactive improvements

[Table of Contents]

2    Discussion of Strategic Goals and Key Performance Indicators

The strategic goals of XSEDE cover a considerable scope. Additionally, the specific activities within our scope are often very detailed; therefore, to ensure that this significant and detailed scope will ultimately deliver our mission and realize our vision, we decompose the three strategic goals into components or sub-goals to be considered individually. 

In determining the best measures of progress toward each of the sub-goals, KPIs that correlate to impact on the scientific community are used. These often pair measurements of outcome with an assessment of quality or impact to provide both a sense of scope and significance of the supporting activities. In some cases, metrics for impact, outcome, or both are not currently being collected. In these cases, the best available metric is used. 

 

2.1       Deepen and Extend Use

XSEDE will 1) deepen the use—make more effective use—of the advanced digital services ecosystem by existing scholars, researchers, and engineers and 2) extend the use to new communities. We will 3) contribute to preparation—workforce development—of scholars, researchers, and engineers in the use of advanced digital technologies via training, education, and outreach; and we will 4) raise the general awareness of the value of advanced digital research services

 

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

2.1.2 Extend Use to New Communities

New communities are defined as new fields of science, industry, and under-represented communities. New fields of science are those that represent less than 1% of XSEDE Resource Allocation Committee (XRAC) allocations. The Novel & Innovative Projects (NIP) team and the Broadening Participation team both work to bring advanced digital services to new communities. XSEDE measures both the number of new users and the number of sustained users on research projects from under-represented communities and non-traditional disciplines of XSEDE resources and services as the indicators of progress (Table 2-2).

Table 2-2: KPIs for the sub-goal of extend use (new communities).

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1 - Target has been updated as NIP updated target for RY1RP4.

2.1.3 Prepare the Current and Next Generation

Part of XSEDE’s mission is to provide a broad community of existing and future researchers with access and training to use advanced digital services via the sub-goal of preparing the current and next generation of computationally-savvy researchers. While many activities support this sub-goal, such as the various Champion (§3.6), Student Engagement (§3.4), and Education (§3.2) programs, the training offered through Community Engagement & Enrichment (CEE) impacts the most people directly. This, and a complementary measure of impact as indicated by those same individuals, are therefore considered the key indicators (Table 2-3) of performance toward this goal. 

Table 2-3: KPIs for the sub-goal of prepare the current and next generation

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1 The target total for synchronous and asynchronous attendees (2,000 and 4,000, respectively) was previously listed as 5,600 / yr. This was an error. The target total should be 6,000.

2.1.4 Raise Awareness of the value of advanced digital services

While many PY1 activities, such as our Workforce Development (§3.2), User Engagement (§3.3) and Broadening Participation (§3.4) efforts, and the visibility of our Champions and other Campus Engagement (§3.6) activities, contribute to our efforts to measure our ability to raise the general awareness of the value of advanced digital research services, we have chosen to focus on measures in four areas (Table 2-4): website, social media, public relations, and media hits. Desirable trends in these key outcomes can be correlated to success for this sub-goal. 

Table 2-4: KPIs for the sub-goal of raise awareness of the value of advanced digital research services

 

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

2.2       Advance the Ecosystem

Exploiting its internal efforts and drawing on those of others, XSEDE will advance the broader ecosystem of advanced digital services by 1) creating an open and evolving e-infrastructure, and by 2) enhancing the array of technical expertise and support services offered. 

2.2.1 Create an Open and Evolving e-Infrastructure

There are a variety of factors that affect the evolution of the e-infrastructure. These range from external factors, such as the number of XSEDE Federation members and the variety of services they provide, to internal factors, like Operations (§6) of critical infrastructure and services and the evaluation and integration of new capabilities. While we actively seek new Federation members and Service Providers, and partnerships with national and international cyberinfrastructure projects, we view our role as connector of these elements to be the most impactful. Thus, XSEDE focuses on the number of new capabilities made available for production deployment along with the satisfaction rating of all XSEDE Community Infrastructure (XCI) services as indicators of performance with respect to this sub-goal (Table 2-5). 

Table 2-5: KPIs for the sub-goal of create an open and evolving e-infrastructure

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1This number is for RACD only as XCRI satisfaction numbers are unavailable for this reporting period.

 

2.2.2 Enhance the Array of Technical Expertise and Support Services

To enhance the technical expertise of our staff to offer an evolving set of support services, we will continue many activities including workshops, symposia, and training events hosted by Extended Collaborative Support Services (ECSS) and Service Providers (§4.6). The average rating by staff for training will continue to be a KPI; the staff climate survey question that measures this has been changed to focus on staff having the training they need, instead of staff turning to XSEDE training to fulfill these needs (Table 2-6). This represents a change in approach to staff training that reflects budget constraints, where we will leverage existing training offered by the Service Providers, universities, and professional associations alongside our own training to enhance the expertise of staff.

Table 2-6: KPIs for the sub-goal of enhance the array of technical expertise and support services.

 

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

- Data reported annually.  Due to the shortened reporting year, the annual XSEDE Staff Climate Study did not occur during RY1. 

^ The number 3.70 was erroneously reported in RY1, RP2.

 

2.3       Sustain the Ecosystem

XSEDE will sustain the advanced digital services ecosystem by 1) ensuring and maintaining a reliable, efficient, and secure infrastructure, and 2) providing excellent user support services. Furthermore, XSEDE will operate an 3) effective, 4) productive, and 5) innovative virtual organization

2.3.1 Provide Reliable, Efficient, and Secure Infrastructure

Many activities support this sub-goal—such as User Information & Interfaces (§3.5), Security (§6.2), Data Transfer Services (§6.3), Systems Operations and Support (§6.5), support for Allocations (§7.2), and Allocations, Accounting & Account Management (§7.3)—but perhaps the truest measure of an infrastructure’s reliability is its robustness as reflected by sustained availability. Thus, the KPI for this sub-goal is a composite measure of the availability of XSEDE infrastructure components and the number of hours of downtime with direct user impacts from security incidents (Table 2-7). The composite measure is a geometric mean of the availability of critical enterprise services and the XRAS allocations request management service. 

Table 2-7: KPIs for the sub-goal of provide reliable, efficient, and secure infrastructure

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

2.3.2 Provide Excellent User Support 

Although nearly every group in the organization has some support function, we have chosen to focus on metrics with respect to two primary support interfaces to the community: the XSEDE Operations Center (XOC) metrics with respect to ticket resolutions times and the average satisfaction rating of the allocations process. The XOC is the frontline centralized support group that either resolves or escalates tickets to the appropriate resolution center depending on the request. Two KPIs are measured (Table 2-8): the mean time to resolution on support tickets that are resolved by the XOC or routed to, and resolved by, other XSEDE areas, and the average satisfaction rating for the allocations process measured via a quarterly survey of users who have interacted with the allocations request system and the allocations process more generally. 

Table 2-8: KPIs for the sub-goal of provide excellent user support

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

2.3.3 Effective and Productive Virtual Organization

During the first five years of XSEDE, in conjunction with developing a methodology for driving and assessing performance excellence, XSEDE adopted the Baldrige Criteria 4 and has assessed and applied applicable criteria from all seven criteria by that methodology. These include annual reviews of the vision, mission, strategic goals, project-wide processes and standards (KPIs and area metrics); user and staff surveys (§3.3, §8.5); stakeholder communications (§8.2); advisory boards (§8.1); community engagement (§3); workforce enhancement (§3.2); and the analysis of organizational data that leads to organizational learning, strategic improvement and innovation. Thus, XSEDE has chosen to monitor its continuous improvement efforts as an organizational measurement of recommendations addressed by the relevant project areas (Table 2-9). The idea here is that a stable or expanding number of improvements shows that XSEDE continues to systematically evaluate the organization and make informed, proactive improvements. 

Table 2-9: KPIs for the sub-goal of operate an effective and productive virtual organization.

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

2.3.4 Innovative Virtual Organization

Measuring innovation for an organization like XSEDE (or for organizations in general) is difficult and represents an area of open research. A partial measure is the number of staff publications produced since this shows that XSEDE staff is involved in novel activities that achieve peer-reviewed publication. Additionally, after much thought and discussion both internally and with external stakeholders and advisors, we have identified two additional indicators that strongly correlate to innovation: 1) the ratio of proactive organizational improvements to those that were reactive and 2) the number of improvements that are innovative or lead to innovations. The first indicator is a measurement of organizational maturity and agility; the second measures innovative actions directly (Table 2-10). While these provide some indication of innovation, they are still not satisfactory. These KPIs will continue to be the subject of an open conversation within the organization and with stakeholders and advisors as XSEDE assesses these measurements and how to best quantify innovation. In particular, this will be discussed in earnest within the Strategy, Planning, Policy, Evaluation & Organizational Improvement team (§8.5). 

Table 2-10: KPIs for the sub-goal of operate an innovative organization

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1This metric is one of two new metrics that resulted from splitting a former metric, called "Number of strategic or innovative improvements," into two. The former metric had a target of 9/yr and results were 3 for RP2 and 8 for RP3 in RY1. These two new metrics each have a target of 10/yr and will be reported annually. 

2Number was updated from what was originally reported due to reporting error.

3L2 Directors are currently responding to climate study recommendations; data will be available in RY2 RP2. 

[Table of Contents]

3       Community Engagement & Enrichment - CEE (WBS 2.1) 

At the core of Community Engagement & Enrichment (CEE) is the researcher, broadly defined to include anyone who uses or may potentially use the array of resources and services offered by XSEDE. The CEE team is dedicated to actively engaging a broad and diverse cross-section of the open science community, bringing together those interested in using, integrating with, enabling, and enhancing the national cyberinfrastructure. Vital to the CEE mission is the persistent relationship with existing and future users, including allocated users, training participants, XSEDE collaborators, and campus personnel. CEE will unify public offerings to provide a more consistent, clear, and concise message about XSEDE resources and services, and bring together those aspects of XSEDE that have as their mission teaching, informing, and engaging those interested in advanced cyberinfrastructure.

The five components of CEE are Workforce Development (§3.2), which includes Training, Education and Student Preparation, User Engagement (§3.3), Broadening Participation (§3.4), User Interfaces & Online Information (§3.5) and Campus Engagement (§3.6). These five teams ensure routine collection and reporting of XSEDE’s actions to address user requirements. They provide a consistent suite of web-based information and documentation and engage with a broad range of campus personnel to ensure that XSEDE’s resources and services complement those offered by campuses. Additionally, CEE teams expand workforce development efforts to enable many more researchers, faculty, staff, and students to make effective use of local, regional, and national advanced digital resources. CEE expands efforts to broaden the diversity of the community utilizing advanced digital resources. The CEE team tightly coordinates with the rest of XSEDE, particularly Extended Collaborative Support Services (§4), Resource Allocation Services (§7), XSEDE Community Infrastructure (§5), and External Relations (§8.2). 

CEE is focused on personal interactions, ensuring that existing users, potential users and the general public have sufficient access to materials and have a positive and effective experience with XSEDE public offerings and frontline user support. As such, the CEE area metrics are designed to broadly assess this performance. CEE focuses on metrics that quantify how many users in aggregate are benefiting from XSEDE resources and services. Additionally, CEE focuses on how well the user base is sustained over time and how well training offerings evolve with changing user community needs.

Table 3-1: Area Metrics for Community Engagement & Enrichment 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1  Number was updated from original reported as this number was calculated incorrectly.

2The target total for synchronous and asynchronous attendees (2,000 and 4,000, respectively) was previously listed as 5,600 / yr. This was an error. The target total should be 6,000.

3.1 CEE Director's Office (WBS 2.1.1)

The CEE Director’s Office has been established to provide the necessary oversight to ensure the greatest efficiency and effectiveness of the CEE area. This oversight includes providing direction to the L3 management team, coordination of, and participation in, CEE planning activities and reports through the area’s Project Manager, and monitoring compliance with budgets, and retarget effort if necessary. The Director’s Office also attends and supports the preparation of project level reviews and activities.

The CEE Director’s Office will continue to manage and set direction for CEE activities and responsibilities. They will contribute to and attend bi-weekly senior management team calls, contribute to the project level plan, schedule, and budget, contribute to XSEDE quarterly, annual, and other reports as required by the NSF and attend XSEDE quarterly and annual meetings.Lastly, the Director’s Office will advise the XSEDE PI on many issues, especially those relevant to this WBS area.

3.2 Workforce Development (WBS 2.1.2) 

The Workforce Development mission is to provide a continuum of learning resources and services designed to address the needs and requirements of researchers, educators, developers, integrators, and students utilizing advanced digital resources. This includes providing professional development for XSEDE team members.

Workforce Development provides an integrated suite of training, education, and student preparation activities to address formal and informal learning about advanced digital resources addressing the needs of researchers, developers, integrators, IT staff, XSEDE staff, faculty, and undergraduate and graduate students. CEE –Workforce Development provides business and industry with access to XSEDE’s workforce development efforts including training services and student internships that have historically proven beneficial to industry.

Workforce Development is comprised of three areas: Training, Education, and Student Preparation. The Training team develops and delivers training programs to enhance the skills of the national open science community and ensure productive use of XSEDE’s cyberinfrastructure. The Education team works closely with Training and Student Preparation to support faculty in all fields of study with their incorporation of advanced digital technology capabilities within the undergraduate and graduate curriculum. The Student Preparation program actively recruits students to use the aforementioned training and education offerings to enable the use of XSEDE resources by undergraduate and graduate students to motivate and prepare them to pursue advanced studies and careers to advance discovery and scholarly studies.

Table 3-2: Area Metrics for Workforce Development.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

- Data reported annually. 

1Not measured yet. Survey will go out in RY2 RP1. 

2Number was updated from original reported in RP1 (276) as this number was calculated incorrectly. 

3.3 User Engagement (WBS 2.1.3) 

The mission of the User Engagement (UE) team is to capture community needs, requirements, and recommendations for improvements to XSEDE’s resources and services, and report to the national community how their feedback is being addressed. XSEDE places an emphasis on maintaining consistent user contact, traceability in tracking user issues, and closing the feedback loop.

Table 3-3: Area Metrics for User Engagement.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1Resolution may be dependent upon SPs and other XSEDE groups.

2Query was not catching all PIs, modified for RP4.

**Quarterly metric, 5,181 total contact emails sent in RP2-RP4. 

3.4 Broadening Participation (WBS 2.1.4) 

Broadening Participation engages under-represented minority researchers from domains that are not traditional users of HPC and from Minority Serving Institutions. This target audience ranges from potential users with no computational experience to computationally savvy researchers, educators, Champions, and administrators that will promote change at their institutions for increased use of advanced digital services for research and teaching. 

Broadening Participation will continue the most effective recruitment activities— conference exhibiting, campus visits, and regional workshops—while increasing national impact through new partnerships and the utilization of lower cost awareness strategies to continue the growth in new users from underrepresented communities. The Diversity Forum and the Minority Research Community listservs and community calls focus on user persistence in their use of XSEDE services and their deepening engagement through participation in committees such as the User Advisory Committee (UAC) and XRAC, participation in Champions, bridging, and other programs. Persistent institutional engagement is enabled by curriculum reform and larger numbers of researchers adopting the use of advanced digital resources as a standard research method.

Table 3-4: Area Metrics for Broadening Participation.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

- Data reported annually 

1 The staff climate survey for this plan year is underway and data is not available to report at this time. 

3.5 User Interfaces & Online Information (WBS 2.1.5) 

User Interfaces & Online Information (UII) is committed to enabling the discovery, understanding, and effective utilization of XSEDE’s powerful capabilities and services. Through UII’s ongoing effort to improve and engage a variety of audiences via the XSEDE web site and user portal, UII has immediate impact on a variety of stakeholders including the general public, potential and current users, educators, services providers, campus affiliates, and funding agencies. These stakeholders will gain valuable information about XSEDE through an information rich website, the XSEDE User Portal, and a uniform set of user documentation.

Table 3-5: User Interfaces & Online Information.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

- Data reported annually 

1 Number was updated from original reported as this number was initially calculated incorrectly.

2 This measurement is based on an XSEDE Microsurvey focused on XSEDE User Documentation. The measurement is the mean satisfaction rating for the question “Please rate your level of satisfaction with the technical documentation available on the XSEDE User Portal (XUP) and XSEDE Web Site." 

 

3.6 Campus Engagement (WBS 2.1.6) 

The Campus Engagement program promotes and facilitates the effective participation of a diverse national community of campuses in the application of advanced digital resources and services to accelerate discovery, enhance education, and foster scholarly achievement. 

Campus Engagement, via the Campus Champions, works directly with institutions across the US both to facilitate computing and data-intensive research and education, nationally and with collaborators worldwide, and to expand the scale, scope, ambition and impact of these endeavors. This is done by increasing scalable, sustainable institutional uptake of advanced digital services from providers at all levels (workgroup, institutional, regional, national, international), fostering a broader, deeper, more agile, more sustainable and more diverse nationwide cyberinfrastructure ecosystem across all levels, and cultivating inter-institutional interchange of resources, expertise and support. CE also aims to assist with the establishment and expansion of consortia (for example, intra-state, regional, domain-specific) that collaborate to better serve the needs of their advanced computing stakeholders.

Table 3-6: Area Metrics for Campus Engagement.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

[Table of Contents]

4       Extended Collaborative Support Service - ECSS (WBS 2.2) 

The Extended Collaborative Support Service (ECSS) improves the productivity of the XSEDE user community both through successful, meaningful collaborations and well-planned training activities. The goal is to optimize applications, improve work and data flows, increase effective use of the XSEDE digital infrastructure and broadly expand the XSEDE user base by engaging members of under-represented communities and domain areas. The ECSS program provides professionals who can be part of a collaborative team—dedicated staff who develop deep, collaborative relationships with XSEDE users—helping them make best use of XSEDE resources to advance their work. These professionals possess combined expertise in many fields of computational science and engineering. They have a deep knowledge of underlying computer systems and of the design and implementation principles for optimally mapping scientific problems, codes, and middleware to these resources. ECSS includes experts in not just the traditional use of advanced computing systems but also in data-intensive work, workflow engineering, and the enhancement of scientific gateways. 

ECSS projects fall into five categories: Extended Support for Research Teams (ESRT), Novel and Innovative Projects (NIP), Extended Support for Community Codes (ESCC), Extended Support for Science Gateways (ESSGW), and Extended Support for Training, Education and Outreach (ESTEO). Project-based ECSS support is requested by researchers via the XSEDE peer-review allocation process. If reviewers recommend support and if staff resources are available, the ECSS expert and the requesting PI develop a work plan outlining the project tasks. The work plan includes concrete quarterly goals and staffing commitments from both the PI team and ECSS. ECSS managers review work plans and also track progress via quarterly reports. 

Table 4-1: Area Metrics for Extended Collaborative Support Services.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1 Target has been updated for RY1 RP4.

2New metric for PY7.

4.1 ECSS Director's Office (WBS 2.2.1)

The ECSS Director’s Office has been established to provide the necessary oversight to ensure the greatest efficiency and effectiveness of the ECSS area. This oversight includes providing direction to the L3 management team, coordination of, and participation in, ECSS planning activities and reports through the area’s Project Manager, and monitoring compliance with budgets, and retargeting effort if necessary. The Director’s Office also attends and supports the preparation of project level reviews and activities. The ECSS Director’s Office will continue to manage and set direction for ECSS activities and responsibilities. They will contribute to and attend bi-weekly senior management team calls, contribute to the project level plan, schedule, and budget, contribute to XSEDE quarterly, annual, and other reports as required by the NSF and attend XSEDE quarterly and annual meetings. The Director’s Office will advise the XSEDE PI on many issues, especially those relevant to this WBS area. The office consists of two Level 2 Co-Directors (Ralph Roskies, who manages ESRT and NIP activities, and Nancy Wilkins-Diehr who manages ESCC, ESSGW, and ESTEO activities) and two project managers (Karla Gendler and Marques Bland). 

Roskies and Wilkins-Diehr carry out the post-project interviews with all project PIs who have received ECSS support, both to get their assessment of how the project went, and to hear and act on any concerns they may express. Wilkins-Diehr also organizes the monthly symposium series, one of the contributors to staff training, and runs the Campus Champions Fellows program. Roskies reviews most stories destined for inclusion in XSEDE Highlights and also convenes User Advisory Committee meetings.

The two project managers aid in the management of the day-to-day activities of ECSS which includes the management of project requests (XRAC and startups), active projects, project assignments and staffing. They continuously refine the ECSS project lifecycle, further defining processes to aid in the management of over 100 active projects. They also administer JIRA for the management and tracking of projects, both for the managers and directors of ECSS and for ECSS staff.

4.2 Extended Support for Research Teams (WBS 2.2.2) 

Extended Support for Research Teams (ESRT) accelerates scientific discovery by collaborating with researchers, engineers, and scholars in order to optimize their application codes, improve their work and data flows, and increase the effectiveness of their use of XSEDE digital infrastructure.  

ESRT projects are initiated as a result of support requests or recommendations obtained during the allocation process. Most projects focus on home-grown codes, as community codes fall under ESCC (§4.4) but are not exclusively restricted to this classification. The primary mandate of ESRT is the support of individual research teams within the context of their research goals. 

Table 4-2: Area Metrics for Extended Support for Research Teams.

 

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1New metric for PY7.

4.3 Novel & Innovative Projects (WBS 2.2.3) 

Novel and Innovative Projects (NIP) accelerates research, scholarship, and education provided by new communities that can strongly benefit from the use of XSEDE’s ecosystem of advanced digital services. Working closely with the XSEDE Outreach team, the NIP team identifies a subset of scientists, scholars and educators from new communities, i.e. from disciplines or demographics that have not yet made significant use of advanced computing infrastructure, who are now committed to projects that appear to require XSEDE services and are in a good position to use them efficiently. NIP staff then provides personal mentoring to these projects, helping them to obtain XSEDE allocations and use them successfully. 

XSEDE projects generated by, and mentored by, the personal efforts of the NIP experts should stimulate additional practitioners in their field to become interested in XSEDE. Strategies used will include building and promotion of science gateways serving communities of end-users and the enhancement of the Domain Champions program by which successful practitioners spread the word about the benefits of XSEDE to their colleagues. 

Table 4-3: Area Metrics for Novel and Innovative Projects.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

4.5 Extended Support for Science Gateways (WBS 2.2.5) 

Extended Support for Science Gateways (ESSGW) broadens science impact and accelerates scientific discovery by collaborating in the development and enhancement of science-centric gateway interfaces and by fostering a science gateway community ecosystem. 

ESSGW projects primarily begin through user requests from the XSEDE allocation process. Similar to ESRT and ESCC, ESSGW projects progress through three activities. First, the project is assigned to an ECSS expert. Second, the project is quantified with the formation of a work plan through collaboration with the research group. Third, when the project is completed, the ESSGW expert produces a final report with input from the research group. A successful project is the completion of all three phases. Each state of the progression is measured to provide an assessment of progress. Submission of work plans within 45 days of initial contact, 90% of projects with work plans completed, and 85% of completed projects with final reports within three months are additional criteria for success. 

Table 4-5: Area Metrics for Extended Support for Science Gateways.

 

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1New metric for PY7.

4.6 ECSS Communities – Extended Support for Education, Outreach, & Training (WBS 2.2.6) 

Extended Collaborative Support for Training, Education & Outreach (ESTEO) prepares the current and next generation of researchers, engineers, and scholars in the use of advanced digital technologies by providing the technical support for Training, Education and Outreach planned activities. 

Typical events include train-the-trainers events, on-site classes requested by Campus Champions, regional workshops, conferences, and summer schools (including the International HPC Summer School). Staff also create and review online documentation and training modules. This on-demand training is increasingly popular with the user community when both time and travel budgets are limited. 

Table 4-6: Area Metrics for Extended Support for Education, Outreach, & Training.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

- Data reported annually.

[Table of Contents]

5        XSEDE Cyberinfrastructure Integration (XCI, WBS 2.3) 

The mission of XSEDE Cyberinfrastructure Integration (XCI) is to facilitate interaction, sharing, and compatibility of all relevant software and related services across the national CI community—building and improving upon the foundational efforts of XSEDE.

XCI envisions a national cyberinfrastructure that is consistent, straightforward to understand, and straightforward to use by researchers and students. Service to XSEDE Service Providers is a particularly important aspect of XCI’s activities. Service Providers should find it as simple as possible to provide resources to the national research community through shared expertise and when appropriate, software and tools. We strive to make it possible for researchers and students to effortlessly use computational and data analysis resources ranging from those allocated by XSEDE to campus-based CI facilities, an individual’s workstation and commercial cloud providers, and to interact with these resources via CI software services such as science gateways and Globus Online. Through XCI, XSEDE serves an aligning function within the nation, not by rigorously defining a particular architecture, but rather by assembling a technical infrastructure that facilitates interaction and interoperability across the national CI community and which is adopted by campus, regional, and national CI providers because it makes their task of delivering services easier and the delivered services better. The suite of interoperable and compatible software tools that XSEDE will make available to the community will be based on those already in use and services will be added that address emerging needs including data and computational services. 

The XSEDE User Requirements Evaluation and Prioritization Working Group (UREP) prioritizes users’ needs for XCI. The UREP provides the primary direction and prioritization for the Requirements Analysis and Capability Delivery team (RACD). Given that as a source of prioritization, the group of NSF-funded Level 1 SPs is generally customer number one for XCI. These are the nationally-accessible cyberinfrastructure resources created and operated by Service Providers that are required to be interoperable with XSEDE services in general and to be allocated via the XRAC process. The Level 1 SPs, then, are the cyberinfrastructure service providers most invested in terms of having important requirements from XSEDE and important requirements for XSEDE (and XCI) to fulfill. Level 2 and 3 SPs represent the next two most motivated and engaged groups of cyberinfrastructure providers in the US. The SP Forum is another source for user needs and priorities; the members of the SP Forum constitute national leaders and exemplars for the US national cyberinfrastructure community as a whole. By engaging with Level 1, 2, and 3 SPs and the SP Forum in particular, we believe that we can get the most detailed statements of needs and priorities. New tools implemented under the leadership of XCI are most likely to be widely and quickly adopted by the national community of CI providers if they are first adopted by members of the SP Forum. Because the XSEDE Campus Resource Integration team (CRI) deals primarily with Level 1, 2, and 3 SPs, along with campus cyberinfrastructure administrators and support experts, the SP Forum and Campus Champions are XCI’s primary sources of direction regarding prioritization of efforts.

The Community Software Repository (CSR) is a new service and tool catalog for the national research community to facilitate connecting resources, software, and services into the broader cyberinfrastructure ecosystem. To access the CSR, please visit:   https://software.xsede.org/xcsr/xsede-use-cases.

Table 5-1: Area Metrics for XSEDE Cyberinfrastructure Integration.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1This number is for RACD only as XCRI satisfaction numbers are unavailable for this reporting period.

5.1 XCI Director's Office (WBS 2.3.1)

The XCI Director’s Office has been established to provide the necessary oversight to ensure the greatest efficiency and effectiveness of the XCI area. This oversight includes providing direction to the L3 management team, coordination of, and participation in, XCI planning activities and reports through the area’s Project Manager, and monitoring compliance with budgets, and retarget effort if necessary. The Director’s Office also attends and supports the preparation of project level reviews and activities.

The XCI Director’s Office will continue to manage and set direction for XCI activities and responsibilities. They will contribute to and attend bi-weekly senior management team calls, contribute to the project level plan, schedule, and budget, contribute to XSEDE quarterly, annual, and other reports as required by the NSF and attend XSEDE quarterly and annual meetings.

5.2 Requirements Analysis and Capability Delivery (RACD, WBS 2.3.2) 

The Requirements Analysis & Capability Delivery (RACD) team facilitates the integration, maintenance, and support of cyberinfrastructure capabilities addressing user technical requirements. The process begins by preparing Capability Delivery Plans (CDPs) that describe the technical gaps in XSEDE’s prioritized Use Cases. To fill the gaps, RACD evaluates and/or tests existing software solutions, engages with software providers and facilitates software and service integration. To ensure software and service adoption and ROI, RACD will involve users, Service Providers and operators in an integration process that uses engineering best practices and instruments components to measure usage. Once components are integrated, RACD will facilitate software maintenance and enhancements in response to evolving user needs and an evolving infrastructure environment.

Table 5-2: Area Metrics for Requirements Analysis and Capability Delivery.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1 This metric is reported in each reporting period, although the target is annual. The annual figure is the average over all reporting periods.

New metric for PY7.

5.3 XSEDE Cyberinfrastructure Resource Integration (WBS 2.3.3) 

The mission of the Cyberinfrastructure Resource Integration (CRI) team is to work with SPs, CI providers and campuses to maximize the aggregate utility of national cyberinfrastructure. CRI will coordinate interactions between SP’s and XSEDE in the SP Forum, engage with national CI providers, gather requirements for tools and training that assist users of XSEDE and other cyberinfrastructures, identify and disseminate benefits and costs of interoperating with XSEDE, and create toolkits that fit and improve common usage modalities.

CRI’s activities are reflected in the uptake of CRI-integrated toolkits, such as the XSEDE Campus Bridging Cluster toolkit and XSEDE National Integration Toolkit, but also Globus Transfer clients and other toolkits as developed. 

Table 5-3: Area Metrics for XSEDE Cyberinfrastructure Resource Integration.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1Data for this metric was previously underreported.

[Table of Contents]

6        XSEDE Operations (WBS 2.4) 

The mission of XSEDE Operations is to install, connect, maintain, secure, and evolve an integrated cyberinfrastructure that incorporates a wide range of digital capabilities to support national scientific, engineering, and scholarly research efforts. 

In addition to the Operations Director’s Office (§6.1), Operations staff is subdivided into four teams based on the work breakdown structure: Cybersecurity (§6.2), Data Transfer Services (DTS, §6.3), XSEDE Operations Center (XOC, §6.4), and Systems Operational Support (SysOps, §6.5). The Operations management team meets weekly and individual Operations groups meet approximately bi-weekly with all meeting minutes posted to the XSEDE wiki. 

Table 6-1: Area Metrics for XSEDE Operations.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

6.1 Operations Director's Office (WBS 2.4.1)

The Operations Director’s Office has been established to provide the necessary oversight to ensure the greatest efficiency and effectiveness of the Operations area. This oversight includes providing direction to the L3 management team, coordination of, and participation in, Operations planning activities and reports through the area’s Project Manager, and monitoring compliance with budgets, and retarget effort if necessary. The Director’s Office also attends and supports the preparation of project level reviews and activities.

The Operations Director’s Office will continue to manage and set direction for Operations activities and responsibilities. They will contribute to and attend bi-weekly senior management team calls, contribute to the project level plan, schedule, and budget, contribute to XSEDE IPR, annual, and other reports as required by the NSF and attend XSEDE quarterly & annual meetings. Lastly, the Director’s Office will advise the XSEDE PI on many issues, especially those relevant to this WBS area.

6.2 Cybersecurity (WBS 2.4.2) 

The Cybersecurity Security (Ops-Sec) group protects the confidentiality, integrity and availability of XSEDE resources and services. Users expect XSEDE resources to be reliable and secure, thus the security team’s goal is to minimize any interruption of services related to a security event. 

Table 6-2: Area Metrics for Cybersecurity.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

6.3 Data Transfer Services (WBS 2.4.3) 

The Data Transfer Services (DTS) group facilitates data movement and management for the community by maintaining and continuously evolving XSEDE data services and resources. 

Table 6-3: Area Metrics for Data Transfer Services.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

6.4 XSEDE Operations Center (WBS 2.4.4) 

The XSEDE Operations Center (XOC) staff serve as user advocates, providing timely and accurate assistance to the XSEDE community, while simultaneously monitoring and troubleshooting user-facing systems and services.

Table 6-4: Area Metrics XSEDE Operations Center

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

6.5 Systems Operational Support (WBS 2.4.5) 

Systems Operational Support (SysOps) provides enterprise level support and system administration for all XSEDE central services. 

Table 6-5: Area Metrics for Systems Operational Support

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

[Table of Contents]

7     Resource Allocation Service (WBS 2.5) 

The Resource Allocation Service (RAS) is building on XSEDE’s current allocation processes and evolving to meet the challenges presented by new types of resources to be allocated via XSEDE, new computing and data modalities to support increasingly diverse research needs, and large-scale demands from the user community for limited XSEDE-allocated resources. RAS is pursuing these objectives through three activities: managing the XSEDE allocations process in coordination with the XD Service Providers, enhancing and maintaining the RAS infrastructure and services, and anticipating changing community needs.

Table 7-1: Area Metrics for Resource Allocation Service

 



 

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

7.1 RAS Director's Office (WBS 2.5.1)

The RAS Director’s Office has been established to provide the necessary oversight to ensure the greatest efficiency and effectiveness of the RAS area. This oversight includes providing direction to the L3 management team, coordination of, and participation in, RAS planning activities and reports through the area’s Project Manager, and monitoring compliance with budgets, and retarget effort if necessary. The Director’s Office also attends and supports the preparation of project level reviews and activities. The RAS Director's Office also contributes to an analytics effort to support NSF, Service Provider and XSEDE2 in understanding and projecting the stewardship of, demand for, and impact of CI resources and services.

The RAS Director’s Office will continue to manage and set direction for RAS activities and responsibilities. They will contribute to and attend bi-weekly senior management team calls, contribute to the project level plan, schedule, and budget, contribute to XSEDE IPR, annual, and other reports as required by the NSF and attend XSEDE quarterly and annual meetings. Lastly, the Director’s Office will advise the XSEDE PI on many issues, especially those relevant to this WBS area.

7.2 XSEDE Allocations Process & Policies (WBS 2.5.2) 

Allocations enable the national open science community to easily gain access to XSEDE’s advanced digital resources, allowing them to achieve their research and education goals. 

Table 7-2: Area Metrics for XSEDE Allocations Process & Policies

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

New metric. Data retroactively entered for all of PY6. 

7.3 Allocations, Accounting & Account Management CI (WBS 2.5.3) 

The Allocations, Accounting & Account Management CI (A3M) group maintains and improves the interfaces, databases and data transfer mechanisms for XSEDE-wide resource allocations, accounting of resource usage, and user account management.

Table 7-3: Area Metrics for Allocations, Accounting, & Account Management CI

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1 Prior to RY1 RP4, this metric was titled "Percentage of approved feature change requests implemented." The title was changed as a more intuitive summary of the RAS and A3M team's success and improvements in estimating, planning, and completing work. The target was also lowered from 100% to 95% as a reflection of the team's potential inability to complete tasks that are assigned at the very end of the quarter but can not be completed within that reporting period. 

[Table of Contents]

8     Program Office (WBS 2.6) 

The purpose of the Program Office is to ensure critical project level functions are in place and operating effectively and efficiently. The oversight provided via the Project Office is necessary to provide consistent guidance and leadership to the L3 managers across the project. A common and consistent approach to managing projects and risks is provided by theProject Management, Reporting and Risk Management (PM&R) team, while Business Operations manages all financial functions and sub-awards. The crucial aspect of communications to all stakeholders is the focus of the External Relations team. Finally, Strategy, Planning, Policy, Evaluation & Organizational Improvement focuses attention in precisely those areas to ensure the best possible structure continues to exist within XSEDE to allow the support of all significant project activities and enable efficient and effective performance of all project responsibilities.

Table 8-1: Area Metrics for Program Office.

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1 L2 Directors are currently responding to climate study recommendations; data will be available in RY2 RP2. 

2This metric is one of two new metrics that resulted from splitting a former metric, called "Number of strategic or innovative improvements," into two. The former metric had a target of 9/yr and results were 3 for RP2 and 8 for RP3 in RY1. These two new metrics each have a target of 10/yr and will be reported annually. 

3This metric was reported incorrectly in IPR1. It has been corrected.

8.1 Project Office (WBS 2.6.1) 

The Project Office has been established to provide the necessary oversight to ensure the greatest efficiency and effectiveness of the Program Office area and to establish responsibility for assuring advisory activities of the project occur. This oversight includes providing direction to the L3 management team and coordination of and participation in Program Office planning activities and reports through the area’s Project Manager. The Project Office also attends and supports the preparation of project level reviews and activities. Importantly, the Project Office is responsible for ensuring that the XSEDE Advisory Board, the User Advisory Committee, and the SP Forum are functioning. The Project Office is responsible for coordination of project-level meetings such at the bi-weekly Senior Management Team (SMT) teleconference calls and the project quarterly meetings. Lastly, the Project Office will advise the XSEDE PI on many issues, especially those relevant to this WBS area.

8.2  External Relations (WBS 2.6.2) 

External Relations’ (ER) mission is to communicate the value and importance of XSEDE to all stakeholders (including the internal audience) through creative and strategic communications. 

Table 8-2: Area Metrics for External Relations

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

8.3 Project Management, Reporting, & Risk Management (WBS 2.6.3) 

The Project Management, Reporting & Risk Management (PM&R) team enables an effective virtual organization through application of project management principles, provides visibility to project progress, successes, and challenges, brings new ideas and management practices into the project and disseminates lessons learned in XSEDE to other virtual organizations. Communication is critical to success in this highly distributed virtual organization.

Table 8-3: Area Metrics for Project Management, Reporting, & Risk Management

 

* NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1There are no relevant report submissions in RY1 RP2. RPs are submitted in the quarter following the reporting period and thus, this variance will be reported in the subsequent quarter. 

8.4 Business Operations (WBS 2.6.4) 

The Business Operations group, working closely with staff at the University of Illinois’ Grants and Contracts Office (GCO) and National Center for Supercomputing Applications’ (NCSA) Business Office, manages budgetary issues and sub-awards, and ensures timely processing of sub-award amendments and invoices. 

Table 8-4: Area Metrics for Business Operations

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

1Sub-award institutions do not have XSEDE2 contracts in place yet, so no amendments or invoices can be charged against the grant yet. 

2Metrics were changed from a number of days to process sub-award amendments/sub-award invoices to be a percentage processed within a range of days. Targets duration ranges of < 41 and < 45 calendar days, respectively,  were also modified. This change took effect RY1RP3. 

8.5 Strategic Planning, Policy & Evaluation (WBS 2.6.5) 

XSEDE dedicates effort to project-wide strategic planning, policy development, evaluation and assessment, and organizational improvement in support of sustaining an effective and productive virtual organization. 

XSEDE has engaged an independent Evaluation Team designed to provide XSEDE with information to guide program improvement and assess the impact of XSEDE services. Evaluations are based on five primary data sources: (1) an Annual User Survey that will be part of the XSEDE annual report and program plan; (2) an Enhanced Longitudinal Study encompassing additional target groups (e.g., faculty, institutions, disciplines, etc.) and additional measures (e.g., publications, citations, research funding, promotion and tenure, etc.); (3) an Annual XSEDE Staff Climate Study; (4) XSEDE KPIs, Area Metrics, and Organizational Improvement efforts, including ensuring that procedures are in place to assess these data; and (5) Specialized Studies as contracted by Level 2 directors and the Program Office. 

Table 8-5: Area Metrics for Strategic Planning, Policy & Evaluation

 

 * NOTE: RY1 is unique and only spans September 2016-April 2017.  RY1 RP1 doesn’t exist due to the shortened RY. RY1, RP2 only includes two months (September-October 2016). 

- Data reported annually.

^ The number 3.70 was erroneously reported in RY1, RP2.

L2 Directors are currently responding to climate study recommendations; data will be available in RY1 RP3. 

This metric is one of two new metrics that resulted from splitting a former metric, called "Number of strategic or innovative improvements," into two. The former metric had a target of 9/yr and results were 3 for RP2 and 8 for RP3 in RY1. These two new metrics each have a target of 10/yr and will be reported annually. 

3 This metric was reported incorrectly in IPR1. It has been corrected. 

 

[Table of Contents]

Footnotes
Ref Notes
1 https://www.nsf.gov/about/performance/strategic_plan.jsp
2 http://www.nsf.gov/cise/aci/cif21/CIF21Vision2012current.pdf
3 http://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf12051
4 http://www.nist.gov/baldrige/

 

  • No labels