Key Performance Indicators (KPIs)
What is a KPI?
Key Performance Indicators (KPIs) are indicators of the project's performance in key areas that relate to our long-term strategic goals and sub-goals. KPIs are often aggregated metrics from different areas within the project.
Example: For the goal of "extending use of the XSEDE ecosystem new communities," a KPI might be "the number of users on projects from new communities." This KPI would consist of users from all the projects with non-traditional communities across the project:the number of users on projects from new communities = (Industry Challenge Users) + (users on startup projects from new communities engaged with NIP) + (users on XRAC projects from new communities engaged with NIP) + (users on projects from under-represented communities and MSIs)
In XSEDE we have found it useful to pair an outcome metric with a short-term impact metric so the above metric may be paired with an "average satisfaction rating" from the users counted as users of new projects. These two measurements together give a sense of both the scope and quality of the associated activities.
The key concept is not that the KPIs themselves must have a direct causal effect on eventual outcomes, but rather that the KPIs are chosen so that actions and decisions which move the metrics in the desired direction also move the organization in the direction of the desired outcomes and goals.
KPI Guidelines (What makes for a good KPI?)
Measure an outcome for which you are the responsible manager
Show progress toward a project goal
Have an appropriate target
Be accompanied by a description in “layperson” terms
Be accompanied by language that explains how it supports a project sub-goal
Be accompanied by an analysis that adds context
Is the result good or bad?
What does it mean in terms of the area (shows success, plans to improve, etc.)
- Names should be descriptive, yet concise.
- Names should not embed other information (e.g., targets).
Example: Use: "number of widgets built with a target of a 10% increase" instead of:"‘increase the number of widgets built by 10%"
- Changes should be infrequent and must be executed through a project change request.
- Targets should be program year targets stated as a percentage of the previous program year.
- Results should be reported quarterly as a cumulative percentage increase (once a non-zero baseline has been established) over the program year with cumulative actual results (data) in parenthesis beneath the percentage.
- Further analysis of area metrics (e.g., comparing to the same quarter in the previous year, etc.) should be used to provide further meaning to the results.
- Additional area metrics (e.g., comparing to the same quarter in the previous year) should be used to provide further meaning to the results.
Annual KPI Reviews
As part of the annual planning process, the SP&E group executes a project-wide review of KPIs. This consists of
- a plenary review by all area managers and directors at the quarterly meeting following a panel review
- feedback from the panel is discussed
- problems are identified
- solutions are proposed
- each area within the project then includes both proposed area metrics and KPIs as part of their annual planning presentation
- Target changes are permissible during PYx Planning without a PCR. However, all target changes MUST be discussed at the QM when L2 area PYx Plans are presented. Therefore, all target changes should be discussed internally (within the L2 area) prior to the Planning QM.