Skip to end of metadata
Go to start of metadata

Attendees:


Canonical Meeting Template (as a reminder):

1. Review of meeting agenda
2. Addressing issues

  • Description/Status of issue
  • Review of action items from previous meeting if any
  • Recommended actions and discussion of same
  • Close issue or identify next action items

3. Status updates from SMT members

  • Status update on significant activities
  • Review of prior open issues if any (and not addressed earlier in agenda)
  • Identification of new issues if any (and not addressed earlier in agenda)

Agenda/Notes

1. Review of Meeting Agenda: Any comments/additions? 

 

2. XSEDE2 Reporting (Karla Gendler

Previously:

  • Previous discussion archived: SMT Meeting - January 4, 2018
  • (1/4) Karla - end of January is the end of a quarter. February 1st begins IPR5 and due 2/15. Then will move into PY8 Planning through the Quarterly Meeting draft due to XAB 4/10 (meet 4/17). Starting May write RY2 annual report
  • (1/18) Karla - IPR5 is coming up and due 2/15 to NSF. Monday 1/22 email will go out, so work with your area PM. Risk review also underway, see email from Kandace. Planning will begin immediately after reporting. Will write mid-year review panel response document as well.
    • (1/18) John - Bob hopes to get us the document by 1/19; might be good to get that done early to prevent overlap

Discussion:

  • (2/1) Karla - IPR 5 begins today, work with your PM. 100% of risks reviewed. PY8 planning email coming soon.

Decisions:

  • note

Actions:

  • Maytal requested a Quarterly Leadership Radar identifying major project level activities, new tools/functionality released, etc. - Ron to proposal options to serve the need

 

3. XSEDE2 Mid-Year Review (John Towns)

Previously:

  • (1/8) John - Jan 11-12 in Alexandria
  • Observe review as first with Bob Chadduck
  • Assign owner of 3 discussion topic areas ACTION Laura move to wiki
    • Onboarding - Nancy
    • Cybersecurity - Adam; other operational topics - Greg
    • KPIs and Metrics review - Ron introduction and overview of process; driven by evaluation processes - Lizanne
  • One more SMT on 11/9, then Thanksgiving, then Quarterly, then holidays
  • Small group meeting with John (Laura requested from Amber)
  • Present metrics review findings at the December review and then transition during at PY8
  • Dave L - collecting metrics immediately to see if they collect what we want them to
  • (11/9) John: Bob C. has asked for more information on split-funded staff members (staff partially funded by and XSEDE and other externally funded projects). Objective is to show the benifet of sharing knowledge to and from XSEDE. This will be shared during the mid-year review in Jan. and then in more detail during the June panel review.
  • (12/20) On-site names sent to NSF for security preparation; Per Bob, we should allow additional time to get through security. There are instances of it taking one hour to get through security with a pre-scheduled appointment.
  • (12/20) wiki is updated with latest agenda from NSF and dry-run schedule
  • (12/20) we expect to receive the names of the panel members as well as the remote coordinates, sometime during the first week in Jan.; Panel names/bios will be posted on the wiki, remote coordinates will be sent to appropriate XSEDE staff members via email.
  • (12/20) Pre-review materials, due to NSF by 1/4, to include draft of presentation slides, SP Forum letter regarding onboarding topic
  • (12/21) Ron - wiki updated with all information to date; as more information becomes available it will be on the wiki. Remote coordinates will not be posted but sent to those who should tune in
  • (12/21) John - KPIs and metrics status before the review w/SP&E team and Bob. Slides should be ready for run-through on 1/4
  • (12/21) Greg - cybersecurity down to 30 minutes
  • Added in schedule and finance update; took away from security and KPIs
  • Target 20 minutes to talk and 10 minutes to answer questions
  • Nancy down to an hour
  • (1/4) Ron - reviewed slides earlier today. Will have remote connectivity for the dry run on Wednesday. NSF is setting up the coordinates for the review on Thursday. Please see additional information on the wiki page. Watch for panelists next week and a logistics email later today. Draft slides due to NSF at the end of the day today. 
  • (1/4) ACTION for Kelly, Nancy, Dave H, and Greg–Ron sent email w/budget and execution of the 3 topical areas for your feedback
    • Nancy - ECSS staff on new projects
    • Dave - a lot of work goes into onboarding
  • (1/18) Final version of responses to Panel questions due today EOD.
  • (1/25) Feedback on mid-year review
    • John - thank you everyone. Gratifying yet depressing that the team can function with out me.
    • John - via observation, there were a number of problems with the way the review was conducted and expressed this to Bob who did not object. These issues will be communicated formally to Bob via a letter that is expected to get to him by tomorrow.
    • John - anticipating panel report by 1/18; will need timeline for completion
    • Craig - glad you raised issues with Bob; glad Bob's reaction was such
    • Kelly - a lot of panelists were not brought up-to-speed on project and dominated the review
    • Sergiu - Bob is also new. It would have been nice to have an introduction
    • Kelly - delineation between SPs and XSEDE. Before next review, turn that into graphic and hand it out ACTION Ron
    • John - I thought that a few of these things reflected poorly on the project and shouldn't have because they didn't know. Bob assured me that we were in very good standing after this review
    • Dave - keep the responses lightweight in the spirit of the intended review
    • John - asked for clear and specific guidance on the level of response
    • Greg - John, you were missed. Bad vibes from Irene on incident a year ago
    • Nancy - bothered by the digital humanities comment. Ron did a great job for his first time.
    • John - my style is different and managing the panel can be tricky
    • Greg - managing the program officers and the panel
    • Kelly - the feeling in the room was different than the questions we received

Discussion:

  • (2/1) Ron - sent review document. Some need to be assigned
  • Trends transparently + other non-KPI topics–John to take question; could opt to say it is not a valuable use of our resources. It is also fair to say that the guidance is diametrically opposed to previous recommendations. Take your best shot at your questions and John will review and revise as needed
  • Kelly - they are right about the figures/charts/illustrations notion and the need for better marketing. 
  • John - often we are asked to do better on X without any suggestion on how. ROI on responding to every element of the report is not that great
  • New team = user vantage point
  • It is ok to say that the topic is outside the scope of this review and that we look forward to discussion during the full review
  • Craig - working on article for Science Node
  • Going forward, assume every review is comprehensive
  • Nancy - "within NSF priorities" wonder if we need clarification
  • Balance between talking about how resources are being used and defining who we are going to help
  • How do we implement judging what is within NSF priorities
  • There could be an NSF-favoring part of the algorithm in selection
  • Explicitly state how everything ties back to NSF, not to change what we do but to tell the story toward the bigger NSF view. Be more judicious about what we emphasize
  • Digital Humanities is outside the scope of NSF, we were given guidance to support any domain
  • Reframe our work with connections to NSF
  • Compare the schedule for the document against the IPR and Planning schedules
    • Karla - have the panel document due after the IPR
    • John - I'm willing to run the schedule as is as long as everyone else is ok
      • IPR due Thurs the 15th
      • Panel response due Friday the 16th
      • Planning begins the 12th
      • John on Jury Duty that week, so it may be TBD

Decisions:

Actions:

  • Delineation between SPs and XSEDE. Before next review, turn that into graphic and hand it out ACTION Ron


4. XSEDE2 Knowledge Base (Craig Stewart)

Previously:

  • (10/26) Sergiu - What is the status of the XSEDE Knowledge base, https://portal.xsede.org/knowledge-base ? Is it still being maintained, ensuring that all information therein is up to date and does not contradict the content of information presented by other forms of XSEDE documentation?
  • (10/26) Maytal - The knowledge base is managed/run by IU. It was part of XSEDE2 that no longer got funded but it was my understanding that IU wanted to continue to maintain the KB even without funding. Our contacts for IU regarding KB have traditionally been Paul Brown padbrown@iu.edu and Chuck Aikman caikman@iu.edu so they are the right people to talk to regarding updating of KB. We just pull the information from the KB we don’t handle the detailed articles. 
  • (10/26) Laura - Ron manages Project Documents which is a wiki page linked from the About tab on the website. Several project managers are able to load documents into IDEALS as well.
  • (10/26) Sergiu - ticket system and front-line interaction, users who read knowledge base articles conflicting. Missing references to XStream. Changes in XSEDE documentation and how that gets propogated to knowledgebase articles.
  • (10/26) John - re-affirmation that Indiana would like to do this. Don't recall this was a problem previously? Is there a process that is now broken? How was it done in the past?
  • (10/26) Craig - will verify that with IU team. None of that team is seeking updates. Need to change the pipeline. They used to get information by solicitation. ACTION: Craig will verify charity work on the condition that they receive information ready for KB. Create a process for contribution to KB
  • (10/26) John - different than the way we did it before, so need to put process into place. Could be part of SP onboarding process as well. Have to isolate where information has been coming from: SP, XOC monitoring for content, others.
  • (10/26) Sergiu - delete articles that are no longer pertinent or out-of-date
  • (11/9) Topic not discussed on call.
  • (12/21) Sergiu - question is whether or not the team at IU wish to continue service and if they are what is the mechanism. Reminded Craig of his action: Craig will verify charity work on the condition that they receive information ready for KB. Create a process for contribution to KB. Need to understand the scope of it to keep it managemeble; define who then should be involved
  • (1/4) will await responses from either Craig or Sergiu
  • (1/18) Sergiu - should remove obsolete pages; determine if they continue and establish a pipeline through XSEDE; ACTION Ron follow up with Craig
  • (1/31) - Sergiu - For CEE, Susan Lindsey commented:

"I don’t have edit privileges to the KB either, and can’t correct or update any information.  Sergiu and Craig, can you let me know what is being planned for the KB?  As of now, we have stale information being presented to the user with no way to correct it.  We are starting to get more and more tickets asking about discrepancies between information reported on the website/portal contradicting KB pages.

Craig responded with a proposal I believe he wishes to discuss with the SMT (John was cc'ed):

 There are as we look forward two options:

      • I could get someone to do at least a fair job, maybe a good job, maintaining KB content (keeping content up to date, adding new content) with 0.25 FTE. So figure $37,500-ish for 0.25 of a person – salary, benefits, F&A
      • The other option is to eliminate use of the KB. (People outside IU don’t get edit rights, and for no other entity do we even offer usage of the system for free as XSEDE is now receiving)

 I would be happy to – and could quickly – write a PCR to propose refunding this activity. Personally I think the question of the value of the KB – which people were ambivalent about – has been settled by the discussion of the content being not up to date. If people weren’t using it, no one would notice.

 Should I proceed in writing up a PCR?"

 

Discussion:

  • (2/1) Craig - get someone who is a good writer .25 to .50 to restore the KB as an excellent resource, reporting to me to work together on. Or could turn it off. 
  • J Ray - could be a topic on UREP for an environmental impact statement of support
  • Dave H - where would the money come from? PCR would have to take money from someplace else. I would be disinclined to fund because there is a no-cost solution
  • Nancy - User Engagement? PIF? Hit statistics compared to regular documentation; could content be duplicated?
  • John - it comes back to the SMT one way or the other. What would the consequences be to shut it off?

Decisions:


Actions:

  • For the KB there are two rational options going forward:

    • Option 1: Pay for a person to update information, seek information proactively, and edit/ monitor content at a total cost (direct and indirect expenses) estimated at $37,500 per year, with person reporting directly to Stewart at IU
    • Option 2: Eliminate use of the KB
    • Hopefully the unhappiness with lack of updates indicates the value of the KB; Stewart can prepare a PCR by the end of the day Monday next week, given a statement of interest of receiving a request



5. New XSEDE KPI Set (Lizanne DeStefanoRon Payne

Previously:

  • Last SMT review (Dec. 2017) resulted in follow-up items that needed to be addressed prior to approving the new KPI set
  • (1/18) All open action items from Quarterly meeting review have been addressed and closed.
  • (1/18) Discussion about approving new KPI set
  • (1/25) Karla - please send KPI set for review

Discussion:

  • (2/1) Ron - have gotten some feedback on final KPI set
  • Lizanne - sustained user definition from Dave, chose the one that was most consistent. 
  • Dave - sustained user definition thin
  • Sergiu - drop ECSS from this. Usage vs. users on portal. We are measuring who is using resources not portal
  • John - user of XSEDE services often don't use a compute resource
  • Karla - CEE is based on a portal account, ECSS is based on allocation. User on each case is defined differently. You can have a portal ID and not have a project or allocation. ECSS or NIP is looking at allocation usage. Number of new users and sustained users taken a step further. Sustained user = at least 10% usage on an allocation on a machine
  • Lizanne - all sustained users is a project-wide metric
  • John - suggestion to use different terminology to avoid confusion
  • Lizanne - use the 10% metric as an L2 metric
  • Sergiu - if the project-wide metric is for portal only, then you are set
  • Phil - wording of non-traditional disciplines. Running jobs via an allocation vs. portal usage
  • Karla - CEE and ECSS are separate, the issue is...line 3 and line 12–when you go into the description and definition, NIP on line 12–the portal isn't how NIP defines a user
  • Lizanne - in some areas, allocation is a good indicator of usage. Some users of XSEDE do not have an allocation. Having 2 definitions was confusion. All of those engage compared to those who come back
  • Karla - 2 NIP ones using the same definition of a user that is different
  • Lizanne - then perhaps we need to use different terminology as John suggests
  • Sergiu - how many portal users were from certain domains
  • John - consistent with the broader definition of user. ECSS has defined user differently. If we want to use a broader definition, they may have an alternate metric called, Allocation Users, for example. Not segmenting user community. If the primary owner is CEE, it does induce an L2 KPI for ECSS or a pair of them
  • Karla - Implemented in RY3 in May
  • Nancy - remove the non-traditional
  • Dave - the definition of sustained is to create an account and do something in another reporting period...very low bar. Propose to having activities after creation
  • Lizanne - CEE and training want a definition that captures many of their users. Could take out the registration piece, need to take this back to CEE. Allocations vs. usage is very different
  • John - suggest project-level KPI go with it as stated, but take "sustained" under review at a later time as CEE as owner rather than ECSS. Then add allocation user separately for ECSS. Changing the target is a more complicated discussion
  • Lizanne - next step is retrospective analysis and trend data. Could set 2 different bars and watch the trend on both to see what it looks like
  • Dave - we should set as high a bar as reasonable outcomes
  • Lizanne - we do need to pull the plug on this soon, need to set a drop-dead date for the approval of the KPIs
  • Nancy - those changes sound good. Same list of non-traditional users? As long as we coordinate on that.
  • Sergiu - Dave has the list
  • Dave - social media hits as a KPI as a concern especially with concern to the strategic response from the Panel response
  • Lizanne - have had extensive conversation with ER
  • Emre - clarification for non-XSEDE folks in some places
  • John - hearing no objections, you have a green light Lizanne

Decisions:

  • note

Actions:



6. UREP Prioritization Results for January 2018 (David Lifka)

Previously:

Discussion:

  • note

Decisions:

  • note

Actions:



Status updates

Follow this basic outline:

  • Status update on significant activities
  • Review of prior open issues if any (and not addressed earlier in agenda)
  • Identification of new issues if any (and not addressed earlier in agenda)
  • Identification of recent improvements implemented

Updates from SMT members:

  • PI (John/Ron) - 
  • CEE (Kelly) - 
  • ECSS (Nancy/Sergiu) - 
  • XCI (Dave L/Craig) - 
  • Ops (Greg/Victor) - 
  • RAS (Dave H/Ken) - 
  • Program Office (Ron/Laura) - 
    • Hotel contract for March Quarterly is expected to be complete by early next week. An email update with the hotel information will be sent as soon as the contract is secured.
    • NSF authorization to spend 100% of PY7 funds is being process by NSF. BusOps is requesting updated supporting documents for the 5-6 with funding changed in the last 5 months. The remainder of the PY7-round 2 amendments are being queued in the UIUC system awaiting the NSF authorization to spend.
    • ER is asking for any ideas on XSEDE stories that should be considered for the upcoming June NSF Review. Send ideas to ER.
    • Reminder: Record improvements: https://docs.google.com/spreadsheets/d/19YRAPeAForh5I9Xvhj8nKjROe3-JN7a1caLYbNamPaw
    • Reminder: Enter Publications in portal
  • SP Forum (J Ray) - 
  • UAC (Emre) - UREP voting
  • PM&R (Karla/Scott) - 
  • NSF (Bob) - 

Next Meeting:  February 15, 2018

  • No labels