Friday, 10/30 – 11am-3pm ET | 10am-2pm CT |  9am-1pm MT | 8am-12pm PT

Executive Summary of Meeting


PRESENT (tick)/(error)
XAB Members
Lisa Arafune(tick)
Leah Bendavid(tick)
Ken Bloom(tick)
Elizabeth Cherry(error)
Peter Couvares(tick)
Rudi Eigenmann(error)
Brook Milligan(tick)
Russel Poldrack(tick)
Jennifer Schopf(tick)
Ani Ural(tick)
Service Provider Forum
David Hancock(tick)
User Advisory Committee
XSEDE Leadership
JP Navarro(tick)
Richard Knepper(tick)



Access recording of this Zoom meeting here


Time (EDT)DurationItem w/ Notes (presentation materials linked)Lead
11:00 AM EDT

10 mins

Welcome (especially to our three new members: Brook Milligan, Russ Poldrack, Jennifer Schopf!)

Lisa Arafune
11:10 AM EDT20 mins
Project Level Topics
  • Things going well for the project.
  • 1-year extension was approved (project will run through August 2022) with no budget cuts.
  • Will have project's last NSF review in June 2021 (will talk more about this at April meeting)
  • Relations with NSF have been good without a lot of issues
  • XSEDE delivers preliminary draft transition plan that outlines what we will transition to any subsequent awardee(s). This was done last year. Program Officer has asked us to publicize this transition plan on our website. Doc includes most important things XSEDE believes would need to continue post-award. Publicizing this allows any potential competitors to "see our cards." Therefore it is publicly available but hasn't been publicized. What is the right balance? Is NSF asking too much? While we need to be able to transition things to subsequent awardees, we compromise competitiveness if we publicize the plan. Plan to ask Bob Chadduck if this is normal and if we're being asked to further than other NSF awardees are asked to go.
    • Peter: How much of XSEDE's secret sauce is strategy/vision vs teams/people/demonstrated ability to execute. If a large amount is strategy then this request could be risky. If more about people/ability to execute then not as worrisome.
      • JT: Strength is not what we produce but ability to produce it.
    • Ani: Could important pieces that would damage competitiveness be removed?
      • JT: Have expressed concerns about this to Bob. People could find it if they look for it. Have to separate what we're expected to do as part of this grant vs. what we might do as part of subsequent grant. Some could be removed, but tool development has to be open source since funded by NSF. Removing from doc could reduce exposure of planning. Don't want to look like we're withholding things that should be public.
    • Russ: If doc is findable, then anyone who is a competitor will likely find it. Spin as a strength: we are the best people to do this. We aren't worried about telling others what we're doing because no one else could do it better.
    • Jennifer: Transition plans for other projects have been between projects and program officer. Suggest asking who NSF thinks the audience is.
      • JT: can only speculate that it is competitors
      • Jennifer: every program officer is different. If audience is users that is different than competitors.
    • David: Plan speaks to the nature of what the next awardee will have to do–not so much users.
    • JT: By doing this, are we indicating that we expect a transition?
      • Jennifer: NSF asks for this at the end of all large projects whether or not you're expected to continue. Making public is unusual though. Publicize info about transitioning users possibly.
      • JT: The doc currently includes a great deal more, including all priorities we believe exist for the community.
    • Jonathon: Use doc to say what is as a hand-off to the proposal when you get to that point. Foundation for what you say in the proposal
  • Ken: Who are you responsible to? To services or the team currently offering them?
    • JT: 2 stakeholders: NSF as funder; Community. Intent is to avoid disruptive transition we had from TeraGrid to XSEDE.
  • Ruth: The doc is already out there even if hard to find.
  • Brook: What is timing of this?
    • JT: Draft transition was done about a year ago. Now being asked to provide an update to that since we've been extended. In final year will provide final transition plan. Project has been doing a lot of documentation to ensure we're ready to transition to a new team if needed. Enterprise services will be able to be handed over easily. Doc includes list of services, list of docs about policy/procedures. Will do more detailed plan when we know who the subsequent awardee is. Still no solicitation for follow-up, but hope that is coming soon. Don't think they will extend the project another year. Not a lot of tech detail, but does include what services are and what they do.
    • Brook: difference between higher level plan and the actual transition that will come later. Seems inappropriate for them to ask for that from a project like this. Provide to NSF but not as an open thing.
John Towns
11:30 AM EDT10 mins

XSEDE Interconnection Diagrams

  • Demonstrate how various elements of the program fit together. Very hard to see from outside the project so want to convey that they don't stand apart.
  • Initial diagrams were done by L2s independently.
  • Received a recommendation from NSF panel to better articulate the connection between scientific impact, workforce development, and contributions made by the XSEDE 2.0 project
    • One way to do this would be to diagram at the project level
    • Will begin working on this into the winter by gathering info from the L2 areas and then develop that into one concise diagram
    • Could have value relative to transition plan as well.
  • Jennifer: This diagram could be useful but consider keeping it as an internal doc
  • Ken: Many moving parts in XSEDE, so explaining that as clearly as possible is valuable
  • JT: Will have diagrams to share at a later meeting.
  • Ruth: Would be useful to new SPs
  • Brook: Diagrams would have helped augment presentation for new people
  • David: Help people outside project to understand how linked all parts of XSEDE are.
    • Tim: Conveying this to panel and public is something we hope to achieve
  • JT: Will proceed with this and come back to you all with progress on the diagrams at a future meeting.
Tim Boerner
11:40 AM EDT20 mins

Evaluation Update

Staff Climate Study

User Satisfaction Survey

Evaluation Slides

  • User Survey:
    • Russ: it would be nice to see distributions rather than just the mean for these plots
    • Lizanne: There is a giant report that includes that info.
    • Couple uses for this survey including trying to find the trouble spots. Like to see negative comments to find the issues.
    • Lizanne: Try to highlight concerns. There weren't necessarily qualitative comments for those, but the full report includes this.
    • Ken: Are gains statistically significant?
      • Yes because of sample size. Look more at clinical significance, and clinically it looks flat.
  • Climate Study
    • Brook: These are % of respondents, right? What about % of people in these categories?

      • Lizanne: that is exactly what we do. Look at composition of who responded and compare. If any discrepancies, we weight.

    • Ken: Statistically significant gains.
  • Ken: How are we evaluating scientific impact of XSEDE?
    • Lizanne: Eval team doesn't do this. Contribute by providing data. Currently working with IRIS (U of MI) to look at economic impact, publication impact, etc.
    • Phil: John has presented info from Univ of Buffalo/XD MoD on publications from XSEDE, comparing groups that have utilized XSEDE vs. others in their field in terms of publications coming out of them. There does appear to be a difference.
  • Brook: outcome of longitudinal studies could benefit new proposal, but timing would be too late to accomplish.
    • Lizanne: Some would be useful to new proposal (front loading that). Other outcomes take a long time to mature, so waiting til out years allows us to see what real results are.
Lizanne DeStefano
12:00 PM EDT20 mins

Program Office Topics

  • Program office doesn't directly interact with end users, but in general it interconnects in some way with all areas of the project

Ron Payne
12:20 PM EDT30 mins

RAS Topics

Dave Hart
12:50 PM EDT10 minsBreak 
1:00 PM EDT30 mins

XCI Topics

Brook: How are new use cases identified for evaluation in the registry?

  • JP: Any and every way we can! Created form in Research Software Portal where people can submit use cases. Also meet regularly with different communities (campus champions, SPs, etc.) to share thoughts on what they think we should be working on.
  • Victor: SP Coordination with science gateways; Open Storage Network–info discovery on documents.
Rich Knepper, JP Navarro, Shava Smallen
1:30 PM EDT30 mins

ECSS Topics

Phil Blood & Bob Sinkovits

2:00 PM EDT30 minsKelly Gaither
2:30 PM EDT30 minsVictor Hazlewood
3:00 PM EDT 

Final questions/comments

  • Feedback on how we're presenting what we do?
    • Russ: Felt more like practice run for presentation for NSF than soliciting advice from XAB. This format doesn't take advice of XAB.
      • JT: need to be more deliberate about making time for specific questions for board. Will work to improve on that in the future. April meeting is an early practice for June review.
    • Ken: What did project leaders gain from the leaders today?
      • Bob S: Valuable part of this meeting was the first session responding to John's questions.
      • Kelly: We do a good job of what we do. Get frustrated not being able to pay for growth I would like to see us have. Not sure how to crack that nut. What could we do if we had more, how do I communicate that message & communicate full value of what we do? Hope that you could think about it and give feedback over a course of time. Frustrated that in the CI call, all programs we tried to build up were excluded.
        • Bob S: A lot to catch up on for new folks coming on to XAB. Maybe could put something together like package of materials to cover a lot of what we went through today and then use more of the call to ask questions & solicit feedback.
        • Jennifer: All KPI stuff is interesting & important, but would have been better to have that all in a package to review ahead of time. Had we known Kelly's question before the presentation we could have listened with that lens on and provided advice.
        • JT: Like the idea of providing questions in advance for folks to respond to. Always want to improve.
      • Rich: Curious about how we can extend/develop, potential for creating really healthy national ecosystem that relies on many institutions & areas of expertise.
    • Brook: This was similar to video that was shared earlier. Maybe highlighting changes instead of full picture.
    • Jennifer: XAB isn't reviewers, so show problems you're having and get input/feedback. What is keeping you up at night.
    • Peter: Like to bring big issues/ problems to board. Discussions can move problems along. What keeps you up at night.
      • Kelly: terrified that all our hard work will be dropped on the floor
      • JT: most of us are wondering about what the solicitation will look like. Can't use the XAB to think about what our proposal might look like.
    • Jonathon: XSEDE is no stranger to remote presentations; but I think, particularly with Zoom, information delivery should be provided asynchronously, and this time should be saved for questions and discussion. I suggest even the set of questions and discussion topics should be distributed ahead of time.
    • Jennifer; If I was a reviewer looking at this, I would say kudos this is amazing.

Next meeting: Tuesday, December 15; 2pm ET (1CT/12MT/11PT)

Adjourn (Ken, Emre move to adjoun)