Peter: We used to do predominantly in person events: advising in person, outreach and information sessions in person, and all of our undergraduate research presentations and creative arts festival in person. Since we were forced to move all of that online in spring 2020, we've realized that some of those types of events thrive more in an online environment than an in-person environment. So it's caused us to rethink what is better situated in an online versus an in-person environment.

Megan: In general, we've divided our programming such that when it's largely content-based and more on providing information, it's a digital event. That has allowed us greater flexibility to offer programming in off-hours beyond the traditional 9 to 5, which often better meet student needs. Conversely, we typically host in-person events when the workshop is more facilitated, discussion-based, or has an actionable output students create during the workshop itself.

Peter: We wanted to still have our Undergraduate Research Exposition in 2020 and provide some semblance of normalcy for students during an otherwise chaotic time. Obviously, we knew we couldn't have it in person at all, and so Megan found the team at ForagerOne through Council on Undergraduate Research meetups and realized right away it could meet our needs.

Megan: Because Northwestern is on the quarter system, our office was dealing with a slightly delayed timeframe compared to many of the other undergraduate research offices who were looking to host undergraduate research events in April and early May, whereas our event was not until the very end of May. I was able to learn from other folks who were adapting at a rapid speed, and I was impressed by how the team at ForagerOne identified a need and functionally built out this platform very quickly. While obviously the target audience for Symposium has expanded beyond undergraduate research since then, the initial premise that it was designed specifically towards undergraduate research needs was very appealing to me. I also liked how intuitive and easy to use the platform was, regardless of whether you were an administrator, student, or audience member.

Functionally, we used Symposium a second time last year and chose to keep the event entirely digital again. Pre-COVID, our in-person Expo and Creative Arts Festival had a budget of about $18,000. Last year, as an entirely digital event, the event cost us only $7,000. While this did offer us significant savings, we do still realize the value of in-person events, so this year we are offering a “hybrid” option that is functionally two part: a digital Expo (featuring both poster presentations and oral presentations), an in-person Creative Arts Festival, and an in-person Research Expo Winner’s Circle featuring students who won awards during the digital Expo. The Winner’s Circle will allow us to elevate some of our undergraduate research to a more prestigious level, as this event will include attendance from senior university leadership. We are projected to still save $3,000 from our pre-COVID format, while functionally offering more programming than we did prior to the pandemic.

Peter: When our Research Expo and Creative Arts Festival was strictly in person, it was always in the middle of the week at the end of the academic year. Generating attendance, both within and beyond the university community was hard. It often conflicted with class schedules, and it was virtually impossible for family and friends outside Chicagoland to attend. Symposium changed all of that; suddenly not only could more folks attend, but they could attend at their own convenience during a 36 hour window, and they could engage in the presentations through comments. Friends and family were able to attend, students were able to support their friends, faculty and lab colleagues showed up, all because of the virtual environment. To give you a sense of just how much audience attendance shifted, last year we had nearly 7,000 visitors in digital attendance, with over 19,000 unique presentation views of the 150 submissions (indicating the audience viewed more than one!). Previously, our in-person event at the student center would draw in perhaps 300 audience members, if we were lucky.

Megan: The digital format also allowed us to better engage our Northwestern campus in Qatar (NUQ). While Qatar students have always been welcome to submit and present at the Expo and Creative Arts Expo, usually only a few students attended because the university only funded a select number of international travel costs. Consequently, it was typically limited to senior students who not only came to Evanston to present, but who also had the opportunity to participate in graduation events on the main campus. While it was still exciting for these students to come, they largely did not have the support of friends or mentors to stop by their presentation since those folks were all still in Qatar. Now, more (and often younger) Qatar students participate, and they have access to share their presentations in the same way students on the main campus do. Last year, 25% of Creative Arts Festival performances were from NUQ students (only one of whom was a senior); whereas previously it was only about 10%.

Peter: Lastly, we also partnered with University Admissions to promote the Expo and Creative Arts Festival to both admitted and prospective students. We know that interest in undergraduate research is a major factor for many applicants, so this is a way to see the breadth and depth of what students can accomplish through research as undergraduates. While a large number of our presentations are from STEM or social sciences, 10% of presentations were from performing arts and 15% were from humanities. Since our event focuses on presenting to a non-expert audience, the presentations are very accessible for high school students, which also helps to make undergraduate research feel more achievable. Over 20 presentations had more than 150 views, and over 950 comments - questions, thoughts, and words of support - were left across all presentations.

Peter: As I mentioned above, connecting with admissions to support and promote opportunities for undergraduate research is a big priority of the University. It fits nicely into the broader research agenda not only for the opportunities it affords students, but also for the way it supports faculty work. The Office of Undergraduate Research is regularly invited to go and speak at Admitted Student Days, but we’ve never been able to feature student work so prominently because our Expo never aligned with when Admitted Student Days were scheduled. The flexible nature of a digital Expo and Creative Arts festival took away that barrier; our Admissions team was extremely eager and willing to share the link to their listservs because the event didn’t require anyone to show up on campus on a random Wednesday to see it.

Megan: As an internal strategic priority, our office also serves as a central resource for undergraduate research; while we are not the only place on campus that provides opportunities and funding, we are a critical link across programs, and we are housed within the Provost’s office because we support student researchers across all undergraduate schools. However, research takes place not only on our Evanston campus, but with support from faculty on our Feinberg campus, and, as I mentioned before, our Northwestern campus in Qatar. As we approach the Expo and Creative Arts Festival, we send out an invitation to attend broadly to faculty across the campus community, to senior leadership, to our advisory council, to anyone who has served on our faculty review committees or as a student sponsor. This invitation is much more broadly accepted now, especially since the Expo is no longer limited to a specific timeslot in a two-hour poster session here or a one-hour oral panel there. Since the Expo has open access for 36 hours, this now means that if a faculty member is a practicing clinician at the medical school and is scheduled to be on patient service, they can still login, see the presentation, and comment on it if they so choose. It really adds value to the students as they receive helpful critical feedback from faculty experts, or supportive comments to simply know that their presentation was received and appreciated.

Peter: I think our answer connects to how we determine if something should be hybrid versus in-person. First, what are the goals of the program or workshop? If it's strictly content we're trying to disseminate, then that works well in a virtual environment because it makes it very easy for students to attend. However, if we're trying to build community or honor specific students or events, then we want to do that in person.

Megan: The reality is hybrid events aren't going away, so the question is really how can we do these best. I personally don't think it typically works well when you try to do a hybrid event with both a live and digital audience for the same event. So, for example, you bring in a keynote speaker to an in-person audience and you happen to stream it as well. Yes, it’s technically a hybrid event, and yes, you are able to provide the content to a broader audience, but I have never attended an event where I felt both kinds of audiences were equally involved or addressed. In that scenario, I think it's just as valuable to provide the recording to anyone afterwards, rather than try to finagle technical challenges across both formats.

In my opinion, I think it's better to embrace the best of both kinds of programming, and bring those together as a hybrid set of events. That's why we've chosen to still keep the majority of our Expo presentations digital in an asynchronous format, and we strongly encourage engagement via commenting. To complement this and bring in the live element, we pair this live streamed Q&A (so, still digital, but synchronous), AND we’ve added capstone event a week later to elevate the prestige of the awarded presentations as a live, synchronous Winner’s Circle to senior university leadership. Each event has a specific audience in mind, while trying to navigate the advantages of each particular format.

Peter: Overall, I think this really depends on how you define a hybrid event. Do you define hybrid as a combination of recorded events and live streamed events? Are you also trying to balance synchronous and asynchronous events? Or is this exclusively defined based on whether something is available digitally versus physically in-person? Last year, we considered our event to be hybrid as we did several live streamed events including a keynote address and our Oral Panel Q&As, but functionally none of the events were physically in-person. This year we’ve expanded our definition of hybrid to not only include recorded/live-streamed, but also digital/in-person.

Megan: Northwestern University’s Expo has always focused on communicating with a non-expert audience. It's not as much about the content itself, so much as it is about how that content is communicated. To us, this is really important, because students can't always control how much they accomplish through research, whether it's over a summer or during an academic year. Regardless, there is still value in figuring out how to share a narrative about what the student did do or learn, even if their experience was not necessarily generative of new data.

As part of this, we recruit faculty from across fields of study (typically somewhere between 75 to 100 faculty members), with a goal of each student presentation being evaluated by at least 3, if not 4, faculty judges. This allows students to get feedback not only from experts, but also from people beyond their field. It challenges them to think about how they develop their narratives, and how they engage audience members with different backgrounds.

Pre-COVID when the Expo was entirely in-person, this meant there was a limited time frame for judging, and I needed faculty to take time out of their day to come to a physical location to review the presentation. Recruiting was a very laborious process because I needed to email a much broader array of faculty in order to find folks who were not only willing but also available. Now, my targeted recruitment of faculty is more effective and efficient (usually a single mailing), because it’s easier to commit to reviewing a few presentations within a 48 hour window at the convenience of their own schedule. I no longer have to manually track not only who has agreed to participate, but also their availability and schedule them accordingly, as faculty just register through the Symposium system to indicate their intended participation.

Then, when I need to assign judges to student presentations, my workflow is INFINITELY easier through Symposium. Because I am very intentional in making sure students have BOTH expert and non-expert judges, I have never done random judging assignments. I carefully craft assignments to also make sure that judges feel like they are getting something out of it too - I want them to be able to both provide area expertise as well as learn something new. Before this feature was built out in the Symposium platform, it probably took me 3 full work days to complete judging assignments and cross check my work to make sure judging was evenly split across judges and every student was covered. Now that this is a feature in Symposium, it is incredibly easy to turn on filters by areas of expertise, and it’s very clear how many presentations have been assigned to a particular judge. Judging assignments now takes me a day at most - and I’m STILL very particular about how I assign.

Functionally, building out our judging rubric into the Symposium judging evaluation platform has ultimately yielded better feedback for students, too. Faculty type up more comments, and do so in a more thoughtful way than when this was collected manually on paper tacked to clipboards, hurriedly scribbled during the two hour judging window. I no longer have a day and a half long process of manually sorting the paper comments out by student presenter, scanning them in, and then emailing them out to each student individually. With Symposium, it takes me about 3 hours. I view responses in a single spreadsheet, and I can quickly add in equations to calculate the average scores to determine student winners in each category. When it comes time to release feedback to students, I simply finalize the master template email and then Symposium mail merges the comments and feedback to each student for me. Even if our office ever receives pushback from the University to move back to a larger in-person Expo format, I’ll never go back to the old format for judging.

Peter: #Facts.

Peter: Re-align your objectives for an event with the different requirements and audiences. As Megan said earlier, the pedagogical purpose of our Research Expo is to teach students how to communicate effectively to people who don't have a background on the same topic. As educators, the approach we use to teach communication in an in-person setting is not sufficient for a digital setting. It is a different set of skills to teach students how to be effective communicators on a screen, and the students have different tools at their disposal to focus the audience’s attention on a particular aspect of their slide or poster. Functionally, I will be doing one set of oral communication workshops with presenters who submit to the larger, digital format Expo, and I will run a second set of oral communication workshops for awardees selected to present in-person at the Winner’s Circle Event a week later. They are two types of communication skills, and both are valuable. You have to differentiate how they work in order to be effective in both of them.

Megan: I think the other thing is not overdoing it with any one particular kind of programming format. Virtual has its benefits; hybrid has its benefits; in-person has its benefits, and they all also have their drawbacks. Functionally to decide, you need to do a benefit analysis to know how to provide the greatest benefit for the greatest good. For example, I know that if we forgo an in-person poster session, this means that students won't get the same kinds of authentic conversations about one particular aspect of their project, and they won't have to navigate multiple people coming up to their poster at the same time while they’re in the middle of the presentation: yes, it's admittedly a loss. But, when I think about the benefits of a digital session, students are getting better, more authentic feedback from all judges equally, rather than decent feedback from the one who stopped by first but a vague “good job” from the one who walked past it at the end of the session. Overall, it’s probably a more equitable approach to deciding presentation winners. When I also think about this Research Expo in conversation with the other opportunities a student may have to present their work, this may functionally be the only opportunity for students to share their work with family and friends in an accessible manner, which may also help students may also feel like it's a better use of their time. With our new in-person Winner’s Circle event, we’re also able to elevate the prestige of student winners, while simultaneously providing senior leadership with a curated sampling of the best undergraduate research presentations.