Reviewing Applicants

This guide walks through the process of reviewing Outreachy final applications and selecting an intern after the contribution period has closed.

How to review applications

Tip

You can make a copy of this Google Sheet template to track the review process more easily

Triaging process

If the projects in a given round have received a lot of contributions, it may be required to triage them to generate a shortlist of applicants to review more in-depth. A decision to triage may be made because, for example, a few applicants are flooding the project(s) with a high number of low quality contributions. The decision to begin triaging can also be made before the official end of the contribution period and deadline for submitting final applications.

Review process

Each applicant should be assigned two reviewers (mentors) who will rate the contributions and final application content based on Outreachy’s system (“1 - Struggling” to “5 - Amazing”). Each mentor most likely feels most comfortable reviewing the applications for the projects they are mentoring for. We should be mindful of that but also try to distribute the reviewing work as evenly as possible across the mentors, especially if one project receives a lot more applications than another. A good starting point is to assign each mentor to review applications to their own projects as Reviewer #1, then assign them to an application for a different project as Reviewer #2, then tweak the numbers from there. Each mentor has the right to ask to be reassigned from an application they do not feel able to review.

Mentors should try to keep their review notes private from each other so as not to bias one another before the final selection meeting when applicants are discussed.

In addition to the rating according to Outreachy’s system referenced above, each mentor may also want to rank the candidates. For example, rank their top candidate as number 1, their next candidate as 2, and so on. This is completely optional but may help when discussing and comparing ratings with the other mentors.

Final selection meeting

Once the mentors have reviewed and rated all their assigned applications, a meeting should be held to discuss the applicants and decide ultimately which applicants will be invited to intern on each project.

The outcome of this meeting should be each mentor selecting an intern for their project by clicking the “Mentor this intern” button on the Outreachy platform. The ratings assigned during the review should also be entered into the platform as well, even for those applicants that were not selected, as this informs Outreachy regarding their demographics.

Note

If there are large disparaties in rating for an applicant from different mentors try to resolve this disparity before entering the rating on the platform.

Guidelines for reviewing applications

We should adhere very closely to Outreachy’s guidelines for evaluating applications.