Create a Better Online Submission Review Process

Create a better online submission review process

The question all award, grant, and scholarship managers ask – how to create a better online submission review process? There’s no doubt, allocating time and resources to the analysis and fine tuning of your online submission management process (or your manual process) can improve responses, minimize handling time and improve the quality of submissions—as well as building a positive reputation for your organization.

Second only to the submission management process in value to you and your organization is the online submission review process (or manual process). Whether you are running a contest or a qualification program, it’s to your advantage to spend some quality time improving the experience of your reviewers.

A streamlined, easy-to-navigate review process provides positive benefits for your organization beyond simply making the process itself run smoothly. A well-run process means:

• Reviewers won’t quit midstream
• They will be willing to volunteer again
• Inevitably, the quality of the review and evaluation will be enhanced
• Volunteers will be willing to act as advocates and supporters of your company, spreading the word that you have a superior program, which can have positive effects beyond submission and review

Overall, best practices for the review management process are very similar to best practices for the submission management process: 1) be courteous of others’ time, 2) make it easy and 3) facilitate communication.

Reviewr has assisted with the organization and implementation of hundreds of award, grant, scholarship, fellowship and competition cycles over the years. The following core elements of the review process have emerged as primary considerations to help organizations ensure review cycles are successful and reflect well upon their organizations. It’s interesting to note that these core elements apply whether a review process takes place online or off-line. This is how to create a better online submission review process.

Core Elements of Successful Award, Grant, Scholarship, Competition and Call for Entry Submission Review Processes

Rapid Submission Processing
It’s important to administer any process with a sensitivity to the time required of every stakeholder. However, when reviewers of these submissions are volunteers, it is doubly important to be courteous of their time. In a way, your reviewers are your customers. You need to sell them on the process, so they want to continue assisting you, so they will be inspired to do a good job, and so they’ll speak well of your program to the world.

It’s obvious that time sensitivity involves providing a streamlined, relatively-headache-free process in the first place. The easier the process is to follow and the fewer potential glitches, the quicker reviewers can perform their tasks.

Another key to speeding up the review process is allowing reviewers to pre-review materials before meetings. The upfront knowledge they gain will accelerate meetings, and it also gives them time to think carefully about their decisions, which can lead to the improved quality and reputation of your program.

Although speed is important, your submission review process should build in ample time for reviewers to complete their responsibilities. This allows them to weave the tasks into their regular schedules easily, and makes it more likely they will follow through.

Easy Submission Access
Remember our obsession with counting keystrokes when we all first began using the Internet? The fewer the keystrokes, the quicker we could move through the mountains of information suddenly coming our way. We aren’t as obsessed with keystroke counting these days, but there is an element of truth to it.

The same goes for downloading. Don’t make your review committee download anything unnecessarily. It takes time to download documents, and this additional step in the process also increases the chances of malfunctions — both human and machine. It’s better to present document images within the original review screen, which allows reviewers to view documents while simultaneously scoring and writing comments.

The worst thing you can do is send reviewers to multiple software programs, such as Dropbox, Excel and email, to retrieve application or entry materials. The time it takes to download items from multiple locations can add up fast. Before you know it, reviewers are spending as much time downloading as reviewing.

For similar reasons, avoid using spreadsheets to gather or present information. It takes time to create and understand the data within a spreadsheet. Reviewers hate using them as much as you hate creating them. Instead, present important information in the form of easy-to-digest, visually appealing charts and graphs. You’d be surprised how this element alone can transform the reviewing experience from a chore to a pleasure.

Qualitative Feedback
Many award, grant, and scholarship review processes depend upon quantitative feedback from judges. This can indeed provide important objective data to support and justify final decisions. However, some programs make the mistake of limiting feedback to objective data only.

Qualitative feedback in the form of comments can be more powerful for the applicant than the score itself, because it provides perspective and reasons for the score. This allows the applicant to improve future applications, as well as potentially improving fundamental skills related to the application.

If you allow reviewers to see one another’s qualitative feedback, it can spur conversation among them and lead to deeper insights about each application. This could result in better decisions and, in the case of selection processes, could lead to more successful tenures of the candidates after they begin their work.

Surprisingly, unveiling qualitative feedback can provide visibility into the review process itself. This allows your team to continually analyze and assess the process and results to improve your program.

Simplified Scoring Rubrics
Online award, grant, and scholarship program designers with mathematical minds sometimes believe complex algorithms lead to more precise results. For these folks, algorithms sound like fun, but judges and reviewers who must navigate algorithms in the form of complex response cards probably don’t agree. It takes more time to complete complicated forms, and reviewers must think harder to complete them. The added difficulty (and additional time required) could be enough to convince a judge not to return.

More often than not, complex algorithms are not needed to identify top candidates anyway. Superior applicants can be identified just as easily with simple scorecards requiring only yesses and no’s from the judges. Make it even simpler by turning words into visuals, such as thumbs-up and thumbs-down or a number scale.

In addition, limit the number of questions or categories to a minimum. Don’t add entries simply for the sake of creating a more robust review process. Your reviewers will greatly appreciate the trimming of responses to only those absolutely needed to get the job done. In fact, most judges and reviewers would much rather spend any additional time offering qualitative review, which can be of more value to applicants AND decision-makers.

Limited Reviewer Responsibility
Where possible, limit the number of applications reviewed by any one judge or reviewer. Put yourself in their shoes. Think about the amount of time it takes for each review. How much time would you want to allocate out of your day to provide volunteer reviewing services? Use this as a guide to determine how many submissions each reviewer should be asked to complete.

One of the best ways to convince reviewers to never volunteer again is to overwhelm them with stacks of applications or submissions that seem impossible to complete. Depending on the nature of your program, it may not be necessary for every reviewer to review all applications.

Better yet, allow reviewers to determine specifically how many applications/submissions they will receive. You can survey reviewers to get a consensus and then split applications equally, or you can simply allow individual reviewers to state how many applications they would like to review. This, along with your efforts to streamline the process, may just be enough to convince reviewers to continue working with you year after year.

Committees with Committee Chairs
Just as it’s human nature for applicants to wait until the last minute for submissions, reviewers often wait until the last minute to complete reviews. This presents a problem for the person ultimately responsible for completion of the program. A program leader can find himself or herself embroiled in numerous phone calls and emails to follow up with negligent judges.

An alternative is to organize reviewers into committees with team captains who can assist with follow-up. Each reviewer captain can take responsibility within his or her committee for reassigning applications to idle judges, handling communications, resolving issues, answering questions, reviewing results and performing other important tasks to ensure completion of their team’s assigned tasks.

Teamwork
To provide an extra layer of enrichment and value in your award or application program, allow reviewers to collaborate with one another about individual entries in a common forum, then compile their comments for presentation to the applicant. In addition to providing more robust information for contestants or candidates, the sharing of information, ideas and opinions among judges and reviewers likely will lead to more thorough reviews and increase the value of your program both for applicants and your organization.

Virtual Accessibility
To reduce the stress of your team, don’t treat reviewers as if they were a jury, locking them in a room for hours or days. A reviewing marathon won’t likely to lead to the highest-quality reviews. Give reviewers time to think about entries and provide quality feedback at their convenience.

Likewise, refrain from sending heavy three-ring binders and CDs with spreadsheets for reviewers to fill out and mail back. Such a process can be a burden that ruins the reviewing experience.

Consider Online Submission Management Software

In this day and age, cost-effective, all-in-one online submission management software is available to streamline your entire reviewing process. Online programs such as Reviewr integrate all of the core elements of successful review processes, creating a seamless program that’s easier than ever for you and your team to implement. 

We will leave you with a quote from the American Society of Association Executives that perfectly sums up with investing in your digital experience is necessary. “Associations that focus on leveraging data, improving processes, and investing in new technologies, the hard work will likely pay off in more engaged and satisfied members”.