As your approach to fraud management expands to a trust and safety mindset, you may find yourself hiring team members to investigate trends, communicate with customers, and ultimately review orders that require human intelligence to make decisions. What qualities make someone good at manual review, and how can you ensure your team’s performance stays consistent?
The Qualities of a Good Reviewer
A few key skills help the best reviewers quickly and accurately separate trusted transactions from suspicious:
Inquisitiveness: The best reviewers tend to be people who want to understand why things work the way they do, and to question things when they don’t make logical sense. This inclination leads to discerning underlying patterns that can tell a helpful story about a user or transaction. For example: digging into a shipping address to discover it’s the workplace of the cardholder, thus eliminating fraud concerns. Fraudsters depend on merchant teams that accept things as they are presented, no questions asked, to bypass rules and reviewers alike. Having someone who challenges assumptions to find the truth or real state of things is critically important to undermine this reliance.
Open-mindedness: Because merchants work with only a limited amount of information about orders, underlying assumptions can lead to inaccurate results in reviews. The best reviewers remain open to the possibility that their assumptions about how to fill in gaps created by missing data were wrong. This is especially helpful when, as often occurs, data elements provide conflicting information. Having an open mind to accept orders is a benefit to companies’ abilities to grow and support customers that would traditionally be denied, especially when their information isn’t perfectly aligned.
Decisiveness: Exploring possibilities and reconsidering assumptions are both important, but at a certain point a reviewer needs to make a decision. Being able to come to a conclusion quickly while taking all the right information into account is largely a product of being confident enough to make a decision and move on without agonizing over it. Having transparency in your tooling is critical to enabling decisiveness in your analysts. This will aid your reviewers’ confidence that they aren’t missing any details that would have informed their decision.
Monitoring Team Performance
The value of manual review is equal to money saved (or lost to cost) compared to what the outcome would have been for an automated review of the same transactions. This is a function of the reviewer’s accuracy (contributing to savings) and speed (reducing costs).
The two are related, and so a business decision has to be made at some point about how quickly you want your team to work—with the awareness that quicker decisions will necessarily be made without the most thorough investigation possible. It’s important to know that, when setting review targets for your team, they will inherently have some reviews that take longer. You may find value in working through reviews that can be done quickly, and saving more difficult reviews for later in the workday when you have more time to dedicate to them. Accuracy can be harder to measure, but one method some merchants use is to audit reviews performed by individuals on your team. To do this, get a list of manually reviewed orders for a given period of time (say, a week), with the order ID, the name of the reviewer and the resolution (usually cancelled or accepted). Depending on how the data is stored, you may have to instead get this information by resolution instead of by order, but the underlying idea is the same.
Take a sample of resolved, and ideally randomly sampled, orders—for instance, 20 cancelled and 20 accepted orders—and check to see if the reviewer used good reasoning in each case. If you do this regularly, and speak with the reviewer about decisions they reached where the reasoning isn't clear, you can help your team achieve greater accuracy over time. It’s equally important to share these insights with the broader team. What can help one individual can often help everyone, and hopefully you’ll build a team mindset that sharing should be accepted and welcomed as a pathway to better reviews.
Writing a Guidebook
One of the most valuable steps you can take to keep your review team’s performance consistent is to assemble a guidebook for your team to reference. Don’t feel like you need to have every possible scenario covered, but focus instead on broad guidelines: what criteria should cause an order to be cancelled, what follow-up steps should be taken whenever an order is reviewed, and so on. Having points of contact listed and escalation paths documented is also advisable for when things don’t go to plan, which is fairly routine when fraudsters are involved.
Get feedback from your team on where they feel they could use more guidance, and incorporate this into your guidebook over time. This will help in onboarding new team members and will give everyone on your team a consistent set of guidelines they can turn to when unsure of how to resolve particular cases. Additionally, having new team members provide feedback on your guidebook can introduce fresh eyes and mindsets to your existing policies and procedures. This is great to avoid simply sticking with procedures as they are because “that’s how we’ve always done them.”
Prevent all types of online fraud and abuse with a single fraud detection solution for end-to-end protection.
