Insights Featuring Gartner
During a recent webinar hosted by Sift, featuring Akif Khan, Senior Director and Analyst at Gartner Research, we presented the key factors online retailers should consider when selecting a machine learning vendor for fraud prevention. Dr. Khan’s areas of expertise include fraud prevention using both rules-based and machine learning (ML) systems, behavioral biometrics, device identification and bot mitigation, in addition to traditional and evolving techniques to validate a consumer identity in digital interactions. He also advises clients on prevention of fraud via account takeover.
As fraud continues to evolve, most online retailers are overwhelmed by choice and frozen by indecision when it comes to selecting a fraud prevention vendor. As a former merchant myself, I can attest to how overwhelming vendor selection can be. It took me and my team a lot of bumps along the way to learn these best practices.
Although many focus their vendor selection on machine learning capabilities, the truth is that all vendors are ML vendors to some degree. They either embrace rules along with ML, resent rules and focus more on ML, or don’t use rules at all and rely solely on ML. So the true differentiators between vendors don’t lie in their ML capabilities—which are oftentimes proprietary anyways—but in other more targeted considerations that can yield more valuable insights.
When choosing a vendor, it’s important to take a holistic approach and consider all of the factors—pricing, ease of integration, outsourced vs. internal operating models, references, dashboard and reporting visibility, and machine learning capabilities—to determine which solution is right for your business. Keeping all of this in mind, Khan recommends five key factors for selecting a machine learning fraud prevention vendor, detailed below.
1. How much data the vendor processes
When comparing fraud prevention solutions, the more data, the better. Larger, more comprehensive datasets are more likely to be accurate because it’s easier to spot trusted or risky data. So while it’s important for any vendor to have advanced ML algorithms that are ever-evolving, it’s even more crucial to have access to as much data as possible. Khan provides a spot-on analogy for comparing vendors with varying amounts of data:
“A real-world analogy to this would be perhaps imagine you have an illness and you’re going to two different doctors. Those two doctors might have had exactly the same training, and they might use the same methods. But if one doctor has seen a lot more patients than the other one, you’re probably going to want to see the one that’s seen more patients, because they’re probably going to be more likely to make an accurate diagnosis, taking advantage of all of that data they’ve accumulated from seeing all of those patients,” explains Khan.
2. How applicable the data is to your business
Having a large base of data is only one part of the equation, though. It’s also a matter of how relevant the data is to your business. There are four models vendors may be using:
- Retailer-specific: Data only from your business.
- Industry-specific: Data from other retailers in the same industry.
- Region-specific: Data from other retailers in the same geographic region.
- Universal: Data from all retailers using the vendor’s solution.
It’s important to ask vendors what kind of data they process and from which buckets. To create the most accurate set of data, you’ll want a combination of all these models.
3. How vendors retrieve data
Another factor to consider is where the outcome data is sourced. Most vendors rely on supervised machine learning, which uses algorithms trained on historical data to flag which transactions are fraudulent. When asking vendors about where they get their source-of-truth data, make sure you’re familiar with the options:
- Manual review decisions: Rely on human judgment, making it more time-intensive and less useful.
- Chargebacks (via retailer): Rely on retailers to label chargebacks, making it a bit more useful than manual review decisions.
- Chargebacks (via acquirer or processor): Rely directly on the acquirer or processor to get chargeback information. This is a more automated option that means less effort for the retailer.
- Card scheme reports: Not all vendors will have access to this, but it offers the most reliable and comprehensive way to understand transaction outcomes.
4. How models are adapted and retrained
It’s best to steer clear of stagnant models when considering any vendor—if they’re not refreshing their ML algorithms regularly, their data may not be accurate or useful. ML models should learn and adapt in real time, not simply push out a score in less than a second or have to wait a quarter to get a refresh.
Vendors should constantly be adapting and retraining their models to ensure they’re utilizing the most recent historical data, which in turn impacts the accuracy of weightings, scoring, and penalties within the model. As an online retailer, you should ask how adaptive a vendor’s models are and how retraining may impact data outcomes.
5. The transparency and usability of model outputs
Just like working with any vendor, transparency is key. Having insight into ML models can make a huge difference when trusting a vendor to handle your business’s fraud prevention. If you have an internal manual review team, you can utilize these outputs to make decisions, drive accountability, and build trust with customers. Khan offers insight into how fraud prevention vendors can provide the most value to a business:
“I find that the vendors that I think are offering more value today are those which are really looking beyond the payment fraud, looking beyond account takeover, and are actually really able to offer value at these different events within the customer journey,” says Khan.
Any vendor can minimize risk—it’s crucial to overall business growth to also look at minimizing friction and false-positive rates to improve the customer experience. This is why so many companies have adopted a trust and safety mindset—to not only protect, but grow.
If you’re interested in how Digital Trust & Safety can help address your specific online retail challenges, prevent fraud, and help your business grow, I highly encourage taking the Sift Digital Trust & Safety Assessment. You’ll receive custom recommendations from our team of experts with decades of fraud-fighting experience at companies like Facebook, Square, and Google.
*Sift webinar featuring Gartner analyst based on Gartner, How to Select a Machine Learning Vendor for Fraud Detection in Online Retail, Akif Khan, Jonathan Care, 1 March 2019.