Who are these guys?

The investigators were baffled. After 3 hours of investigation, they still haven't made any progress in understanding who should they be looking for as the prime suspect for the assault case. The problem? No, not a mismatching DNA sample. Not a picture that's not on the immediate suspects list. Not even scarcity of able people willing to crate a drawing based on the description from the victim. The problem, suprizingly, was that the victim would not let out any revealing detail about their assailant: gender? against the sexual harrassment act. Skin color? Dude, we're against any type of discrimination. Religion? get out of here. Lucky for the investigators, the guy (oops) was of average height. At least that went through.

Imaginary? Indeed. Possible? Of course. I once spent 15 minutes listening to a friend of a friend describing a very similar case, until I was able to understand what the person's profile was. Because in social interactions PC sometimes deters us from using specific observations. Makes sense. In the world of Risk management, however, such a starting point can be the blow of death to your ability to understand what exactly is attacking your system, and stop it.

Profiling is the name of the game, and some of us are not playing it, and are wrong at doing so. Because fraudsters lie every time they need to. They lie about their identity, they hide their connection, they use other people's details and they will come back to demand the service you are not giving them and might end up convincing you. But what's their motivation? Is a WFH scammer the same as a 419 fraudster or a WOW gold trader, or for that matter - a cusotmer that maliciously reports not receiving an item they have in fact received? Of course not. They have different starting points, different sets of tools and conceptions, they might even be from completely different regions of the world (quick hint: they are). And that renders behavioral attributes that either are not reflected in your analysis (beacuse the fraudster's age and favorite social network do not reflect in the account time-on-file or time before a Chargeback comes in).

When not profiling, you are bound to looking at losses as they appear, and then reverse engineer them using business dimensions to try and understand what going on. You might discover that your UK market for new intangible item transactions is high on chrageback rates. Is this a bad finding? Absolutely not, data driven analysis HAS to be the first step of any research - because segmenting the world is the first step toward prioritizing work and creating a souns results-related risk policy. But whan you don't ask yourself "why is this happening" and "who are these users causing losses" and even "what's their story?", you are missing on three big things:

1. the ability to further segment the world based on how bad user behaviors look in your system, and differentiate malicious intent from system errors (classification errors and others) and mistakes (the human factor - flakes, friendly fraud and others)
2. the ability to identify the good guys, and provide them with better treatement, even when they resemble bad guys in business segmentation
3. the chance of foresight - understanding where the bad guys might go next

Behavior based analytics isn't the sole answer to all BI problems. On the contrary - without a proper data driven segmentation, experts' intuition is both invalidated (and though usually is useful, is risky when it's the only thing you're using for long term planning) and will take a lot more time to create (since prioritizing where to look first is the proper use of your Oracles). But it is the single most important frame of thought your risk management team is probably not using - and whether this is happening because of PC, lack of domain expertise of just disinterest, you cannot let it pass you by.

Stop! Are you a fraudster?

A few days ago, Slashdot reported this blog post which neatly explains why CAPTCHAs are doomed. The post is very interesting; first, because the analysis in the post hits some good points such as why developing explicit, single factor screening mechanism in a global economy just doesn't make sense (and why a good ESP game is much cheaper than the next generation of OCR). Second, because it raises (maybe unknowingly) the most important point - that screening mechanisms and risk controls often turn away a lot more good business than they stop the bad guys. But third, and most importantly, is that it falls into the same pit by suggesting a few alternatives that are just as bad.

Let's admit it - we're not dealing just with a bunch of script kiddies with a knack for defacing popular sites. We're dealing with serious "bad guys" with a lucrative opportunity to use our systems, with a big shiny dollar sign at the end. And we want to stop them from doing so. Our only problem is that when we do so, we tend to make the legit buyers' lives much harder, because fraudsters are always more prepared and have more incentive to complete a purchase than the average buyer.

If you go back to square one, you'll discover that when coming to design a payment system one has to choose between an open and closed door approach. This might seem simple, but closed (only allow buyers you trust to make a purchase) vs. open (allow all to buy, then detect the bads while they buy) approaches not only define your risk aversiveness in general but also dictate your risk management strategy. True, the long term goal in each is to get to a nearly-perfect system and hedge the risk (more on hedging - thank you Tal - in a future post), but how do you get there?

All in all, we're looking to prevent scalable negative actions; reach a point where every frauster can only hit you once and you're in a completely new ballpark. For most merchants, the problem is the fact that fraudsters return to known exploits, and for most fraudsters the problem is finding and reusing without bouncing off rules and limitations. CAPTCHAs and Captcha-like mechanisms reduce the ability to quickly open many accounts ("horizontal scalability") while soft limits and caps limit the ability to create large losses through a single account ("vertical scalability"). By combining the two, one would expect, we make the fraudsters' livesharder, raise the "cost of fraud" and reduce risk. Somewhat right, somewhat wrong.

In themselves risk controls are not bad ideas, but to make good use of them they need to be utilized properly. Here's a common approach: "Heck, the last fraudster did one hundred 5$ digital goods transactions, let's stop anyone from doing that. Then that other one opened 5 accounts that are linked, let's not let any linked accounts in our system". Synchronous, always on controls, espcially explicit ones, raise the incentive to reverse engineer them. Use too many quotas and limits and you reach an unmanageable system with thousands of rules you forgot exist, and which effect on future (legitimate) buyers you cannot predict. This is why, among all recommendations, I would support heuristic profiling. It's a big word, true, and we'll need to shed some light on this subject before we move on; but only right profiling and segmentation of your legitimate and fraudulent users can allow proper use of risk controls and authentication mechanisms - one that doesn't strain legits for something fraudsters are trained at overcoming, and doesn't create a overgrown operations center (that doesn't justify itself) to manage.

That one small detail

"When the Chinese government instituted the policy in 1979, it touched off a wave of sex-selective abortions as pregnant couples decided that if they could have only one child they would benefit most from having a boy. That helped leave modern China with the largest gender imbalance in the world. Today, there are 37 million more men than women in China, and many of the boys are growing up unable to find a job or start a family.

So what are these “surplus” boys doing to fill their time?"

This isn't just a story about risk management - it's a story of pure business intelligence - it is a story of freakonomics. The German police has spent years chasing down someone that turned out to be a phantom, a woman who wasn't really a feared killer in many different, distant crime scenes - but merely a lab worker whose DNA "slipped" onto the cotton swabs German CSI people used to collect evidence (on another note, wouldn't it be just morbidly funny if that person turned out to be a real-life German "Dexter" copycat?).

So what does an unsanitized cotton swab have to do with abortions in China, and with risk management?

When one approaches modeling of complex situations (either to explain what just happened, or to improve decision making in the future), often the "sense" made in the process gets deterred by the fact that not all the data is revealed. This is why when Freakonomics' author Steven D Levitt says something along the lines of "if we had enough data, we could unravel the mysteries of the universe", many of us nod (however, I must say, we are not always right); we are in constant search for the added detail that, when added to the equation, will help the story make sense. It's not only as extreme as claiming that a rise in abortions is correlated with a drop in crime rates - retailers are always looking for the additional factor that will verify a bank account, provide details for a phone number or do this automated super sophisticated AVS check. But fact is that most of the added data doesn't do the trick, since looking for that additional detail requires a system.

Yes, having a single source of truth helps give foundation, but even the brightest have a hard time without a system - and the right one at that - for collecting, validating and understanding data. I've seen this in organizations here and there and the German CSI story demostrates it well. The CSI department has a system for examining a crime scene and extracting evidence, and they came up with a concrete linking theory between cases. It didn't shed light on the actual identity of the misterious killer, however it gave an interesting spin to a bunch of unsolved crimes, until it didn't make sense anymore.

What the CSI department lacked was a key component of creating robust linking stories - indetifying common resources. That common BIN number in your last week's transactions might be a result of a data breach in the processor level, but might also be a result of a marketing campaign for a new eCard; and that repeated IP creating new accounts may be a script attacking your system but may also be a whole trend-struck fraternity house shopping through the same computer for that special item only you are offering for a great price. Noticing the trend, understanding it and making the right call on how to handle it are key decisions we are facing every day, and not only in eCommerce. Common resources are one simple example where correct classification, using an external resource, makes the difference between turning away good business and letting the fraudsters in; between chasing a phantom killer and tracking down a less-than-perfect lab worker. Using the right contructs for doing this is key in our ever-changing profession.

The single source of truth

"Great Oracle, sleeping through the centuries,
Awaken now at last
And tell us how to save us from ourselves
and how to survive our own rulers
who would make a plutocracy of our democracy
in the Great Divide
between the rich and the poor
in whom Walt Whitman heard America singing"

(Lawrence Ferlinghetti, "To the Oracle at Delphi")

Ok, no politics. Here's the first rule of proper engagement with complex decisions: know the truth. It's so simple yet one of the hardest tasks ever in a large organization, especially one that deals with transactions every day. Because the Knowledge Boom hits hardest where you actually need to make sense of it. This isn't just going through your Google reader and finding the 5 interesting posts to read between the dozens you got last night from TechCrunch and Slashdot. It is (first and foremost) about finding those pieces that really matter, and using them to make money, or prevent from losing it; it is about finding what's the most important piece of data you aren't logging or don't have, and getting it; and finally, it is about making it all connect. Because without all of these, you're left with a blur called your payment system, and your best chance is third party vendors and Chargeback representment, and you know my opinion - it's not the best place to be in.

So, you say, what's the problem? I'll hire someone who understands Risk and I'm good.

Not quite. Here's an interesting dynamic: since many merchants either relate risk management to CS or demand a clear ROI for any headcount they're hiring, the risk or fraud management department often ends up as an underdeveloped group with CS responsibilities. Yes, this means that they'll start calling a lot of people. On the other hand, when the organization grows, in comes the industry veterans with their zest for business intelligence, segmentations and graphs. So you end up with a group of "factory" workers on one hand - who feel the "field" but do not know what to do with this knowledge (little to say generalize on it), and on the other hand you have the top squad, segmenting the world but never actually meeting the real fraud cases on a non-aggregate level. When the second group need to find what is happening exactly, they cannot rely on the first group, and they end up reverse-engineering the answer to "what really happened?" by digging deeper into your already-huge data warehouse, always reaching something that is just that-much better than a random variable, but never the actual answer. I know this is industry standard, I know it works well to a certain extent, I also know it loses flexibility and degrades after a while. In addition, I can tell you that this is the reason to not only fraudsters having a ball, but also (and much worse!) for legitimate people not making it through the grid of filters. So my advice to you is: get an oracle.

Who's or what's an oracle? You can think of this position as, at the very least, the missing link between your field agents and the BI experts. The "oracle" knows what's a good transaction and what's a bad one; they can rationalize the case and furthermore, they can generalize. Because proper usage of rationalization and generalization are key for an efficient decision making process: they lay the foundations for understanding why bad things happen, how do you spot them on time (and not in restrospect analysis of business performance or when the processor is already knocking on your door with chargeback fines), and what should you check. They have the ability to dive into the material and resurface with additional insight, and the ability to test your systems while you develop them. This is much more than a field agent becoming the newest member of the BI team - the oracle is not only a person with a specific talent, but also has the right system to enable their way of thinking that has nothing to do with Customer Support - as important as CS might be in your organization.

What's the talent profile, and what's the right system? Allow me to call this my little trade secret. But you have an important tip now - find an oracle. Find two. Have your own source of truth, that isn't just your most experienced field agent, and make sure they are all in sync (which is a challenge in its own). Then, finally, you'll be able to start planning automated systems that actually do the work your way.