Smart risk management: why the "factory" approach could bring you down

If you deal with payments in any shape or form, you know you’re going to end up with a “risk management” team. A lot of times it creeps up on you: volume picks up and so you know you need someone to look at orders. If you’re running a small shop it’s most probably going to be you, but a lot of companies just hire one or two folks. These people use whatever tool you have to look at transactions – most times a customer service tool – and make up their technique as they go. With time, and sometimes with chargebacks coming in, you realize that your few analysts can’t review all transactions, so you turn to set up a few rules to make queue and transaction hold decisions. Since your analysts are not technology people you resort to hard coding some logic based on a product manager’s refinement of the analysts’ thoughts, again based on a few (or many) cases they’ve already seen. Not a long while passes, and you realize that the analysts are caught in a cat and mouse game where they try to create a rule to stop the latest attack that found its way to the chargeback report, and put a lot of strain on the engineers who maintain the rule-set. Even after coding some simple rule writing interface the situation isn’t better since the abundance of rules creates unpredictable results, especially if you allowed the rules to actually make automated decisions and place restrictions on transactions and accounts.

It’s at this point that you realize you need a statistician to run regressions, so you bring someone in. Hopefully, you have enough of a data set for them to create a decent regression model, and you can just get anyone off the proverbial street since regression is a very common tool. The statistician comes on board and creates a model that has industry standard false positives, let’s say 80%. Your review volume grows with transaction volume and you have to hire even more analysts and customer service folks to deal with complaints by legitimate customers – makes sense, since placing restrictions with 80% false positives will get you a lot of incoming calls. Then you discover that the regression model’s effectiveness degrades pretty quickly since they’re trying to predict what transaction will get a chargeback, but there are multiple reasons for getting a chargeback, making it harder to predict correctly. You then also discover that to create an updated regression model you need to wait for most of the chargebacks to come in so you’d have a good enough set of problematic transactions, meaning that you have at least 3 months’ lead time before a new process can be kicked off. That’s, of course, given that you have engineers on board to code the model.

Next thing you do is buy fraud prevention tools to add to your modeling power; you start creating black lists of IPs and Emails to mark problematic transactions. This improves things a bit but leads to additional false positives since people share resources. You consider buying a platform to manage rule and model deployment but decide that cost is prohibitive and generally, it looks like risk management is taking over your dev resources. So you decide to hire more analysts to do manual reviews, and a product manager to decide on the rule and model roadmap, and a risk operations manager for the growing group of analysts, and a head of risk. The rules and models you already have in place are blocking so many transactions you start to wonder if they’re not slowing down your growth instead of helping you protect your business. It looks like risk is managing you.

There’s something wrong with this model. Sure, some of what I’m describing makes sense for a company that’s just starting out, but getting caught in the factory approach to risk management is a huge burden in later stages, one that can be prevented by realizing that risk management is just one of a series of classification and inference questions a company needs to deal with, and that those require a different way of upfront investment in building a team.

I had two very interesting conversations about this very subject last week with two very inspiring people, but there’s one thing I remember a colleague telling me a few months back when they took a new job. The person insisted on being responsible not only for the new company’s risk management efforts, but also generally for their data and business intelligence. That was based on the same understanding: with the abundance of data created by organizations’ activity, all attempts to organize that data and make sense of it should be bound together. It doesn’t matter whether you’re qualifying leads, improving conversion or reducing fraud – you’re dealing with users and their actions, and how automated decisions impact them. It is the practice of making sense of data, and it transcends using data to control the experience of bad users. Once you realize that, you start demanding more of your analysts: that they be technical, know how to generalize on trends beyond targeted rules, become sources of truth. You understand that off-the-shelf regression cannot be just carried between domains without adjustment. You build a system that can correct itself. And with that, you create a team that can win risk, and do much more than that:  develop data sources, identify trends in huge data sets, and reach actionable insights that transform the way you work with your users, both fraudulent and legitimate.

What’s missing? I think there first needs to be a critical mass of people dealing with data in a way that sees beyond “intuition” but doesn’t get lost with over complicating inference using huge data sets. It takes time for these people to develop skill and want to continue solving difficult problems; my analysis group had less than 20 people in it and a good part of them have had enough of payments risk and classification for the rest of their career. I’m not even mentioning starting a company. But when you start a risk or data team, make sure you seed it well, or you’ll find that the bad start costs you a lot more money and effort than you have planned. 

Dealing with International Fraud - a Few Basics

When we started looking for customers in the first payments startup I worked for, low hanging fruit were obvious. All you had to do to find them was look for a merchant's international shipping policy - or lack thereof - and continue from there. The value proposition we offered, where we would make final accept/decline decisions and insure them, was just good enough to be true and be worth a lot of money for those who wanted to expand internationally. Still, it wasn't easy to convince these guys to expand, I'll tell you that - for every one who was willing to check us out, at least ten were pretty happy selling internally in the US. Who thought of the international market at that time? Looking back at it, this was around the dawn of managed fraud and risk services, and though we spearheaded the offering for the more dangerous segments we most definitely weren't the only ones.

Now, however, of all the questions I am asked, the ones I hear the most - and with the most urgency in them - are the ones regarding international purchases. Unlike a few years ago, when merchants let themselves brutally limit international buyers and focused on domestic markets, it's clear today that global expansion is a key for sustained success. Every beginning publisher wants to talk localization. And they should: this is way more general than digital goods and content. While US eCommerce is forecasted to grow to 8% of all retail purchases in 2012, according to Gartner, European b2c sales are forecasted to outgrow US sales, and grow 20% in 2010, according to eMarketer. This is an amazing opportunity – and it means that a lot of real goods need to be shipped around the world. However, when you get to actually approving these transactions, often you find that you just don't get the tools you're used to outside of the biggest eCommerce markets and some don't even exist outside of the US.

So how do you deal with those tricky international purchases?

• Remember what international fraudsters aren’t – they’re not the people they are stealing from. Sounds very basic, but it will serve you well – most fraudsters are young, computer savvy males from 3rd world countries trying to use Western world cards and bank accounts. Note obvious mismatches in details: if details given for the customer (phone number, card bin country, address) just don’t match, come from distant parts of a country or look invented, beware.

• Purchasing history from other merchants, through a 3rd party vendor, serves you mostly when you delay shipment (either because it’s standard practice or you’re suspicious). For all other cases, you need to have velocity checks and an ability to identify returning fraudsters alternating details. There are some good machine-ID companies out there, but you also have to complement with rules that identify purchasing behavior that is different than what you are used to in your industry and shop.

• Contacting users makes sense – but only when you understand what contacting them tells you. Calling a VoIP phone does no good, same as emailing someone whose email domain ranges from the ridiculous @legit.com to the less obvious @army.com; some seemingly fine domains host sites that are nothing but a blank page, so checking occasionally makes sense.

• IP intelligence can teach you a lot – you wouldn’t be surprised to hear that there are more fraudsters and more exploited, Trojan infested computers in big cities with high speed internet. It’s always good to know more about your user’s connection, especially if they are risky – if someone is initiating a payment to your site from within Microsoft’s Azure cloud, you may be up for some trouble.

• Find alternative data sources. No other country has such extensive public data sources of its citizens as the US, but free and paid data bases exist outside of the US too. A good address and name resource like 192.com helps you know more about your customer, and social networks span world wide. Too bad fraudsters can use this too…

• And, last but not least – know that there are legitimate people out there acting very ordinarily, but in a way that might strike you initially as dangerous. Where people relocate between states in the US, in the EU they do so between countries. Belgium and France share a language, and exactly as an Austrian might have a German bank account, so can someone from the Turkish minority. Time to polish your skills in geography, and read some Wikipedia pages!

Applying the above should take you a few additional steps in your way to open up your site to international commerce. And one additional thing to remember: deploying a great set of filters in place is close to useless without having a team reiterate on it and improve it as user behavior changes - the alternative is reactive risk management, slowly closing down itself using black lists and limitations until you resort back to the good ol’ US domestic shipping. Don’t let that happen to you, the international opportunity is too big to miss on.

No more secrets: managing risk when access control breaks

This post is a first in a series I will be exchanging with Allison Miller, one of my esteemed colleagues in Paypal's Risk organization, in her reinstated blog.

“Man may be defined as the animal that can say "I," that can be aware of himself as a separate entity”. (Erich Fromm)

“Identity” is a widely debated term, in various areas; Philosophy, psychology and social sciences discuss various aspects of the individual’s and a society’s identity and its representation in media, art and academic thought – from the Buddhist extremity of no-self to the capitalist self-definition based on what you buy, the variety of ancient and modern thought around definitions and applications of identity is vast. Loyal to the spirit of individualism in the Western world, the development of the New Age movement over the last decade led to the calling to each of us to find our own “true identity” through introspection; supported by modern psychology, the journey of identity constantly drives for defining, consolidating and presenting our personalities through titles that illuminate various aspects of our day to day behavior as part of a healthy, consistent and coherent identity that is who we are.

Too much information: you may just have all the data you need

"This was not a failure to collect intelligence, it was a failure to integrate and understand the intelligence that we already had." NYTimes quoting President Obama after his meeting with national security advisers about a terror plot to bring down a commercial jetliner on Christmas Day. (Jan 6th 2010)

Going to the movies with friends from the intelligence community is never a cheerful experience. Spending two hours in a conspiracy movie with people who sometimes while seeing a (seemingly) absurdly powerful data collection device say “ah, I know this system”, will make you a firm believer in conspiracy theories or at least a more paranoid individual. But even the most tech savvy and well informed of those people talk like Pres. Obama in that quote above – it’s not lack of data, it’s our inability to process it that limits us. Maybe project ECHELON really stores all of our communication – but what super computer and what sophisticated algorithms can process and identify all of the world’s pictures, plethora of dialects in written natural languages and voice calls? You know what? If you know the answer, I’m not sure I want to know.

A man on a plane

Following the latest news of the attempt to blow up a Delta flight, and the reintroduction of debates about terror and security worldwide, I want to share some random thoughts this incident brought about.

The weakest link

A reliable source is one that provides you data and information you can use with little to no validation; a source you can trust as part of the group of sources you use to evaluate the riskiness of a specific situation. Be it a credit report from Experian, a Whitepages entry from Whitepages.com or a customer calling in to report, you need to know the possibility of your resource being compromised and the information you receive being mistaken or, much worse, maliciously injected by fraudsters. This is the basic malfunction that drives SQL injection attacks, if you don't sanitize DB entries you're most probably in for a big bad surprise. The weakest link – in this case, it seems to be Nigerian aviation security controls – has failed the whole chain. It may be improper screening, low budget security tools or just procedures not permeating through the system, but it let someone with malicious intent onboard and only luck failed him. The fact that Netherlands security just passed the stick on and let all passengers continue shows that the hand-over between security personnel in different airports might need some additional reinforcement, because terror is constantly looking for ways to inject itself in. There should be additional focus around determining the reliability of various airports as a reliable source of validated passengers and acting accordingly.

Lists don’t work

So his name was on a list. So what? Here’s what lists do: they make legitimate people’s lives harder (ever tried boarding a plane in domestic US with an Arab name or with a Middle Eastern passport? Enjoy the ride…) but much worse than that, they transform risk measures into binary checks (on the list? Stop. Not on the list? Carry on), a classic case of “searching under the streetlight”. So he WAS on the list but not under “really bad” but only under “naughty”? Come on. I have preached against black lists in the past (Hebrew only) and this is another case where, clearly, some old fashioned flight track analysis crossed with previous alerts could have made the trick. The data was there – it’s all a matter of interpretation.

Hindsight’s 20:20

I take off my shoes in remembrance of the shoe bomber; I don’t carry liquids in remembrance of the 2006 bomb-as-a-soft-drink plot; and I get sniffed by an automated sniffer every once in a while in a random US terminal. As far as I’m concerned, I should probably stop flying soon and leave air travel to terrorists and security, in an everlasting cat and mouse game. The most important thing about attacks that materialize (even if they fail) is learning from them. If all we get is another restriction, we are missing the point here. Every false positive and false negative (in any automated or manual decision making process) needs to serve as feedback to the system to improve on – in its ability to make better decisions, not in the restrictions it applies on the general population. Hopefully, the conclusions will not end up only bringing another top-dollar cutting-edge new machine to sniff people at airports, but will aid in making flying safer and easier for legitimate travelers while shutting it down for terror.

42% of users have a good reason to fear

Working in the risk management business, I often get these layman questions about ePayment security. They are close relatives of questions IT people are being asked about hardware purchasing; when people finally find that item they wanted to find or a bargain they can’t resist, they want to make sure they don’t get scammed. Who’s better for that than your friendly neighborhood risk management specialist? I’ve given my part to eCommerce, you should know, and if retailers felt a $3000 shift in their revenues this year – this one’s on me, guys. No need for commission this time.


Seriously, though – why are thousands and maybe hundred-thousands of interactions related to purchasing on the web really important? As I mentioned in my previous post about Square’s trust issue, good payment services instill trust (among other things); and for an industry based on users exposing themselves and their financials, trust – created, in my case, by getting a recommendation from an authority – is one of the main challenges for emerging companies.

Where is my mind? Way out, in the water

(As I'm writing this, EA has announced it has bought PlayFish. All the more reason for a call to the industry to stop panicking and start taking responsibility for its own faith with big fish coming to play. But read on...)

One of the many highly useful skills I learned in Officers' course was artillery aiming. There was a lot more fun stuff I could imagine doing in any given afternoon, but there's definitely nothing like it. And when you just don't have an option (and believe me, in officers' course you don't have an option), you just give it your best shot. Pun intended.


So there I was, trying to get 155 mm cannon to hit a barrel. I don't know if you know how these things go, but artillery aiming is some simple arithmetic and a lot of art. You aim the cannon one way, then course correct the other, then again - in shrinking intervals, until you hit the target (or 50m away from it, which is considered good enough). It must have taken me 5 or 6 attempts to hit the goddamn thing - the gun crew was not a group of happy campers, nor was I. But all in all, it was a good drill, and I passed the test, and got my rank of deputy lieutenant, and mom was happy.

Offer walls and marketplaces: the real alternative to "scamville"

Let me just say one thing up front: well done, Mr. Arrington! From the first clash with Offerpal (former, it seems) CEO Anu Shukla, through this post and others, there's been quite a stir around offer walls and the big question of the legitimacy of their offers (some news sites in Israel literally copied the post's words. But that's another type of scam). Beyond the provocation, there are a few actual issues here, that I think are left out since "scamville" and CEOs being replaced are much more sexy.

Here's the thing: if the social gaming industry is a viable industry (which I think it is) it should, at one point, start to mature as one. Maturing doesn't mean moving slower or becoming less appealing to users, on the contrary, there's still huge potential and a momentum so strong can't just be stopped by a few posts. But what it does mean is that you start getting attention for your mishaps and you need to start addressing this attention in a tone that is way, WAY milder and more responsible than just saying "this is sh*t and bullshi*t" (look here for some current thoughts of industry leaders and how I'd respond to them).

The EU is less united than expected

This mystery research, widely advertised today by the EU union's research department, puts cross border shopping declines inside Europe at 60%. I once wrote a post about 3rd world shoppers unable to shop, but this situation is a much graver one. Unfortunately, the pros' call to invest in better, more intelligent risk management to open up to international purchases goes unnoticed, while merchant insist on making lives harder for legitimate buyers.

Hopefully SEPA will help solve at least part of the issues dealt with here, at least giving a head start for merchants and buyers on their mutual trust issue.

Reconstructing Zynga: the industry's opinion on fraud in social games

My previous post about fraud in Social Games raised a few objections and spun a few sub-discussions. That's great, because it shows people are interested, and there's a LOT to be discussed in this field. I wanted to circle back to some of the main points that were raised in this discussion.

There's nothing new about fraud. Really. Ever since people walked this planet, I would assume, there has been fraud - more and more as time advances and human kind introduces additional currencies that replace tangible goods. It's beyond the limited availability of tangible goods; being able to control supply and demand through a symbol (call it cash, checks, virtual currency or repackaged subprime mortgages) is the basis for modern economy. But is the fact that fraud isn't new merely a reason for underestimating it? Definitely not; if it were, then why is the Spanish Prisoner scam, better known in its current days' reincarnation as the Nigerian Scam, still rampant on the web?

And now for something completely (?) different

I'm diverting from Risk per se the deal with another decision-automation question I'm wondering about.
High-tech fluctuates. It boomed on the verge of the new millennium, and did so (albeit differently) before the latest downturn. And when booming, help is required. High-tech companies don't usually post a "help wanted" sign on their office wall (though some in Israel did), and getting to a good position requires some work beyond coming from a good school. In the days of the "bubble", just knowing a few people would secure you a position somewhere in the space, but nowadays it takes a lot more than that - employers demand good grades, subject matter expertise and experience - all of which are no mere feat for new graduates.

Jacob doesn't mind

Let's say there's a guy names Jacob. This guy, he's 23 years old, has somewhat of a steady job, largely sales and maintenance for a nice apartment complex in southern California. He uses PayPal, a lot more than he would like. He also has a Facebook account and a MySpace page; he follows friends on Twitter (and sometimes updates his own status messages there). He has an iPhone 3G; he's on top of things. If he was ever hit by fraud, he would probably tell his friends about it.


You know what? The industry is missing on many of Jacob's friends. Not because they don't have credit cards or because they don't shop online - it's because we haven't changed with them. Why? Because Jacob doesn't mind - he doesn't mind his information being out there on the web (as long as it's kept with a privacy policy). He doesn't mind some interaction with risk controls because web 2.0 and post 9/11 safety education taught many users that it's ok to be asked questions by those with authority. And in the land of risk management online, we are the authority. And we are limiting our business. Jacob and his friends don’t mind working with us to make their lives better – we simply won’t let them.

Deconstructing Zynga: what's up in Social Gaming fraud

Talking to friends in a party I had to hold myself from becoming too smuggy-smug-smug. Yep, the lot of "I'm too good for Mafia Wars" geeks fell prey to the eggplant-growing rhythm of Farmville. Eggplants. My friends. I don’t even like eggplants, but still felt responsible in a way, though they’re only a drop in Zynga’s estimated 15M+ daily users (the numbers keep growing...). But things were only getting better for me that day.

“You know”, said one of the guys, “this social gaming stuff is really worth a lot of money. I know someone who made $100K off this thing”.

KACHING!!! Immediately he had my full attention. You don’t just MAKE $100K playing social games by the book, even if you break a finger playing Texas Hold’em. I had to know.

Heartland my love

So the security-related part of the web is stirring over the Heartland breach going to court, and having fun mocking Heartland for falling for the oldest trick in the SQL-injections book. Since Israel's IDF's chief of staff was also a victim of his credit card being stolen, newspapers in Israel feasted over this "hot news" item, to the extent that one blog even names Albert Gonzales (the "brain" behind the attack. I wonder who Pinky is) "The Al Capone of Cyber Thieves".

Geez.

A flurry of blog posts and articles followed, telling us that checking your credit report is important (really?) and pulling some chargeback stories from the attic. One even went as far as interviewing the manager of operations for one of Israel's issuers. Don't get me wrong, while I'm against trying to scare people, public education makes sense (though many time is useless, as I have claimed in the past [Hebrew]). But the part I'm much more interested in is not the fact that a breach happened, those happen all the time although some retailers just hide their negligence. What I’m interested in is the publication of such an indictment, and its effect of the psychological aspect of committing internet fraud.

You see, analysts profile people. We know who the average fraudster is: a young, tech-savvy male with a knack for gadgets and digital goods, who thinks he could get away with it pretty easily. The “getting away with it” part is the important one; be that the average fraudster or a desperate housewife looking to earn a few dollars defrauding buyers on eBay, the mental state needed to commit a felony on the web is much less delinquent in nature. Because the web is not “the real world”. Because doing it over the computer pushes it away from me. It’s not me; actually, it’s my avatar. And pressing charges in the real world against people who wronged in the virtual world makes it as real as it gets. This, in turn, makes people a lot more aware of what they’re doing when they’re stealing – and the heuristic of a self-aware fraudster are different than those of one that isn’t. A fraudster who isn’t afraid of getting caught looks a lot more like your average Joe, and this is something we want to prevent. This is not only because risk analytics become easier (and legit people’s lives become better, since we need less “tricky” controls), but because indicting fraudsters is the right thing to do. Security and trust are, I believe, the key foundations of a thriving online community, and I’d like to help keep it as such.

This summer is about digital goods

"Bogdan Ghirda is paid £70 a month to do what most bosses would fire him for. From the moment he arrives at work he plays computer games on the internet."
(From the 2005 Observer article, "Virtual sweatshop")

Gold farmers didn't invent digital goods, though they've been around for quite a long time. People are not only buying MMO money - the market has expanded. What started as a black secondary market for harvested goods soon became a profitable channel for gaming companies that make their money - surprise surprise - based on the interface of your all-favorite social networks. Yes, while Facebook is struggling for monetization, companies like Zynga make hundreds of millions of dollars by running social games that are multi player, asynchronous, and let you buy any type of addition, from "special powers" for your vampires to "new clothing" for your soccer team.

You gotta love this culture. Honestly, it's amazing to see the thought, time and money invested in these games. There are numerous trends in this area, attracting more and more talented people who feel the buzz and want to take their share. And as they advance in creativity, these games move to main stream social network users but continue to evolve in the complexity they provide and the story they allow you to tell.

With them, obviously, come the fraudsters. In an industry so used to checking physical shipping destinations (via AVS) and managing proofs of shipment as a tool for dispute resolution between sellers and buyers, how do you deal with instantly delivered, non tangible goods where quality is sometimes purely in the eye of the beholder? In addition, fraudsters looking to steal digital goods are usually a mixture of sophisticated internet users and kids using their parents' money, sometimes referred to as "friendly fraud". So, if you're in the Risk business, mobile payments or into social networking in general, expect a pretty hot summer in everything digital, with fierce behind-the-scenes competition and major losses to fraud. I am looking forward to seeing which will be the winner in this field - is Paypal stirring something up with the new API, are small players like Boku.com going to lead or is Facebook going to make its debut in payments supporting the tidal wave of social gaming on its site? The coming months will tell...

Ain't doing it right

"How many legs does a dog have if you call the tail a leg? Four; calling a tail a leg doesn't make it a leg." (ascribed to Abraham Lincoln)

In our business, to make a good decision, it is essential to know what really happned. So we discussed finding the single source of truth, but have not discussed ways for keeping it truthful. Oddly enough, the concept of immediate, detailed feedback is not as common as one would expect.

In your community of domain experts, the concept of "truth" should not only be determined but also enforced by members of the community. Note: not by a moderator; the members must know what the "truth" is (in procedures, in decisions and in deriving conclusions) but also be ready and empowered to call out their and others' mistakes. Because direct feedback is what enforces people to improve in the specific of their work. You do not only need people who can tell a tail from a leg - you need to give the one who detects it the means to show their finding to the general community.

This is not a matter of virtue, it's a matter of getting your business runnig the way it should. What happens if you under develop this area in your organization? Well, first you get only hindsight feedback, allowing you to know what's happening in delays of months and months (how much time does it take 90% of chargebacks to come in? exactly), but you also get feedback in aggregate levels (saying, for example, how many of person X's decisions were reversed) - meaning that you can't really find the trend and fix it.

I can't tell you it's fun - commenting, moderating or acting on the results of such feedback cycles - but one thing's for sure, it's way more effective than pretending your Risk experts live in DisneyLand. Giving and receiving proper feedback improves every bit of the cycle - and makes your business better at one of its core competencies.

Too much data, too little information

So, you have this big 1000 user system, with its flows and checkpoints and flags and pointers. If you've grown it well you have a dashboard showing you login numbers, counts of transactions, dollars moving around. You control it all from your NOC, pressing the little red buttons whenever necessary, moving dials and reading graphs. But the thing is, that seeing the bits and pieces of online life on your screen doesn't necessarily, and sometimes doesn't at all, help understand what's going on.

What IS going on in your system? What are users doing, and will that translate into the bottom
line? What can the numbers tell you?

Well, we've been through a few ideas. Experts knowledge ties symptomatic indicators with identities and with what they intend to do, so that you can at least start making sense. Collecting the data is one aspect, and using it to understand is a whole new area. When we reach tips and tricks on how to develop your own methodology, some of this might start ringing a bell. But this post is about one system that shouldn’t be adopted as your main tool if you’re the risk management expert – it’s about advising you to not count on hindsight based on business results.

No, no, don’t get me wrong – business results are important, one of the most important aspects of the business (and some will argue – the single most important – but that is another discussion). But using the bottom line (or even a highly detailed version of it, including a drill down of, for example, every auth rejection code) to indicate what the risks are in the system or worse yet – to indicate what needs to be fixed – is a call for bad judgment. Consider my favorite example, a hospital. If you needed to weigh two hospitals one against another, would you use the percentage of deceased patients as an indicator? Would it matter that one has an oncology department and the other doesn’t? Would it matter that one is in Mozambique and the other is in Mexico? Of course it would, since when all else is equal (in staff, training and tools – like your company compared to other retailers), fraud-on-entry (the hospitals’ location and the indigenous diseases you’d expect) and fraud MOs (the types of diseases that are actually seen and treated or not treated) have a big impact on the bottom line. Trying to use the numbers post risk controls, chargeback, CHB dispute and collections to understand what could have happened is trying to pin down a moving target – and the wrong one at that. Worse of all would be trying to design future systems based on the current snapshot, since you do not have any indication of what users do – just how much money it costs you, and user behavior is much more volatile than your incoming chargeback count.

When you come to understand what’s going on, business results are highly important. But letting them steer all of your team from looking at user behaviors will put you exactly where you don’t want to be – patching up holes in your system using a highly delayed hindsight mode. To be successful, combining data analysis and behavioral research is a must.

Differential diagnosis, people!

House - "Haven't done the MUGA."
Wilson - "Then how do you know she needs a heart transplant?"
House - "Got my aura read today. Said someone close to me had a broken heart."
(Season 1)

Yes, I admit it, I'm an avid "House, MD" fan. The fun part about this show is that a lot of people find meaning that's beyond the plain action to relate to - much different, I assume, than what the writers meant. Some watch it for plain medical aspect, like a good mystery story; some treat House as their fictitious mentor; some like the twists of the tale. I sometimes watch it like a tale of business intelligence and a general case of decision making with partial information.

Here's how it usually goes: in comes a case. It either looks suspicious upfront or bad indicators come up immediately at the beginning (by the way, did you notice that in most of the first half of season 1, it was seizures?). Then they go through "Differential diagnosis" and run various tests; additional symptoms are discovered, and usually the truth is discovered by connecting details that hid from the doctors (because "everybody lies") or simply because they didn't connect the dots.

Yeah, real life medicine isn't that simple, and sometimes even knowing what happened is too complicated to be nailed down case by case. Obviously catharsis doesn't come, like clockwork, every 35 minutes - just in time for the drama. But it's pretty similar, isn't it? In comes buyer A, and presents the details of person B. Not much to say about buyer A - their IP connection (anonymized?), their email (opened yesterday?), purchase details, maybe shipping address. Nothing much on person B either - name, address, credit card number. Would you let the purchase go through? Differential diagnosis, people! What test can we run to verify this person, or establish fraudulent behavior? What does it mean if they can verify the email, answer a call to their mobile phone, tell you that the issuing bank is Citi? What additional indicators are we missing? Because that's what the "game" is - in comes a case - what do you do? No one is dying, but your balance sheet is going to look pretty bad.

The trick about decision making in this case is understanding what the next step is. Our goal, whether asking the customer for additional details or looking for an additional data source (what's next - Family history review? MRI? CT scan?), is to reach a conclusion in as little steps as possible, meaning that we need to be able to choose the steps that contain as much information as possible. BI experts sometimes tend to get as much data as possible, sometimes at enormous costs (these external vendors don't come cheap). House's department costs the hospital millions of dollars a year, but that's human lives. We need to be cost effective.

One major way to work with this is automated decision making systems - expert system - which help experts reach decisions by dealing with the quantity of data by using statistical models for classification. Advanced systems, when correctly fed with symptoms (or fraud indicators), can even suggest tests to rule out corner cases. Constructing such a system is the end station of the long road that starts with the single source of truth - in House's case, the doctor. In fact, expert systems in the field of medicine usually outscore doctors in identifying illnesses based on differential diagnosis - it only makes sense, when you hear House's staff shooting diagnoses based on remarkable memory and years of experience. Which brings out the question - why doesn't House use one? It would immensely scale his ability to save lives.

But then again, how much fun will that be?

Who are these guys?

The investigators were baffled. After 3 hours of investigation, they still haven't made any progress in understanding who should they be looking for as the prime suspect for the assault case. The problem? No, not a mismatching DNA sample. Not a picture that's not on the immediate suspects list. Not even scarcity of able people willing to crate a drawing based on the description from the victim. The problem, suprizingly, was that the victim would not let out any revealing detail about their assailant: gender? against the sexual harrassment act. Skin color? Dude, we're against any type of discrimination. Religion? get out of here. Lucky for the investigators, the guy (oops) was of average height. At least that went through.

Imaginary? Indeed. Possible? Of course. I once spent 15 minutes listening to a friend of a friend describing a very similar case, until I was able to understand what the person's profile was. Because in social interactions PC sometimes deters us from using specific observations. Makes sense. In the world of Risk management, however, such a starting point can be the blow of death to your ability to understand what exactly is attacking your system, and stop it.

Profiling is the name of the game, and some of us are not playing it, and are wrong at doing so. Because fraudsters lie every time they need to. They lie about their identity, they hide their connection, they use other people's details and they will come back to demand the service you are not giving them and might end up convincing you. But what's their motivation? Is a WFH scammer the same as a 419 fraudster or a WOW gold trader, or for that matter - a cusotmer that maliciously reports not receiving an item they have in fact received? Of course not. They have different starting points, different sets of tools and conceptions, they might even be from completely different regions of the world (quick hint: they are). And that renders behavioral attributes that either are not reflected in your analysis (beacuse the fraudster's age and favorite social network do not reflect in the account time-on-file or time before a Chargeback comes in).

When not profiling, you are bound to looking at losses as they appear, and then reverse engineer them using business dimensions to try and understand what going on. You might discover that your UK market for new intangible item transactions is high on chrageback rates. Is this a bad finding? Absolutely not, data driven analysis HAS to be the first step of any research - because segmenting the world is the first step toward prioritizing work and creating a souns results-related risk policy. But whan you don't ask yourself "why is this happening" and "who are these users causing losses" and even "what's their story?", you are missing on three big things:

1. the ability to further segment the world based on how bad user behaviors look in your system, and differentiate malicious intent from system errors (classification errors and others) and mistakes (the human factor - flakes, friendly fraud and others)
2. the ability to identify the good guys, and provide them with better treatement, even when they resemble bad guys in business segmentation
3. the chance of foresight - understanding where the bad guys might go next

Behavior based analytics isn't the sole answer to all BI problems. On the contrary - without a proper data driven segmentation, experts' intuition is both invalidated (and though usually is useful, is risky when it's the only thing you're using for long term planning) and will take a lot more time to create (since prioritizing where to look first is the proper use of your Oracles). But it is the single most important frame of thought your risk management team is probably not using - and whether this is happening because of PC, lack of domain expertise of just disinterest, you cannot let it pass you by.

Stop! Are you a fraudster?

A few days ago, Slashdot reported this blog post which neatly explains why CAPTCHAs are doomed. The post is very interesting; first, because the analysis in the post hits some good points such as why developing explicit, single factor screening mechanism in a global economy just doesn't make sense (and why a good ESP game is much cheaper than the next generation of OCR). Second, because it raises (maybe unknowingly) the most important point - that screening mechanisms and risk controls often turn away a lot more good business than they stop the bad guys. But third, and most importantly, is that it falls into the same pit by suggesting a few alternatives that are just as bad.

Let's admit it - we're not dealing just with a bunch of script kiddies with a knack for defacing popular sites. We're dealing with serious "bad guys" with a lucrative opportunity to use our systems, with a big shiny dollar sign at the end. And we want to stop them from doing so. Our only problem is that when we do so, we tend to make the legit buyers' lives much harder, because fraudsters are always more prepared and have more incentive to complete a purchase than the average buyer.

If you go back to square one, you'll discover that when coming to design a payment system one has to choose between an open and closed door approach. This might seem simple, but closed (only allow buyers you trust to make a purchase) vs. open (allow all to buy, then detect the bads while they buy) approaches not only define your risk aversiveness in general but also dictate your risk management strategy. True, the long term goal in each is to get to a nearly-perfect system and hedge the risk (more on hedging - thank you Tal - in a future post), but how do you get there?

All in all, we're looking to prevent scalable negative actions; reach a point where every frauster can only hit you once and you're in a completely new ballpark. For most merchants, the problem is the fact that fraudsters return to known exploits, and for most fraudsters the problem is finding and reusing without bouncing off rules and limitations. CAPTCHAs and Captcha-like mechanisms reduce the ability to quickly open many accounts ("horizontal scalability") while soft limits and caps limit the ability to create large losses through a single account ("vertical scalability"). By combining the two, one would expect, we make the fraudsters' livesharder, raise the "cost of fraud" and reduce risk. Somewhat right, somewhat wrong.

In themselves risk controls are not bad ideas, but to make good use of them they need to be utilized properly. Here's a common approach: "Heck, the last fraudster did one hundred 5$ digital goods transactions, let's stop anyone from doing that. Then that other one opened 5 accounts that are linked, let's not let any linked accounts in our system". Synchronous, always on controls, espcially explicit ones, raise the incentive to reverse engineer them. Use too many quotas and limits and you reach an unmanageable system with thousands of rules you forgot exist, and which effect on future (legitimate) buyers you cannot predict. This is why, among all recommendations, I would support heuristic profiling. It's a big word, true, and we'll need to shed some light on this subject before we move on; but only right profiling and segmentation of your legitimate and fraudulent users can allow proper use of risk controls and authentication mechanisms - one that doesn't strain legits for something fraudsters are trained at overcoming, and doesn't create a overgrown operations center (that doesn't justify itself) to manage.