No, wait: make that milliseconds.
The odds are that access to you — or at least the online you — is being bought and sold in less than the blink of an eye. On the Web, powerful algorithms are sizing you up, based on myriad data points: what you Google, the sites you visit, the ads you click. Then, in real time, the chance to show you an ad is auctioned to the highest bidder.
Not that you’d know it. These days in the hyperkinetic world of digital advertising, all of this happens automatically, and imperceptibly, to most consumers.
Ever wonder why that same ad for a car or a couch keeps popping up on your screen? Nearly always, the answer is real-time bidding, an electronic trading system that sells ad space on the Web pages people visit at the very moment they are visiting them. Think of these systems as a sort of Nasdaq stock market, only trading in audiences for online ads. Millions of bids flood in every second. And those bids — essentially what your eyeballs are worth to advertisers — could determine whether you see an ad for, say, a new Lexus or a used Ford, for sneakers or a popcorn maker.
One big player in this space is the Rubicon Project. Never heard of it? Consider this: Rubicon, based in Los Angeles, has actually eclipsed Google in one crucial area — the percentage of Internet users in the United States reached by display ads sold through its platform, according to comScore, a digital analytics company.
Rubicon is among a handful of technology companies that have quietly developed automated ad sales systems for Web site operators. The bidders are marketers seeking to identify their best prospects and pitch them before they move to the next Web page. It is a form of high-frequency trading — that souped-up business of algorithm-loving Wall Streeters. But in this case, the prize is the attention of ordinary people. And it all depends on data-mining to instantly evaluate the audiences available to see those online display ads, the ones that appear on Web sites next to or around content.
In industry parlance, each digital ad space is an impression. The value of an impression depends on several factors, like the size of the ad, the type of person who is available to see it and that person’s location.
“The first impression seen by a high-value person on the opening page of a major newspaper first thing in the morning has a different value than a user from China who is 12 and has been on the Web all day long playing games,” says Frank Addante, the founder and chief executive of Rubicon.
Yet for most of us, real-time bidding is invisible. About 97 percent of American Internet users interact with Rubicon’s system every month, Mr. Addante says, and most of them aren’t aware of it.
That worries some federal regulators and consumer advocates, who say that such electronic trading systems could unfairly stratify consumers, covertly offering better pricing to certain people while relegating others to inferior treatment. A computer-generated class system is one risk, they say, of an ad-driven Internet powered by surveillance.
“As you profile more and more people, you’ll start to segregate people into ‘the people you can get money out of’ and ‘the people you can’t get money out of,’ ” says Dan Auerbach, a staff technologist at the Electronic Frontier Foundation, a digital civil rights group in San Francisco, who formerly worked in digital ad data-mining. “That is one of the dangers we should be worried about.”
Of course, ad agencies and brands can tailor ads to Web users without real-time bidding. They can also buy ads without aiming them at narrow audience groups. But for marketers, the marriage of ad- and audience-buying is one of the benefits of real-time bidding.
Not so long ago, they simply bought ad spaces based on a site’s general demographics and then showed every visitor the same ad, a practice called “spray and pray.” Now marketers can aim just at their ideal customers — like football fans who earn more than $100,000 a year, or mothers in Denver in the market for an S.U.V. — showing them tailored ads at the exact moment they are available on a specific Web page.
“We are not buying content as a proxy for audience,” says Paul Alfieri, the vice president for marketing at Turn, a data management company and automated buy-side platform for marketers based in Redwood City, Calif. “We are just buying who the audience is.”
Still, for many consumer advocates, real-time bidding resembles nothing so much as a cattle auction.
“Online consumers are being bought and sold like chattel,” says Jeffrey Chester, the executive director of the Center for Digital Democracy, a consumer group in Washington that has filed a complaint about real-time bidding with the Federal Trade Commission. “It’s dehumanizing.”
FRANK ADDANTE is 36 years old and given to wearing black shirts with a white Rubicon logo on the front. Rubicon is the fifth company he has started or helped to found.
In 1996, in his dorm room at the Illinois Institute of Technology, he developed and introduced a search engine. He later helped found L90, a digital ad technology company that went public and was later acquired by DoubleClick. His fourth enterprise, StrongMail Systems, provides e-mail delivery infrastructure to large companies.
While working in ad technology, Mr. Addante says, he became puzzled by the manual ad sales processes that many Web sites were using. Just a few years ago, he recalled, many sites still executed their online ad deals through the cumbersome back-and-forth of meetings, phone calls, e-mails and even faxes. The fragmented market made it hard for ad agencies and brands.
“That market was very inefficient,” Mr. Addante said in an interview in Rubicon’s Manhattan office, “much like the early days of manual stock trading.”
Of course, other major industries already had automated sales systems. Concert arenas sold seats through Ticketmaster. Airlines sold tickets through a system called Sabre. Hotels offered rooms through Expedia.
So, in 2007, Mr. Addante and three other executives with whom he worked at L90, started Rubicon with the aim of creating an automated marketplace for Web sites to sell their ad inventory. Years earlier, Google invented a similar automated system for search ads.
“Google was the first to automate the buying and selling of search ads,” Mr. Addante says. “We thought, ‘why couldn’t we do this with display ads, mobile and video?’ ”
Although real-time bidding accounts for a small portion of online ad sales, it is growing fast. This year in the United States, advertisers are expected to spend about $2 billion on display ads bought through electronic auction-based exchanges, versus about $733 million in 2010, according to a recent report from Forrester Research. By 2017, the report estimated, that market is likely to reach $8.3 billion.
Rubicon’s customers now include ABC, eBay, CareerBuilder, Glam Media, Time Inc., the Drudge Report and Zynga. Its competitors include major players like PubMatic and Google’s DoubleClick ad exchange.
But Rubicon is not just a sales platform for Web site operators. It’s an analytics system that uses consumer data to help sites figure out how much their visitors are worth to advertisers.
Most sites, Mr. Addante explains, compile data about their own visitors through member registration or by placing bits of computer code called cookies on people’s browsers to collect information about their online activities. To those first-party profiles, Rubicon typically adds details from third-party data aggregators, like BlueKai or eXelate, such as users’ sex and age, interests, estimated income range and past purchases. Finally, Rubicon applies its own analytics to estimate the fair market value of site visitors and the ad spaces they are available to see.
The whole process typically takes less than 30 milliseconds.
“All these calculations have to happen before the Web page loads,” Mr. Addante says. “In our system, inventory is perishable.”
The competition for pricing accuracy has made companies involved in real-time bidding among the Internet’s most aggressive consumer trackers. Among the trackers setting the most cookies on the top 1,000 Web sites in the United States, for example, BlueKai was first, with 2,562 cookies, while Rubicon came in second, with 2,470, according to research conducted last month by the Berkeley Center for Law and Technology.
Consumer advocates say real-time bidding companies are acquiring and commoditizing all of that consumer data with little benefit to consumers themselves — and much digital snooping.
Mr. Addante and other industry executives disagree, saying consumers benefit by receiving ads and offers specifically relevant to them. Their systems do not invade privacy, they say, because they use numerical customer codes — not real names or other identifying details — to collect “anonymous” information about people’s online activities.
For many consumers, however, that Web and search history may seem personal, especially if they visit financial or health sites. Some computer scientists argue that the customer codes assigned to online users are unique ID’s, allowing companies to compile portraits about millions of people — without needing to know their names. Moreover, a few researchers have reported that many sites leak personal information, like names and addresses, to third-party trackers operating on their sites.
That means that rather than being anonymous, those customer code numbers are pseudonymous at best, some computer researchers say.
“It’s like a Social Security number, a number that businesses can use to recognize you on your future visits,” says Rob van Eijk, a computer science researcher at Leiden University in the Netherlands, where he is studying real-time bidding. Yet, he adds, consumers generally remain in the dark as to how automated trading systems rank and shunt them. “Envision a Kafkaesque future,” he said, “where decisions are being made about you and you don’t know what the criteria are based on.”
TICK. Tick. Tick. Tick.
The horizontal ticker at the bottom of Turn’s buy-side trading dashboard registers the groups of users available now to see ads — and lists the bids that Turn’s system recommends for access to them.
The ad spaces, or impressions, sell in lots of 1,000. The price depends on variables like the size and type of ad space, the type of user, and whether the user is in an urban or rural location.
One moment, Turn’s system recommends that an insurance customer bid up to $35.70 per lot being sold by Facebook Exchange, a Facebook service that auctions ad space on the social networking site, and $1.35 per lot being offered by AppNexus, another sell-side platform. That means Turn has identified Facebook’s lots as “premium inventory,” says Mr. Alfieri, Turn’s vice president for marketing, while AppNexus is selling ads on sites where little is known about the users available to see them.
Real-time dashboards like Turn’s, he says, have modernized the online ad trade in the same way that Bloomberg terminals revolutionized Wall Street trading. Ad agencies and brands can now check the intraday prices for various impressions. Many ad agencies have even created in-house “trading desks” to monitor and adjust their bids.
But Turn’s dashboard is more than a real-time ticker. It’s an analytics system that enables clients like insurers or car companies to identify common details among their best customer segments and then bid to show ads to people who resemble those best customers. The machine learning process gets better at pinpointing ideal audiences over the course of an ad campaign.
For example, Turn recently ran an ad campaign for a sneaker company that initially chose to buy a wide variety of impressions nationwide. But as Turn’s system analyzed the early sets of results, it began to separate audiences into the kinds of people who clicked on those sneaker ads, or later searched for the shoes on their own, and those who did not. Identifying common details among those people required the system to comb through its databank of nearly a billion user profiles for each transaction.
(Like Rubicon, Turn uses consumer data from third-party data aggregators for its analyses, Mr. Alfieri said, adding that the company has hired outside software services to strip names and other details from the profiles before Turn receives them).
The results of the sneaker campaign were surprising, says Bill Demas, the chief executive of Turn.
“It turned out that Republicans in certain districts of Texas basically did not exercise. We were able to adjust the campaign to try to aim more at Democrats,” Mr. Demas says. Without analyzing those user profiles, he says, “who would think that party affiliation would be an influence in advertising campaigns?”
In some ways, the consumer segmentation process is not as newfangled as it may seem. For decades in the bricks-and-mortar world, direct marketers have hired third-party data resellers to help them decide which customers should get catalogs or special offers in the mail. Real-time bidding is just a faster, smarter, more automated process for brands to find prospects likely to be the best fit for their products, says Joe Zawadzki, C.E.O. of MediaMath, a buy-side trading platform and data management company in Manhattan.
“How much is a rich person worth? To Mercedes, a lot. To a used Pinto dealer, not a lot,” he says. “It’s a different set of impressions for every marketer. That’s where the magic happens.”
But privacy advocates argue that real-time bidding is more problematic than direct mail because it often involves dozens of business-to-business companies — whose names most consumers have never heard of — collecting information and making instant decisions about them. The concern, advocates say, is that the very same automated bidding system that can distinguish coffee drinkers from, say, tea drinkers, and set different prices to show them ads, is also capable of distinguishing shopaholics or people in debt and potentially auctioning them to high-interest payday lenders.
“The reality looks like ‘we know a person is a sucker and they spend a lot of money on dumb things,’ ” says Mr. Auerbach of the Electronic Frontier Foundation. “Advertisers will spend more money to target them, and they aren’t savvy enough to know what is happening to them.”
AS real-time bidding gains traction, the consumer data-mining that fuels it is escalating. Yet that surge in surveillance may present a serious risk for online businesses.
The volume of data collection on the Web has surged 400 percent, from an average of 10 collections a page in 2011 to 50 a page this year, according to a study published last June by Krux, a company that helps businesses protect and monetize their consumer data. The report attributed the explosive growth to the ad industry’s shift to real-time bidding.
Krux also warned Web site operators about what it called “rogue data collection.” When publishers allow third parties, like real-time bidding platforms or information resellers, to collect data on their site, the report said, those partners often bring in other data miners whose practices the sites themselves cannot control. Those middlemen may use a site’s proprietary data to help competitors, the report said.
“Publishers who leak data leak revenue,” the report warned. “They face threats from middlemen who steal data and use it to create directly competitive audience-based offerings.”
Those threats may increase as real-time bidding moves more aggressively into mobile sites and apps, entities that may collect valuable information about users’ real-time locations and geographic patterns.
In May, Rubicon acquired Mobsmith, a start-up specializing in mobile ad technology. A few months later, the company announced that it was integrating real-time bidding for mobile ads into its system. Mr. Addante says he expects the industry to adopt real-time bidding for mobile ads faster than it had for desktop display ads. He also predicts that consumers will find tailored mobile ads for, say, a cafe or taxi in their vicinity, more pertinent than many Web ads tailored to them.
“I think mobile ads become more of an information provider than what is happening in display advertising where it has become a nuisance,” he says.
Yet the prospect of ubiquitous real-time bidding — online, on mobile devices and eventually on Web-enabled televisions — also hastens our transition to a totally traceable society. What we read and how we spend our spare time used to be private. Now those activities are becoming windows through which marketers scrutinize, appraise and vie to influence us for a price. Soon there may be no personal spaces left for our private thoughts.
“Real-time bidding creates the possibility for companies to tag you wherever you are going, without you knowing or having the ability to influence it,” says Mr. van Eijk, the computer scientist. “It is becoming a huge imbalance for the ordinary user because, in the end, the ordinary user is the product.”
