Oct 31, 2012 · 5 minutes

The Internet-enabled explosion of personal data has improved our lives in many ways, but it has also created an enduring headache: keeping our crap in order.

Twitter lists. Facebook groups. Google circles. Netflix ratings. Contact books. Calendar updates. Our rational minds know that if we stay on top of all these things, they help make our Internet lives better. But our busy, distracted minds know that, taken in aggregate, they require a lot of work. And so we tend to err towards laziness.

Lucky for us, software developers are increasingly beginning to realize that catering to this laziness is a market opportunity. Thanks to open APIs, an era of Big Data, and cloud computing, companies such as Evernote (notes), CloudMagic (personal data), TripIt (travel), and Brewster (contacts) are finding ways to solve our organizational shortcomings without demanding any great user investment.

Let’s start with CloudMagic, a startup based in Bangalore, India. The 30-man team, which has no venture backing, has built a universal search engine for personal data that is blazingly fast and efficient when dredging up information from Gmail, Twitter, Facebook, Google Drive, and Evernote. I use CloudMagic’s Chrome extension and its floating search box, which clips onto the side of the page for every property it is indexing. It also has iOS and Android apps.

CloudMagic co-founder Rishit Jhunjhunwala says that basic algorithms are not enough to keep people organized over a long period of time. A simple system, such as a spreadsheet or a contacts app, might work today, but in three months there will be changes in your life that will throw the data out of whack. “Every few months, your context is going to change some way or the other,” says Jhunjhunwala. For instance, your friend might change jobs or move cities. For the software to be scalable and useful over time, it needs to be able to evolve as as your life changes. Jhunjhunwala thinks search is the solution to that problem.

When it comes to personal data, however, a simple search spider is not enough. Our data is spread across multiple buckets, from email to Facebook to Twitter to Dropbox to LinkedIn and more, and it’s constantly in flux. Plus, if CloudMagic can’t find an email that you sent 10 minutes ago, it might as well not exist. So, unlike normal Web search, CloudMagic has to index billions of items many, many times a day. And according to CloudMagic’s internal metric, each search has to produce a result within 20 milliseconds.

The engineering challenge is immense – Jhunjhunwala says his team is averaging less than five hours’ sleep a night – but the payoff for getting it right is attractive. On the consumer end, having a ubiquitous destination for all email, Facebook, Twitter, Google Drive, and Evernote data is a compelling proposition. And it means I don’t have to spend as much time organizing, or looking for, content on each site.

Contacts-management app Brewster achieves similar efficiency without asking much of the user, and by similarly leveraging search. Organizing contacts is the bane of any journalist’s existence. In the pre-Internet days, a little black book full of phone numbers was a journalist’s “killer app.” In my eight years as a reporter, I’ve also kept contact lists in Word documents, spreadsheets, phone address books, and in my email contacts database. Each one is a total pain in the ass to maintain. Now, my contacts are spread across LinkedIn, Facebook, Gmail, my phone, and Twitter, and I dart between each one depending on who I need to find, if I can remember where I put them...

The beauty of Brewster is not only that it centralizes all those contacts, but also that it allows me to search them by keyword. It lets me create lists if I want to, but it knows that I’m secretly too lazy to do that. So it has sorted my contacts into pre-made lists, such as “Losing Touch,” “Frequently Contacted,” and “Most Mutual Connections.” For the informationally lethargic, these minimum-exertion shortcuts are God-sends.

And these shortcuts are not necessarily bad things. In fact, they’re liberating. In Michael Lewis’ recent Vanity Fair profile of Barack Obama, the President explained why he only wears gray or blue suits.

“I’m trying to pare down decisions. I don’t want to make decisions about what I’m eating or wearing. Because I have too many other decisions to make.” He mentioned research that shows the simple act of making decisions degrades one’s ability to make further decisions. It’s why shopping is so exhausting. “You need to focus your decision-making energy. You need to routinize yourself. You can’t be going through the day distracted by trivia.”
Florida State University psychologist Roy Baumeister calls the decision-making tax “ego depletion.” Baumeister’s research suggests that humans have a limited store of mental energy and that “decision fatigue” erodes your mental acuity. In that sense, my reluctance to build organize my contacts sounds almost strategic.

Brewster founder Steve Greenwood has some sympathy for that point of view. Apps like Brewster that take care of life’s minutiae allow us to concentrate on more important things. “To me it's just the next level evolution for people,” says Greenwood, who believes we’re at a great moment in human progress. “We build these tools to help us create more leverage so we can work on higher-order levels of thought.”

Dan Foody, founder of a Boston-based startup called Cloze, evokes the concept of entropy to support the notion of automated organization. Cloze is a Web service that consolidates your email, business connections, social networks, and address books into a single view and makes it all searchable. Entropy has multiple meanings, but one of them encapsulates the idea that, in the sage words of the Free Dictionary, “all matter and energy in the universe evolve toward a state of inert uniformity.” Translation: shit falls apart.

By letting people stay disorganized, Foody says, you’re really just letting them be the way they always have been, and always want to be. “Organization is not natural,” says Foody, who has a Master’s degree in electrical engineering from Cornell University. “There’s a reason that everything decays to chaos.” Fighting that force isn’t always the right approach, he reckons. “Using it to your advantage might be the best thing to do.”

The winners in this burgeoning market for keeping our digital data in order will be the developers who not only acknowledge our laziness, but actively serve it. And the moral of the story? We're those fat guys on the space ship in "WALL-E." Feed us now.

[Illustration by Hallie Bateman]