Holy Web App! Social Radar 3 in beta testing.

When I tell geeks what type of applications I get to work with on a daily basis I can see the excitement in their eyes. When I tell most people I see the "I have absolutely no idea what you're talking about" look or politely hidden boredom immediately triggered by any hint of "computer stuff" - "um I work in software" ah OK.

Not just any software though - Internet software services. Cloud computing applications and social media listening to be more specific.

The social listening application is called Social Radar and we're currently testing Social Radar 3. There's no doubt Social Radar is one of the most advanced web applications available today. It's certainly best-in-class. It was built for a very, very specific purpose so it won't get the general public excited but companies are extremely excited.

The value behind Social Radar is very straight forward. It is the value of listening - which infers understanding. Companies care, for very specific reasons, what their customers and potential customers think and talk about. What are their favorite products? What features do they like best? What features do they want? If they were going to build the product for themselves, what would they include or change?

We've all heard of the stereotypical "water cooler conversation" (and some of us even still have them!) and companies have always wanted to be a "fly on the wall" during these conversations so they could better understand their customers and better serve their needs. Today many of these customer conversations happen online and companies can, and do, listen. As they should.

It is extremely important to clarify the difference between "listening" and "scraping" as they can be confused in general conversation. "Listening" focuses on conversations which are intended by the author or community to be public - a blog is a good example. If a person is passionate enough to share their thoughts and feelings about the products and services that impact their lives they should also expect that their effort does not fall on deaf ears. "Scraping" is shady bullshit conducted by individuals and companies with no moral compass who decide it's OK to take advantage of outdated legal issues and unsuspecting communities to make a quick buck. These companies use manipulative tactics and malicious software to steal customer information intended to be private. Usually this information is gathered as quickly as possible for as long as possible and quietly sold to the highest bidder. If your email or Facebook account has ever been hacked and a message sent to all your contacts then you've seen how quickly scrapping can collect private information. "Scrapping" is immoral and should also be illegal.

The latest version of Social Radar is the culmination of 4 years of work from Infegy, a Kansas City based company. Infegy has many competitors but none as fully dedicated to the intellectual integrity of social media listening. Infegy has the enormous strategic advantage of being completely self-funded and profitable. We actually like hearing about our competitors taking tens of millions of dollars in venture capital, they're no longer running their own show and we know while we'll probably hear more about them in the news, we'll actually see them less when working with partner companies on individual projects. When competitive firms are acquired we often no longer see them at all. Being 100% independent has allowed Infegy to remain 100% focused on what Infegy does best, listening and the technology required to deliver a state of the art listening application.

Social Radar's latest release takes advantage of several technologies in some very cool ways. Social Radar is actually the world's largest implementation of Sphinx, an open source search server, by several multiples over other large implementations like Craigslist. Social Radar takes advantage of other open source solutions as well. The 50 or so custom machines powering the primary system are running Ubuntu Server as well as MySQL Database. Like Google, Social Radar is a crawler based system and has an army of these specialized computer programs gobbling up social content faster than you can say, "I love my iPhone" or "I loved King's Speech" or "when I hold my phone a certain way I drop the signal". The crawlers are custom built in Python, upgraded from C++. The application itself is also written in Python. Natural Language Processing (NLP) and other analytical capabilities utilize custom built learning algorithms. The front-end is built 100% on top of the API (XML or JSON) and renders completely in JavaScript (sorry Internet Explorer). The system is distributed across dozens of custom built machines and can literally analyze tens of millions of online conversations in a few seconds. Turns out mathematics is useful after high school! The Reporting Engine on this application would make the IT and business teams in any enterprise drool. Just think of a virtual whiteboard and with a couple of clicks being able to pull any data and date range with any number of filters onto the board and then pick them up and move them around or change them all together. So awesome.

To say the engineering knowledge and skill required to create this system were over my head would be a massive understatement but it has been a beautiful event to witness. When it comes to web applications you haven't seen anything yet!