Toluu : TNT — The Network Thinker

TNT — The Network Thinker

This week the Department of Homeland Security received the results from a study by the National Research Council that the DHS funded. The study's conclusion: data mining for terrorists does not work and it invades the privacy of innocent citizens. Searching for the terrorist needle in the haystack of phone-calls and emails is counter-productive.

I told you so... here, here, here, here, and here.

However, if you start with a known suspect or two, you can roll up the rest of the network with normal surveillance techniques. Rather than dig through millions of records looking for that unknown terrorist pattern, give the phone company the numbers of known suspects/terroists and have them return the 2-step network neighborhood of each number -- now you can see the terror suspect's extended social graph. This approach with social network analysis also uncloaks street gangs and criminal n

Non-Obvious TiesSeptember 19

How many NOTs in your network? [NOT = Non-Obvious Tie]

You probably can't answer that, because the connections are...
n o n - o b v i o u s .

Ties/links/connections/relationships that are not obvious to me may be obvious to someone else, and vice versa. Unfortunately, the knowledge of those ties may not be as valuable to those those who know, than to those who do not know.

Confused? Let me share a few stories...

This afternoon I was looking through the access logs to my business web site. I saw some interesting NOTs. Two *.mil orgnaizations were visiting certain pages. Would my friends at Booz-Allen-Hamilton would find that information of great value? BAH sells products and services to DoD clients. The BAH rainmakers would love to know what military branch X and Y are interested in and what search terms they used, and what pages they spent a long time on.

Then I noticed visitors from the Dark Web [web sites that support/idolize Jihad]. I wondered... wouldn't the military folks, that visited just an hour before, like to see what the jihadi supporters were look

Political Book Buying PatternsSeptember 11

I'm surprised it took Amazon this long to exploit their own political book buying data. The Amazon maps go to where the power is -- state by state -- the Electoral College.

The Amazon maps indicate whether each state's residents purchased a larger number of red or blue books. Comparing my recent book network maps to the above pictured book volume map, seems to show that while the Left read a larger variety of books, in most states the Right buys greater quantities of a smaller set of books.

Some interesting questions remain...

are book buyers influential with voters in their social network?
how many voters does a typical political book reader reach/influence?
are Amazon book buyers representative of the political book buying population?

Amazon clusters the political book sales data into 60 day slices. When you go back earlier in the year 2008, you see more blue or neutral shaded states. The most recent 60 day slice of 2008 [before and after the conventions] shows a blossoming of deep red. If polemics predict [no one knows for

Weekend Data MiningSeptember 7

During the Republican National Convention, John McCain's acceptance speech indicated that he was running against the current state of affairs in Washington DC. Will he be different than the current administration of George Bush? Is McCain a Republican of a different stripe? Will a McCain administration be different than a Bush administration? With only opinions and no data, we can argue all day and all night.

Maybe some data can help us see behind the rhetoric? One way to gain insight into possible future behavior is to look at who is donating to the campaign and hoping to influence a new administration.

I downloaded data of the top bundlers of donations for the 2000 and 2004 Bush campaigns and the 2008 McCain campaign. What's the overlap of donors between the Bush and McCain campaigns? Will the same people influence both campaigns/administrations? Or will it be starkly different groups? Or something in between?

Below is a map of those who donated to BOTH Bush and McCain. The campaigns are shown as the two red nodes on the left of the map. The green links show donations coming into the McCain 2008 campaign. The blue lines show donations coming into the Bush campaigns of 2000 and 2004. The 128 bundlers, who have contributed to both McCain and Bush, are shown in the arc on the right.

It's the Network, StupidAugust 27

It is amazing how many of our current problems come down to the realization that it's the network, the connectivity, that matters. In most situations we know how to fix and enhance the nodes in the network. The links, and their patterns and structure, are the hard problem.

We are making progress in alternative energy production, but we still fail at energy distribution. Windmills and solar energy collectors have made great progress -- we just can't get the energy from where the wind blows and the sun shines to where the great population centers are. To do that requires a well-designed power distribution grid. Many critics of the current grid describe it as "third world" in design, quality and capability. Today's New York Times describes the distribution problem well.

Above is a network map of a portion of the US electric grid. Life is great if you live in one of the densely connected clusters using electricity generated nearby. Things start to get real complicated if energy needs to transferred from one cluster to another cluster in grid. Distance destroys. Electricity does not flow like info