A lot of ideas, but neither theoretical enough nor pratical enough 2010-07-20

Reviewer: G. Webster

This book contains a lot of ideas and as such is a good starting point for further reading. But it's not a one-stop resource for actually implementing the algorithms it mentions, as a lot of them are described only in a very high level and incomplete way. For example, in the discussion of model-based recommendation engines in sections 12.3.3-12.3.5, the author gives a very short description of latent semantic indexing (LSI) and some Java code that shows how to use the Weka implementation. But firstly, the description is too short to give the user a real understanding of what is going on theoretically. And secondly, the implementation description doesn't go nearly far enough: it shows that reconstructing the original matrix from the top N dimensions of the singular value decomposition gives a close approximation to the original, but then it just stops there; it doesn't explain how to actually use the decomposition in a recommendation engine. And the section on LSI is verbose compared to the "section" on Bayesian belief networks, which at a single paragraph of text is completely inadequate for either practical or theoretical purposes. And so on throughout the book.

It's a must read.. 2010-05-19

Reviewer: Ankesh Kumar

I'm a start-up CEO, who's had 3 of my engineers review this book. Unanimously, they came back raving about how much they picked up from the book and hence how much time they saved. If you manage any technical resources and are interested in this area, buy a copy for each of your developers, it will save you and your team a lot of time and effort.

Fascinating book about how Web 2.0 sites work. 2010-01-17

Reviewer: L. King

To really understand this book one would probably have to be a Java programmer, which I'm not, but I was able to follow the argumentation. I do have some background with data mining using SAS and SQL and the mathematics described are fairly easy to understand for someone with even a 1st year engineering or applied math background. I also have an interest in linguistics which kept me going.

The basic idea is that one can catalog documents by removing irrelevant words (adjectives, abstract pronouns, conjunctives) and "stemming" the remaining words (ie: reducing "sews", "sewing", "resew", "sewer" to a root "sew") and creating a vector containing each root word and the word frequency and then normalizing it. One simple result is the ability to produce "word clouds". Similarity between documents is measured by taking the dot product of the two vectors. Any document compared to itself would have a dot product of 1. Two documents with no common stem words would have a dot product of zero. Similar docs would have a high value close to 1, say .8. Dissimilar docs would have a low coefficient, say .15. Even mistaking "sewer" (a conduit for waste) and sewer (one who uses a needle and thread) is taken into account because both docs would only be similar on a couple of keywords, and dissimilar on most others.

What's really neat is how this information gets collected and can be applied. Social networking sites, including the one you are reading right now, Amazon.com, collect data on us through our choices. Browse for a book while logged on then that's something you are interested in. Approve a review the words in the review, summary of the book and the title counts towards your interests. Disapprove and that counts against your interests. Write a review and the words you write become part of your cumulative profile as well, reduced to a vector or vectors of keywords and frequencies.

Here's how it gets applied: One of Amazon's marketing tools is it's "recommendation engine". (The book talks about Netflix recommendation engine and business model). By matching your vector against other people who have bought/viewed what you have bought a prediction can be made as to the likelihood of you being interested in the something that they have bought, or not interested in items that they rejected or disliked. The more Amazon caters to what you are interested in, and doesn't bother you with irrelevancies, the happier you may be.

Other applications discussed include the automatic creation of folksonomies (taxonomies based on popular usage) using cluster analysis and categorization using Bayes theorem.

In addition to recommendation engines Alag points out the usefulness of these techniques to Search and points out several search engines that apply this approach (as does Google), tools that search out and provide news based on your preferences, or suggest "friends" (ie: Facebook or eHarmony might use these ideas), search for similar material to identify copyright infringement, email filters that keep out spam for rolex watches or viagra (unless you are interested in rolex watches or viagra), construct a virus detection engine based on code phrases or early detection of epidemics or adverse reactions to medication through similarities in medical reports. Alag himself appears to be working at a biotech firm NextBio that matches public medical and genome related data to data held by private companies.

Some of the basic tools discussed are Lucene, a free version of what Google will sell you for a search engine, Nutch, a free web crawler, both of which require coding and WEKA, a free open source data mining package that looks usable by the rest of us.

Loved the book and the author's organization of the material. Some of the social implications are scary, especially for privacy concerns, but so is the implication of not leveraging the information that one holds within your organization to provide the best possible service. For example the World Bank has the capability (not necessarily using these methods) to match similar projects around the world so that experience gained in one area can be found and applied elsewhere. This is a key fast moving tech that one needs to understand in order to see where we are going as a society. C.I. in Action is merely the opening salvo - the methods and techniques described are the basics but there is much room for refinement and elaboration and this topic could be the start of a whole new field. The book also recommends and has sparked my interest in the site [...] which is probably more accessible to someone without a math or tech background.

Finally a note to SF fans, esp. of Spider Robinson's Callahan's Crosstime Saloon series, this may be the point at which the Web starts to appear to be intelligent. :-)

Collective Intelligence in Action 2009-11-11

Reviewer: Eric Jain

This book is more deserving of the "Collective Intelligence" title than O'Reilly's "Programming Collective Intelligence" as it's not just about algorithms, but discusses blogs, wikis etc, and shows how to do basic implementations of features such as tag clouds or finding related content in that context. Instead of explaining specific algorithms in detail, existing Java libraries are used, e.g. WEKA for data mining and Lucene for search.

There are lots of diagrams, and (somewhat verbose) Java code. The examples in this book are good starting points for further exploration; this book is more about showing what can be done and getting you started in the right direction than providing you with an understanding of the algorithms (as does the O'Reilly book) and libraries that are used.

Good , well written and good overview of available tools 2009-10-08

Reviewer: Vasko Yordanov

The best thing about the book is that it revealed many other open source toolkits relevant to the topic. Provided good topic background.The only remark is that the code examples can be more organised and structured so that you can get the bundle and start experimenting , rather than tie up the code snippets together.