Video: Introduction about Netflix and Recommender systems

eMarketer - Netflix: Personalized Product Recommendations

    This is a youtube clip that gives you a simple introduction about how Netflix uses the collaborative filtering recommender system to improve their business. It conveys some simple ideas and is worth a look. 

Clinical Recommender system

    This is very interesting that the recommender systems can be used in the health care industry. here's the article, Healthcare information systems: data mining methods in the creation of a clinical recommender system. 


    We usually think that the recommender systems are used consumers good such as movies, music, books, and so forth. But right now, recommender systems can be blended in the healthcare information technology. 
    
    For example illustrated in the article, the nursing care plan can use the recommender system to provide clinical decision support, nursing education, clinical quality control, and to be a complement for existing practice guidelines. 

    In the study, they used nursing diagnosis data to develop a methodology, producing a ranked list of suggested care plan based on the information they put in.  

Enterprise Information Systems, May2011, Vol. 5 Issue 2, p169-181, 13p, 2 Diagrams, 3 Charts, 5 Graphs Chart; found on p170

    it's very interesting to see the recommender systems combined with different industry and provide technical assistance to healthcare. It looks like there will be a lot of different applications that recommender systems will implement in our life.

Content-based filtering

    According to Francesco, the author of Recommender System Handbook, content-based filtering is using the technique to analyze a set of documents and descriptions of items previously rated by a user, and then build a profile or model of the users interests based on the features of those rated items. Using the profile, the recommender system can filter out the suggestions that would fit for the user.


     Go deep into the process, there are three steps of the recommendation process.

• Content analyzer: the main responsibility of the process is to represent the content of items. it can extract the information or specific features from the item by feature extraction techniques. 
• Profile learner: this process will collect data representative of the users preferences and try to generalize the data, then construct the user profile. 
• Filtering components: this process will try to match the features of the user profile with the features of the items. And then, the system will recommend items that fit for the user. 

Here is a flow chart of the process captured from Recommender System Handbook.

it tell you how the system processes the information source and user profiles, so that they can recommend items based on the process.

So, compared to collaborative filtering, there are some advantages and drawbacks of content-based filtering that we should understand. 

Advantages

• User independence: collaborative filtering needs other users' rating to find the similarity between the users and then give the suggestion. Instead, content-based method only have to analyze the items and user profile for recommendation.
• Transparency: collaborative method gives you the recommendation because some unknown users have the same taste like you, but content-based method can tell you  they recommend you the items based on what features. 
• No cold start: opposite to collaborative filtering, new items can be suggested before being rated by a substantial number of users. 

Disadvantages

• Limited content analysis: if the content does not contain enough information to discriminate the items precisely, the recommendation will be not precisely at the end.
• Over-specialization: content-based method provides a limit degree of novelty, since it has to match up the features of profile and items. A totally perfect content-based filtering may suggest nothing "surprised." 
• New user: when there's not enough information to build a solid profile for a user, the recommendation could not be provided correctly. 

There are different merits and drawbacks either for collaborative filtering or for content-based filtering. So most of the websites they start to use the hybrid system to combine the advantages of those two method and try to give their customers an easier and more valuable recommendations.

"Recommend" some recommender system based websites

Here are some interesting websites which use recommender systems.

WhatShouldIReadNext.com

You can use the website to help you determine the next book to read. Just add the book title that you enjoyed recently, and the site will give you the recommendation.

Last.fm

This is a radio station that you can listen music online. It uses a recommender system to track what you've listened and suggests music based on your taste.

StumbleUpon

This is a website which can recommend you the websites you may be interested in. It gives you the suggestions based on your ratings of websites.

Netflix

Netflix was a DVD rental delivery and online stream website which uses the recommender system to suggest its users movies or TV series based on their rating data.

Pandora

This is also a music radio website. Users enter their favorite songs and artists into Pandora, then Pandora recommends music similar to your taste. Then you can listen to the new music through the site.

Here is a source for some popular recommender system websites. Check it out.

http://www.deitel.com/ResourceCenters/Web20/RecommenderSystems/PopularRecommenderSystems/tabid/1312/Default.aspx

Collaborative filtering

    There are two type of approaches that are used by the recommender systems. Collaborative filtering and content-based filtering. Today, I'll introduce the collaborative filtering approach here.

    Collaborative filtering methods are based on collecting and analyzing a large amount of information on users' behaviors, activities or preferences and predicting what users will like based on their similarity to other users. 

    If a person A has the same opinion as a person B on an issue, A is more likely to have B's opinion on a different issue x than to have the opinion on x of a person chosen randomly. So as you can see in the above picture, this is the simple illustration about collaborative filtering. 

    So, besides Netflix and Amazon which mentioned in the previous posts, many other websites are using collaborative filtering for recommender system such as Last.fm, a music radio website; Facebook, MySpace, LinkedIn, these network website recommends new connections via collaborative filtering (CF). 

    For websites who want use CF system, they have to be aware of several problems.

• Data sparsity: Cold start problem. Briefly, a brand new product need to be rated by a substantial amount of users before it could be recommended. The product won't be limited if the system is using the content-based approach, which we'll see it in the next post. 
• Scalability: the computing power need to be very strong if the websites have a great amount of products and customers. 
• Synonyms: the same or very similar items which have different names would not be recognize the same within the CF systems.
• Grey sheep: users who are not consistent with their like and dislike. This may cause the CF fails to recommend items.
• Shilling attack: people may give their own items a lot of good rating and bad ratings to their competitors. 

    To avoid these problems, some of the companies prefer to use the content-based approach, and some of them use the "Hybrid" system! We'll see these approach in the next post.

Netflix Prize

    This is a video describing the how the AT&T Labs researchers improved the collaborative filtering algorithm and won the Netflix Prize.

    So, what is Netflix Prize? In short, it's a competition held by Netflix who offered $100 millions to the winner who can improve Netflix algorithm system by 10%. And, as you can see in the video, team Bellkor, which consisted of three researchers from AT&T Labs, won the final prize with the 10% improvement of the original algorithm. 

    One interesting thing brought by the video is the "Napoleon Dynamite", representing some type of movies that have very polarizing rating. This kind of movies is usually controversial, and receive very extreme ratings from viewers. it's really hard to predict rates for these movies.

Why use the Recommender Systems

    As we discuss in the previous topic, we now understand that the Recommender Systems (RSs) is a way to deal with the overload information/products that customers may encounter. And right now, we are going to dig into the reasons why it's a trend that companies tend to use RSs.

• Increase the number of items sold.
• Sell more diverse items.
• Increase the user satisfaction.
• Increase users' fidelity
• Better understand what the user wants.

    It's always their objective for companies to increase their revenues. For those companies which have a great deal of products, RSs are the way to increase their revenues.

    First, increase the number of items sold. For example, if you want to buy a new boots on Amazon, you may first search for a specific brand that you like. But you didn't see any style that is cool enough, so you didn't buy any pairs. When you are about to give up, you'll see that Amazon would recommend you the items that are similar to the items you viewed. you may be able to find something that is fascinating from it's recommendation. And the following will be that you click the botton and buy the shoes, while Amazon successfully use the RSs to increase its revenues.

    Second, sell more diverse items. Here's another example from Amazon. See the cropped picture below. When I look at the macbook air product page, you'll see there's  kind of a bundle suggested by Amazon. it'll suggest you the items that other customers would buy together. Although it's not like a bundle promotion which will offer you some discount (the price is just the sum of the three product...), it still increases customers' incentive to buy them together.

    

    Third, for sure that the customers satisfaction and fidelity will increase. The RSs can reduce the time that customers spending on browsing back and forth in the different categories. It can help the customers to find the stuffs they want in a relatively short time. In addition, the longer the user interact with the websites/RSs, the more effective and accurate the recommendation will be. And, at the end, the cycling of the increasing usage and the increasing benefit will reach satisfaction and fidelity. 

    Finally, understand what the user wants. The data collected by the systems is not only doing good for the customers, but also helping the companies to understand what the user wants. It's a valuable data. it can help the companies to adjust many things like the inventory, the product categories, and even the business strategies. When you understand more from your customers, you are able to exploit more from them. 

    Okay, so the above are several reasons why companies tend to use the Recommender Systems. In the near future, I'll introduce some of the techniques that are used in the RSs. Stay tuned.