Collaborative filtering

    There are two type of approaches that are used by the recommender systems. Collaborative filtering and content-based filtering. Today, I'll introduce the collaborative filtering approach here.

    Collaborative filtering methods are based on collecting and analyzing a large amount of information on users' behaviors, activities or preferences and predicting what users will like based on their similarity to other users. 

    If a person A has the same opinion as a person B on an issue, A is more likely to have B's opinion on a different issue x than to have the opinion on x of a person chosen randomly. So as you can see in the above picture, this is the simple illustration about collaborative filtering. 

    So, besides Netflix and Amazon which mentioned in the previous posts, many other websites are using collaborative filtering for recommender system such as Last.fm, a music radio website; Facebook, MySpace, LinkedIn, these network website recommends new connections via collaborative filtering (CF). 

    For websites who want use CF system, they have to be aware of several problems.

• Data sparsity: Cold start problem. Briefly, a brand new product need to be rated by a substantial amount of users before it could be recommended. The product won't be limited if the system is using the content-based approach, which we'll see it in the next post. 
• Scalability: the computing power need to be very strong if the websites have a great amount of products and customers. 
• Synonyms: the same or very similar items which have different names would not be recognize the same within the CF systems.
• Grey sheep: users who are not consistent with their like and dislike. This may cause the CF fails to recommend items.
• Shilling attack: people may give their own items a lot of good rating and bad ratings to their competitors. 

    To avoid these problems, some of the companies prefer to use the content-based approach, and some of them use the "Hybrid" system! We'll see these approach in the next post.

Netflix Prize

    This is a video describing the how the AT&T Labs researchers improved the collaborative filtering algorithm and won the Netflix Prize.

    So, what is Netflix Prize? In short, it's a competition held by Netflix who offered $100 millions to the winner who can improve Netflix algorithm system by 10%. And, as you can see in the video, team Bellkor, which consisted of three researchers from AT&T Labs, won the final prize with the 10% improvement of the original algorithm. 

    One interesting thing brought by the video is the "Napoleon Dynamite", representing some type of movies that have very polarizing rating. This kind of movies is usually controversial, and receive very extreme ratings from viewers. it's really hard to predict rates for these movies.

Why use the Recommender Systems

    As we discuss in the previous topic, we now understand that the Recommender Systems (RSs) is a way to deal with the overload information/products that customers may encounter. And right now, we are going to dig into the reasons why it's a trend that companies tend to use RSs.

• Increase the number of items sold.
• Sell more diverse items.
• Increase the user satisfaction.
• Increase users' fidelity
• Better understand what the user wants.

    It's always their objective for companies to increase their revenues. For those companies which have a great deal of products, RSs are the way to increase their revenues.

    First, increase the number of items sold. For example, if you want to buy a new boots on Amazon, you may first search for a specific brand that you like. But you didn't see any style that is cool enough, so you didn't buy any pairs. When you are about to give up, you'll see that Amazon would recommend you the items that are similar to the items you viewed. you may be able to find something that is fascinating from it's recommendation. And the following will be that you click the botton and buy the shoes, while Amazon successfully use the RSs to increase its revenues.

    Second, sell more diverse items. Here's another example from Amazon. See the cropped picture below. When I look at the macbook air product page, you'll see there's  kind of a bundle suggested by Amazon. it'll suggest you the items that other customers would buy together. Although it's not like a bundle promotion which will offer you some discount (the price is just the sum of the three product...), it still increases customers' incentive to buy them together.

    

    Third, for sure that the customers satisfaction and fidelity will increase. The RSs can reduce the time that customers spending on browsing back and forth in the different categories. It can help the customers to find the stuffs they want in a relatively short time. In addition, the longer the user interact with the websites/RSs, the more effective and accurate the recommendation will be. And, at the end, the cycling of the increasing usage and the increasing benefit will reach satisfaction and fidelity. 

    Finally, understand what the user wants. The data collected by the systems is not only doing good for the customers, but also helping the companies to understand what the user wants. It's a valuable data. it can help the companies to adjust many things like the inventory, the product categories, and even the business strategies. When you understand more from your customers, you are able to exploit more from them. 

    Okay, so the above are several reasons why companies tend to use the Recommender Systems. In the near future, I'll introduce some of the techniques that are used in the RSs. Stay tuned. 

    

What is Recommender Systems?

    Have you ever been to websites like Amazon, Netflix, or Pandora Radio? If so, you must have known the Recommender Systems, or at least you've used it unawarely.

    So, what is Recommender Systems(RSs) exactly about? The picture below is an example for you.

    As you can see, this is a cropped picture from my Amazon home page. it shows me the item that I've viewed before, and then, it recommends me several items which are viewed by customers who also viewed the same product as I viewed. 

According to Francesco Ricci, the author of Recommender Systems Handbook, RSs are software tools and techniques providing suggestions for items to be of use to a user.The suggestions relate to various decision-making processes, such as what items to buy, what music to listen to, or what online news to read.

    So, why does Amazon do this? Why do they spend money to build a system to recommend you something else?

    A simplest answer would be that they want to make money. They want to maximize the possibility that you'll see what you like on their websites, and click the "add to cart" button. Because most of the time, these kind of websites would have a great number of products, sometimes, customers would feel lost because there are too many categories, items, choices that they don't know how to find what they want. RSs is a way to simplify the choices for customers, bringing the suggestions that would be needed by the customers.   

    It's obvious that more and more e-commerce companies are inclined to use RSs to help their business. And in the next topics, we are going to talk about stuffs like how RSs impact businesses and exploring the techniques that used by RSs.