Comments on: Would you recommend your recommender? http://www.meanboyfriend.com/overdue_ideas/2009/10/would-you-recommend-your-recommender/ Ideas linking Libraries, Computing, E-learning, and anything else that springs to mind. Wed, 28 Jul 2010 15:12:58 +0000 hourly 1 http://wordpress.org/?v=3.0 By: Andre Vellino http://www.meanboyfriend.com/overdue_ideas/2009/10/would-you-recommend-your-recommender/comment-page-1/#comment-368 Andre Vellino Fri, 23 Oct 2009 00:38:12 +0000 http://www.meanboyfriend.com/overdue_ideas/?p=543#comment-368 Yes - evaluating the quality of recommendations in a DL recommender is a tricky proposition. One question is - how "good" (or should it be "useful") is your usage data (i.e. who are your users?). The quality of your user's knowledge ultimately gets reflected in the recommendations that are generated by the algorithm. Another question is "how good is the recommendation algorithm"? That's a much easier question to answer, and there are well known methods for comparing algorithms given a test set (see the criteria for the Netflix competition.) A "middle way" is to use citation data as a substitute for usage data. At CISTI we developed an experimental recommender service that uses (primarily) citation data (but also usage data, except that there isn't much of it) for a small (1.5M article) Biomed collection. We ask the users to evaluate the quality of the recommendations they get as a result. You can try it out here: http://lab.cisti-icist.nrc-cnrc.gc.ca/synthese/welcome.jsp There's more about this project here: http://lab.cisti-icist.nrc-cnrc.gc.ca/cistilabswiki/index.php/Synthese_Recommender Yes – evaluating the quality of recommendations in a DL recommender is a tricky proposition. One question is – how “good” (or should it be “useful”) is your usage data (i.e. who are your users?). The quality of your user’s knowledge ultimately gets reflected in the recommendations that are generated by the algorithm.

Another question is “how good is the recommendation algorithm”? That’s a much easier question to answer, and there are well known methods for comparing algorithms given a test set (see the criteria for the Netflix competition.)

A “middle way” is to use citation data as a substitute for usage data. At CISTI we developed an experimental recommender service that uses (primarily) citation data (but also usage data, except that there isn’t much of it) for a small (1.5M article) Biomed collection. We ask the users to evaluate the quality of the recommendations they get as a result. You can try it out here:

http://lab.cisti-icist.nrc-cnrc.gc.ca/synthese/welcome.jsp

There’s more about this project here:

http://lab.cisti-icist.nrc-cnrc.gc.ca/cistilabswiki/index.php/Synthese_Recommender

]]>