Recommender System Development Practice: Basic Usage of the Surprise Library

First, build the dataset

Here I randomly generate users and items, as well as users' ratings of items, like (1) and dislike (0)

import random
import pandas
import numpy as np

def build_dataframe() -> pandas.DataFrame:
    datasets = {
        'userID': [],
        'itemID': [],
        'rating': []
    }
    # 生成所有用户
    users = [user for user in np.arange(1, 1000)]
    # 从数据库取出所有文章
    items = [item for item in np.arange(1, 10000)]
    # 模拟评分
    for user in users:
        # 选n篇用户看过的文章
        viewed_items = random.sample(items, 100)
        for item in viewed_items:
            rating = random.randint(0, 1)
            datasets['userID'].append(user)
            datasets['itemID'].append(item)
            datasets['rating'].append(rating)

    return pandas.DataFrame(datasets)

Prepare to train the model

Automatic cross-validation

The Surprise library has a series of built-in functions and datasets for you to use. It is so powerful that you only need to simply write a few lines of code to use cross-validation.

To load a dataset from a pandas DataFrame, you need to use the load_frim_df() method, and also use the Reader class, but only the rating_scale parameter must be specified. The DataFrame must have three columns, corresponding to the user (raw) ID, the item (raw) ID, and the rating in this order. Therefore, each row corresponds to a given rating. Because the columns of the DataFrame can be easily reordered.

Here I use a randomly generated dataset of my own; the results are quite poor, so it is better to use the MovieLens dataset

from surprise import NormalPredictor,SVD,KNNBasic
from surprise import Dataset
from surprise import Reader
from surprise.model_selection import cross_validate

print('构建数据集')
df = build_dataframe()

# 同样需要定义 Reader 对象，但只需指定 rating_scale 参数。
reader = Reader(rating_scale=(0, 1))

# 传入的列必须对应着 userID，itemID 和 rating（严格按此顺序）。
data = Dataset.load_from_df(df[['userID', 'itemID', 'rating']], reader)

# 使用MovieLens-100k数据集
data = Dataset.load_builtin('ml-100k')

# 然后即可根据需要来操作此数据集，例如调用cross_validate
print('交叉验证')
cross_validate(KNNBasic(), data, measures=['RMSE', 'MAE'], cv=5, verbose=True)

Run results

构建数据集
交叉验证
Computing the msd similarity matrix...
Done computing similarity matrix.
Computing the msd similarity matrix...
Done computing similarity matrix.
Computing the msd similarity matrix...
Done computing similarity matrix.
Computing the msd similarity matrix...
Done computing similarity matrix.
Computing the msd similarity matrix...
Done computing similarity matrix.
Evaluating RMSE, MAE of algorithm KNNBasic on 5 split(s).

Fold 1  Fold 2  Fold 3  Fold 4  Fold 5  Mean    Std     
RMSE (testset)    0.9861  0.9848  0.9806  0.9759  0.9639  0.9783  0.0080  
MAE (testset)     0.7755  0.7777  0.7767  0.7726  0.7614  0.7728  0.0059  
Fit time          0.59    0.67    0.61    0.58    0.62    0.61    0.03    
Test time         3.59    3.78    3.74    3.48    3.46    3.61    0.13    

{'test_rmse': array([0.98610945, 0.98482529, 0.98062869, 0.97590553, 0.96393148]),
'test_mae': array([0.7754694 , 0.77774512, 0.77666541, 0.77260546, 0.76140967]),
'fit_time': (0.5875513553619385,
0.6657052040100098,
0.614863395690918,
0.5803155899047852,
0.6159660816192627),
'test_time': (3.592573404312134,
3.7826573848724365,
3.736438751220703,
3.481112241744995,
3.4585647583007812)}

Train-test split and the fitting function fit()

If you do not want to use cross-validation, you can also use train_test_split() to split the test set and training set according to a size you specify, and use a selected accuracy metric. The fit() function will use the algorithm on the training set, and the test() function returns the predictions on the test set

from surprise import SVD
from surprise import Reader
from surprise import Dataset
from surprise import accuracy
from surprise.model_selection import train_test_split

df = build_dataframe()
reader = Reader(rating_scale=(0, 1))
data = Dataset.load_from_df(df[['userID', 'itemID', 'rating']], reader)

# 随机抽取训练集和测试集，测试集占总数据集的25%
trainset, testset = train_test_split(data, test_size=.25)

# 将使用SVD算法
algo = SVD()

# 用训练集训练模型，用测试集来得到评级结果
algo.fit(trainset)
predictions = algo.test(testset)

# 计算均方根误差RMSE
accuracy.rmse(predictions)

Run results

RMSE: 0.5205
0.5204505667249893

You can also use a single line of code to train and test the algorithm:

predictions = algo.fit(trainset).test(testset)
accuracy.rmse(predictions)

Run results

RMSE: 0.5194
0.5194365387027055

Training the entire dataset and the predict() prediction function

Obviously, we can also simply fit the algorithm to the entire dataset. Use the build_full_trainset() method to do this, then predict accuracy by directly calling the predict() method. For example, use “user id:196, item id:302, like or not:1” to verify (make sure the training set contains this).

from surprise import KNNBasic
from surprise import Dataset

df = build_dataframe()
reader = Reader(rating_scale=(0, 1))
data = Dataset.load_from_df(df[['userID', 'itemID', 'rating']], reader)

# 取回训练集
trainset = data.build_full_trainset()

# 构建算法并训练
algo = KNNBasic()
algo.fit(trainset)

uid = 1
iid = 1

# 获取指定用户和电影的评级结果
pred = algo.predict(uid, iid, r_ui=1, verbose=True)

Run results

Computing the msd similarity matrix...
Done computing similarity matrix.
user: 1          item: 1          r_ui = 0.00   est = 0.65   {'actual_k': 6, 'was_impossible': False}

Note: predict() uses raw IDs (the difference between raw IDs and inner IDs will be explained in detail later). If the dataset used is read from a file, then the raw IDs are strings (even if they represent numbers). However, I am using a dataframe here, so I don't need to use it this way.

Using cross-validation iterators

To better evaluate the model, we can also implement cross-validation iterators (p-times k-fold cross-validation). For each iterator produced by the split() method, use the test() method to make predictions.

from surprise import SVD
from surprise import Dataset
from surprise import accuracy
from surprise.model_selection import KFold

df = build_dataframe()
reader = Reader(rating_scale=(0, 1))
data = Dataset.load_from_df(df[['userID', 'itemID', 'rating']], reader)

# 定义几次k折交叉验证迭代
kf = KFold(n_splits=3)

algo = SVD()

for trainset, testset in kf.split(data):
    # 训练和测试算法
    algo.fit(trainset)
    predictions = algo.test(testset)

    # 计算和打印均方根误差
    accuracy.rmse(predictions, verbose=True)

Run results

RMSE: 0.5188
RMSE: 0.5192
RMSE: 0.5200

Other cross-validation methods such as LeaveOneOut (leave-one-out) and ShuffleSplit (shuffle-and-split, which randomly “shuffles” the sample set and then splits it into a training set and a test set) can also be used. Surprise's cross-validation tools are designed with inspiration from the excellent scikit-learn API.

The case where the training set and the test set are predefined as separate files is a special case of cross-validation. For example, the movielens-100K dataset already provides 5 training files and 5 test files (u1.base,u1.test,…,u5.base,u5.test). You can use the surprise.mode_selection.split.PredefinedKFold object to handle this case:

import os
from surprise import SVD
from surprise import Dataset
from surprise import Reader
from surprise import accuracy
from surprise.model_selection import PredefinedKFold

# 数据集文件夹位置
files_dir = os.path.expanduser('~/.surprise_data/ml-100k/ml-100k/')

# 这次，使用内置函数Reader读取数据
reader = Reader('ml-100k')

# 定义folds_files为包含文件路径的元组列表：
# [(u1.base, u1.test), (u2.base, u2.test),...(u5.base, u5.test)]
train_file = files_dir + 'u%d.base'
test_file = files_dir + 'u%d.test'
folds_files = [(train_file % i, test_file % i) for i in (1, 2, 3, 4, 5)]

# 分别载入数据集
data = Dataset.load_from_folds(folds_files, reader=reader)
pkf = PredefinedKFold()

algo = SVD()

for trainset, testset in pkf.split(data):
    # 训练和测试
    algo.fit(trainset)
    predictions = algo.test(testset)

    # 计算和打印均方根误差
    accuracy.rmse(predictions, verbose=True)

Run results

RMSE: 0.9509
RMSE: 0.9374
RMSE: 0.9333
RMSE: 0.9330
RMSE: 0.9323

Of course, you can also load only a single file for training and load a single file for testing. However, the folds_files parameter still needs to be in the form of a list.

Tuning parameters with GridSearchCV

The cross_validate() function outputs accuracy metrics of the cross-validation process for a given set of parameters. If you want to know which parameter combination produces the best results, you can use the GridSearchCV class. Given a dictionary that has some parameters, this class will exhaustively try all parameter combinations and report the best parameters for any accuracy measure (averaged over different splits).

The following is an example of tuning the SVD algorithm's parameters n_epochs, lr_all, and reg_all:

from surprise import SVD
from surprise import Dataset
from surprise.model_selection import GridSearchCV

# 自动构建数据集
df = build_dataframe()
reader = Reader(rating_scale=(0, 1))
data = Dataset.load_from_df(df[['userID', 'itemID', 'rating']], reader)

# 指定参数选择范围
param_grid = {'n_epochs': [5, 10], 'lr_all': [0.002, 0.005],
              'reg_all': [0.4, 0.6]}

gs = GridSearchCV(SVD, param_grid, measures=['rmse', 'mae'], cv=3)

gs.fit(data)

# 打印最好的均方根误差RMSE
print(gs.best_score['rmse'])

# 打印取得最好RMSE的参数集合
print(gs.best_params['rmse'])

# 现在可以使用产生最佳RMSE的算法
algo = gs.best_estimator['rmse']
algo.fit(data.build_full_trainset())

Running results

0.5035192355809749
{'n_epochs': 5, 'lr_all': 0.005, 'reg_all': 0.6}
<surprise.prediction_algorithms.matrix_factorization.SVD at 0x23985147a08>

Once fit() is called, the best_estimator attribute provides us with an algorithm instance that has the best parameter set. Here, 3-fold cross-validation is used to evaluate the average RMSE and MAE; any cross-validation iterator can also be used.

Note: Dictionary parameters (such as bsl_options and sim_options) require special handling. See the usage example below:

param_grid = {'k': [10, 20],
              'sim_options': {'name': ['msd', 'cosine'],
                              'min_support': [1, 5],
                              'user_based': [False]}
              }

Of course, the two can be combined, for example in the KNNBaseline algorithm:

param_grid = {'bsl_options': {'method': ['als', 'sgd'],
                              'reg': [1, 2]},
              'k': [2, 3],
              'sim_options': {'name': ['msd', 'cosine'],
                              'min_support': [1, 5],
                              'user_based': [False]}
              }

For further analysis, all information in the cv_results attribute can be exported as a pandas DataFrame and saved as a .csv file:

results_df = pd.DataFrame.from_dict(gs.cv_results)
results_df.to_csv('cv_result.csv')