Recommender Systems in E-learning
Correspondence Address: Prof. Jie Lu, Australian Artificial Intelligence Institute, Faulty of Engineering and Information Technology, University of Technology Sydney, NSW 2007, Australia. E-mail:
In this era when every aspect of society is accelerating, people are always seeking improvement to stay competitive in their careers. E-learning systems fit into the ever challenging situation and provide learners with remote learning opportunities and abundant learning resources. Facing with the numerous resources online, users need support in deciding which course to take, thus recommender systems are applied in E-learning to provide learners with personalized services by automatically identifying their preferences. This position paper systematically discusses the main recommendation techniques employed in in E-learning and identifies new research directions. Three main recommendation techniques are reviewed in this paper: content-based, collaborative filtering-based and knowledge-based recommendations. The basic mechanism of these technique together with how they are used to fulfill the specific requirements in the context of E-learning are highlighted and presented. The observations in this paper could support researchers and practitioners to better understand the current development and future directions of recommender systems in E-learning.
E-learning, also known as online learning, is expanding in the market during the past few years and has attracted great attentions due to the lockdowns caused by COVID-19. E-learning has become an important way to acquire knowledge not only for students in schools and universities but also for life-long learners who seek improvement in their social life and workplace. E-learning provides online courses and learning resources under various categories such as computer science, mathematics and business. For example, Coursera1 provides users with courses, degrees, programs and certificates from top universities such as Stanford and Harvard and companies such as Google and IBM. Faced with various courses and resources, users need to figure out which course to take to help develop their own career and keep competitive in the global market. Therefore, it is crucial to develop recommender systems to support users in choosing courses, resources, or learning materials in E-learning.
The development of Web 2.0 and information technology brings a large amount of information to users, which makes them feel overwhelmed and adds difficulty to decision-making. To solve the information overload problem, recommender systems are developed as a solution which aims to provide personalized services to suitable users at the right time. There are two important components in recommender systems: (1) the users to whom the system provides services; and (2) the items which are products or services to be recommended to the users such as movies, books, or a mobile plan. Usually, a recommender system helps a user select from a large number of items by providing a list of item candidates to reduce the time and efforts of users. Nowadays, recommender systems are widely used in many real-world applications and have become an essential part of web portals.
Depending on the items, recommender systems need different techniques, strategies, and system designs to fit the context of each application. Recommender systems in E-learning share similarities with famous recommender systems in e-commerce such as the one used by Amazon, as they still fall into four common categories: content-based, CF-based, knowledge-based, and hybrid. Users in E-learning recommender systems remain passive and the recommendation mostly relies on the content of the items and the user behaviors collected when they are using the online system. However, these mature e-commerce recommender systems cannot be directly applied to the E-learning area. The users in the E-learning recommender systems are students who are completing their degree in universities or life-long learners who are looking for skills that are helpful to their career. The items in the E-learning recommender systems can be learning styles, learning materials, learning activities, courses, or even learning routes. Compared with classical e-commerce recommendation, recommender systems have unique characteristics in the context of E-learning:
The requirements of learners and content of learning activities contain uncertain and vague information. For instance, the learners may not know which skills or courses they need, but they know what kind of job they are looking for. One learning activity can fall into different categories.
The context of learning is important to learners such as the purpose of taking the course and the learning style. The recommendations for full-time learners and fragmentary time learners should be different.
The learning courses, activities, and materials need to be arranged in order to ensure that the prerequisites of some courses are met. A simple example is that, if the course “System Design” needs the prerequisite course “Database”, then it would be unsuitable to recommend the course “System Design” if the learner has not yet taken the course “Database”.
The need of learners is an adaptive learning route that helps them to gain knowledge continuously. For the life-long learners, what they need is not only one course, but a learning route that contains a package of courses, activities, and materials in an adequate order.
The above characteristics prompted the development of recommendation techniques and strategies for the E-learning context, especially for supporting life-long learners. The functions of the E-learning recommender systems are various, including predicting learner performance, exercise difficulty ranking, learning course/material recommendation, and learning route planning. In this paper, we aim to review and analyze the research on E-learning recommender systems, identify the main techniques, and discuss future directions.
The rest of this paper is as follows. Section 2 introduces the main techniques in recommender systems. Section 3 reviews and summaries the current recommendation models, methods, and applications in E-learning. Section 4 present the challenges and future directions of E-learning. Section 5 concludes our paper.
2. Main techniques in E-learning recommender systems
Recommendation techniques are extensively studied and mainly classified into three categories: content-based, collaborative filtering (CF)-based, and knowledge-based. The three categories all have their own characteristics and suitable application scenarios. Content-based recommendation methods profile users with their past behaviors and recommend items that are similar to user profiles. It is suitable for recommending items with abundant content information such as documents or webpages. CF-based recommendation methods assume that similar users will prefer similar items and generate recommendations with the collaboration of users. It is widely used in many scenarios and well-known through the Netflix competition. Knowledge-based recommendation uses explicit knowledge of users/items and business rules to profile user interest and provide recommendations. It is applied in some complex scenarios where items are not rated or not frequently purchased, such as cars, houses, and policies. The hybrid of the above three techniques is widely applied in many real-world recommender systems as one technique draws on the merits of the others to offset its own weaknesses. In this section, we briefly introduce the basic mechanisms of these techniques with emphasis on the methods regularly used in E-learning recommender systems.
2.1 Content-based recommendation techniques
As the name suggests, the mechanism of content-based techniques is to recommend items whose contents are similar to the ones that learners like according to their historical records. The process of content-based recommendation is: (1) the item profile is established according to the contents of items. Recall that items in E-learning recommender systems can be learning materials, learning objects, learning courses, and learning activities. Content-based recommendation techniques vary depend on the specific items. The items in E-learning such as learning materials and learning objects are usually in unstructured text data and represented by words and weights by vector space model and term frequency - inverse document frequency (TF-IDF). Moreover, items such as learning courses and learning activities are structured data and represented by attributes including categories, prerequisites, lecturers, etc. These attributes may contain complex relationships and uncertainties, which can be represented by attribute vectors or hierarchical trees; (2) the learner profile is built with his/her historical record and usually a combination of the item profiles that he/she consumed in the past. In addition, some meta-data of learners such as learner’s educational background, targeted skills, and their requirements are also added to the profile; and (3) the recommendation is generated by similarity calculation and matching between learner and item profiles. In this section, we describe three types of content-based recommendation techniques: semantic-based, attribute-based, and query-based.
2.1.1 Semantic-based recommendation
Items in E-learning are mostly in text form. We firstly introduce how to extract semantic representations and build item profiles. Traditional recommendation methods profile items with simple representations by bag-of-words and calculate the similarity by cosine or Jaccard. These methods have the advantages of easy deployment, quick output, and practicality, but they fail to consider the word sequence relationship. The development of the word embedding techniques enables the words and text-structured items to be represented by latent vectors. The latent representations of items can be flexibly integrated into an end-to-end neural network with binary classification according to user’s likes/dislikes. For recommending learning materials in history courses, the representations of words/concepts in are generated by skip-gram model as well as by considering the history encyclopedia. The recommendation is then made by calculating cosine similarity. A convolutional neural network (CNN) is used in to process the text information in learning resources, which is able to recommend new learning resources to learners on MOOC platform. If we aim to implement real-time semantic-based recommendation, the recommendation method needs to have the ability to handle streaming data. Currently, studies on real-time E-learning recommender systems are lacking, but real-time E-learning recommendation is highly demanded, which forms a promising direction. The details are given in Section 4.
2.1.2 Attribute-based recommendation
Although the nature of the items in E-learning are text-form, learning resources and learning activities contain structured attributes, such as category attributes (e.g., business, computer science, and mathematics), lecture attributes (e.g., credits and course duration), and prerequisite attributes. These attributes can be represented as linear vectors similar to semantic-based recommendation techniques[21,22]. The relationships between attributes are sometimes complex and form a hierarchical tree structure. Since the learner profile is built with the his/her historical records, profiling both items and learners requires handling tree-structured data. The learner profile is built as a learner tree to model the learner’s dynamic preferences on multiple attributes. Similarity calculation and matching algorithms are developed with the tree-structured learner and item profiles. Another issue that E-learning recommender systems need to deal with is the uncertainty in the item attributes and linguistic terms in learner requirements. With a learner profile built by a multi-criteria student requirement analysis model and a item profile built with attributes of learning materials, a fuzzy matching method is developed to find the most similar learning materials that are suitable for learners. Learner and item profile are built as fuzzy requirement tree and fuzzy category tree in and the content- and CF-based hybrid recommendation method is developed with fuzzy tree similarity measure.
2.1.3 Query-based recommendation
Usually, E-learning recommender systems allow users to provide a query and search for their interested items. The search query is composed of keywords and treated as user profile by the system. The similarities between the user profile and the item profiles in the system are calculated, and then an item ranking list with similarities from high to low is generated for the user to choose. If the query information is used and user personal profiles are not taken into consideration, then the recommendation is not personalized. If the query information is taken from users in real-time, the recommendation for the users are real-time recommendations.
However, the query sent by users have other advantages: (1) the query sent by users can reflect the current interests of the users, which may not be reflected by user’s historical records; (2) the recommendation can be made by a combination of query and user profile established from the historical records[27,28]. Sometimes the query inputs of learners are not accurate since learners may not have the domain knowledge to express what they would like to learn in the new learning area. Therefore, it increases the difficulty to match appropriate learning material and generate useful recommendation. Query refinement is to help users to generate a more accurate keyword that can best describe the user needs[29,30]. In, the system first extracts concepts from learning materials with the help of the encyclopedia and then matches the query to the most similar concepts. The generated concept terms together with the initial query are used to match the learning material and generate the recommendation. The E-learning recommender systems not only allow learners to send one-time queries but also allow them to interact with the system through a chat bot to catch the changed preferences. A learner emotion-aware recommendation method is proposed that allows learns to type in a chat-room page and detect whether the learner has negative emotions or is suffering depression. If the learner has been recognized as feeling negative, then an emotion case base regulation is activated to recommend some strategies to the learner.
2.2 Collaborative filtering-based recommendation techniques
The CF-based recommendation techniques are well-received by the research community. The user preference data used by CF recommendation methods contain explicit ratings (e.g., scores 1-5) and implicit feedback (e.g., viewing and clicking) on items. In E-learning recommender systems, these feedbacks are also collected on learning courses, resources, and materials. When generating recommendations for a target user, the basic mechanism of CF-based recommendation is to find similar users and use them collaboratively to find items that are unconsumed and interesting to the target user. The initial development of CF-based recommendation methods is memory-based, where similarities between users and items are heuristically found. With the development of machine learning and artificial intelligence, model-based CF methods are developed for rating prediction and item ranking. The CF-based recommendation in E-learning is the same as in other application areas, if we only take explicit and implicit feedbacks into consideration. What makes the E-learning recommender system special is that learners’ choices are greatly affected by context. In this section, we introduce two types of CF-based recommendation techniques in E-learning: context-aware and deep CF-based recommendation.
2.2.1 Context-aware recommendation
Context is defined as the situation that affects the interaction between the user and the system, which can be other users, location, or atmosphere. The context information of a learner can be his/her prior knowledge, learning goal, learning style, available learning time, learning progress, location, device, social relations, and emotions.
To take advantages of the context information, many E-learning recommender systems are designed to be able to collect, analyze, and integrate this information with learners’ preference prediction. The learners are clustered by geographical information, device (mobile or desktop), and their learning progress, which are taken into account in similarity calculation. Emotion is also considered as context information in, as the learners are monitored with heart rate, skin conductance, blood pressure, and brainwave to detect their emotion evolution during learning. The experiments prove that the awareness of learner’s emotion by the teachers and the system can help recommend more precise resources and improve the learner’s performance. Learner’s multiple browsing interactions such as click, bookmark, view, etc. are used to profile learners’ preferences. Trust and skills evaluation are used for class formation and help learners to build social relationships. Social relations between learners are modeled by detecting community structure and the learner’s preferences are influenced in regard to his/her closeness to other learners in this community. E-learning recommender systems are designed to allow learners to share questions and assignments, add tags, and use the social interaction and knowledge discussions between students and instructors to enhance the performance of recommendations.
2.2.2 Deep CF-based recommendation
Recently, the development of deep learning has seen its successful application in CF-based recommender systems, which makes it natural to be applied in E-learning recommender systems. CNNs have excellent performance detecting emotions from facial expressions, which shows their potential to be applied in the E-learning recommender systems. Recurrent neural network is suitable for sequential interactions between learners and items as well as for handling the pedagogical relationships between items. For example, to recommend adequate learning problems to learners in the correct pedagogical sequence (e.g., from basic to advanced), the learning path is extracted by recurrent neural network (RNN) from a trial-and-error problem solving process of similar learners who belong to the same cluster. To recommend learning resources, the learning path is built with knowledge points from learning sources. Feed-forward neural network is used for learner ability assessment and then integrated into a recommendation framework with the nearest neighborhood CF algorithm. Deep learning techniques are an essential part of the development of smart E-learning recommender systems.
2.3 Knowledge-based recommendation techniques
The knowledge-based recommendation techniques are applied to scenarios when user ratings are not sufficient and item contents contain complex domain knowledge. They are very suitable to be applied in E-learning recommender systems because of the complex pedagogical relationship between items in E-learning. The advantage of knowledge-based recommendation is that it does not have cold-start or data sparsity problem, but it relies heavily on the pre-defined knowledge structures such as ontology. Ontology is a formal knowledge representation that contains concepts, entities, and their relationships in a specific domain, which is also encoded by ontology language such as web ontology language (OWL). In this section, ontology-based recommendation techniques and applications are introduced.
Ontology-based recommendation methods use ontological terms and ontological inferences to enhance the leaner and item profiling. The ontology-based recommendation method used in E-learning varies depending on how to build the ontology and the knowledge base. A learning object is defined as one learning activity that lasts less than 30 min with one knowledge point. Sequence rules are defined on the parent–child relationship with learning object nodes in the trees and used for recommendation in. These rules are simple, thus enabling the system to provide fast response, but it cannot assert new learning objects and inference relationships. To make the recommender system more practical, user queries are modeled and brought as new concepts if previously unavailable in the ontology by ontology enrichment. Besides queries, learner’s knowledge requirement for the course can also be modeled by their posted questions and answers and integrated into the ontology-based recommendation. Except for the item ontology, learner ontology is also built with their demographic information, learning style, and knowledge level. Context information is also captured by ontology, and OWL rules are used to filter and recommend learning materials[58,59]. The concepts can be linguistic terms and relationships between concepts can be uncertain, thus fuzzy ontology is proposed to deal with the uncertainties in the knowledge representation. Fuzzy ontology is used for leaner profiling, using fuzzy weights as representations that describe the relationships between concepts and the learner. Ontological compatibility degree is calculated between the resource and the user and the most similar resource is recommended.
Instead of using a pre-defined ontology, the relationship of items or learners can be formalized according to self-organization theory. The relationship between items (in, the items are LOs) are modeled with learning object attribute similarities, learner’s preference on LOs, learning object’s state, and learning object’s quality. Similarly, the learner relationships are calculated considering three aspects: profile similarity, knowledge credibility similarity, and learner follower aggregation. Fuzzy logic is applied to more accurately model the learning requirements, learner stage, and learning ability of each learner. With the learner relationships and profiles, learners are classified into cliques and then the sequential pattern mining method is used to find the most similar learners and extract their visited learning objecs and form a recommendation. A multidimensional knowledge graph is designed as a substitute of ontology for representation learning of LOs with six relationships indicating six kinds of learner needs.
2.4 Hybrid recommendation techniques
Although we review the papers in the above three categories, real-world E-learning recommender systems usually combine the best features of multiple recommendation methods into a hybrid method to deliver higher performance and overcome drawbacks of each method. According to the authors of, there are seven basic hybridization mechanisms of different recommendation method combinations: weighted, mixed, switching, feature combination, feature augmentation, cascade, and meta-level. For example, one classical combination is content- and CF-based recommendation method, which is also applied in an E-learning recommender system. CF-based techniques suffer from the cold-start problem but have better scalability and serendipity. Except for the serendipity problem, content-based recommendation is limited to the availability of content information but can contribute to the new user problem, which is the main disadvantage of CF-based recommendation methods. Another specific hybrid is context-aware recommendation and sequential pattern mining in E-learning recommender systems[66,67]. The context-aware part is to integrate the information of learners such as their learning environment and knowledge levels, while the sequential pattern mining (SPM) is to discover the pattern in a learner’s learning path and react to the potential learner’s change in interest.
3. Comprehensive analysis of E-learning recommender systems
This section discusses and summarizes recommender systems in E-learning reviewed above and presents a general framework containing two typical scenarios of recommender systems in E-learning.
3.1 Comprehensive analysis
The details of each reviewed recommender system, including its category, sub-category, specific techniques applied, functions, and periods of use, are summarized in Table 1. We firstly summarize the following functions mainly targeted by recommender systems in E-learning.
Summary of E-learning recommender systems developed, the method category, sub-category, and specific techniques applied and their functions
|Reference||Method category||Sub-category||Specific Technique||Function||Period|
|||Content-based||Semantic-based||Skip-gram model||Learning material recommendation||2015|
|||Content-based||Semantic-based||Convolutional neural network||Learning material recommendation||2017|
|||Content-based||Attribute-based||Heuristic similarity||Learning material recommendation||2017|
|||Content-based||Attribute-based||Heuristic similarity||Learning material recommendation||2012|
|||Content-based||Attribute-based||Tree matching||Learning material recommendation||2013|
|||Content-based||Attribute-based||Fuzzy matching||Learning material recommendation||2004|
|||Content-based||Attribute-based||Fuzzy tree matching||Learning activity recommendation||2015|
|||Content-based||Query-based||Graph modeling||Query recommendation||2008|
|||Content-based||Query-based||Multi-layer graph modeling||Learning material recommendation||2013|
|||Content-based||Query-based||Multi-layer graph modeling||Learning material recommendation||2018|
|||Content-based||Query-based||Query-refinement||Learning material recommendation||2018|
|||Content-based||Query-based||Chat bot||Learning material recommendation||2018|
|||Content-based||Query-based||Random forest||Learning encouragement||2014|
|||CF-based||Context-aware||Nearest neighbor||Questions difficulty ranking||2014|
|||CF-based||Context-aware||IoT technology||Academic performance evaluation||2008|
|||CF-based||Context-aware||Heuristic similarity||Learning object recommendation||2012|
|||CF-based||Context-aware||Fuzzy rule-based||Student performance evaluation||2019|
|||CF-based||Context-aware||Classification||Learning material recommendation||2009|
|||CF-based||Context-aware||Heuristic similarity||Learning material recommendation||2010|
|||CF-based||Context-aware||Social network||Learning activity recommendation||2017|
|||CF-based||Context-aware||Graph data mining||Learning material recommendation||2008|
|||CF-based||Context-aware||Nearest neighbor||Friend recommendation||2005|
|||CF-based||Context-aware||Tag-based||Learning material recommendation||2018|
|||CF-based||Deep learning||Convolutional neural network||Teaching strategy recommendation||2017|
|||CF-based||Deep learning||RNN||Learning path recommendation||2018|
|||CF-based||Deep learning||RNN||Learning material recommendation||2017|
|||CF-based||Deep learning||RNN||Learning activity recommendation||2018|
|||Knowledge-based||Ontology-based||Sequence rule||Learning object recommendation||2009|
|||Knowledge-based||Ontology-based||Semantic web||Learning object recommendation||2015|
|||Knowledge-based||Ontology-based||Association rule||Knowledge requirement acquisition||2009|
|||Knowledge-based||Ontology-based||Heuristic similarity||Learning material recommendation||2017|
|||Knowledge-based||Ontology-based||Semantic web||Learning material recommendation||2017|
|||Knowledge-based||Ontology-based||Case-based reasoning||Learning object recommendation||2009|
|||Knowledge-based||Ontology-based||Fuzzy ontology||Learning object recommendation||2012|
|||Knowledge-based||Ontology-based||Self-organization (SO)||Learning object recommendation||2005|
|||Knowledge-based||Ontology-based||SO, influence model||Learning object recommendation||2019|
|||Knowledge-based||Ontology-based||Jnowledge graph||Learning path recommendation||2018|
|||hybrid||-||CF and content-based||Learning object recommendation||2018|
|||Hybrid||-||Context-aware and SPM||Learning material recommendation||2018|
|||Hybrid||-||Context-aware and SPM||Learning material recommendation||2018|
Learning material recommendation: Adequate learning materials are needed by different users according to their knowledge level and current requirements. These learning material are mainly in text and suitable for handling by content-based techniques.
Learning object recommendation: A definition of “learning object” is given in[
54] as one learning activity that lasts less than 30 min with one knowledge point. Different from learning materials, learning objects refer to different items with the smallest granularity, such as exercises, questions, and examples[ 62]. Learning object recommendation is mostly achieved by knowledge-based techniques.
Learning activity recommendation: Learning activity usually has broader scopes and can refer to a class or an event involving students. This kind of recommendation is usually achieved by content-based, CF-based, or hybrid methods.
Learning path recommendation: A learning path contains a sequence of items which can be learning materials, learning objects, or learning activities. Obviously, this function is more difficult and sequence modeling and RNN are usually applied due to their natural advantages in dealing with sequential data.
Others: There are other functions provided by recommender systems in E-learning, such as exercise difficulty ranking, knowledge requirement acquisition, student emotion detection,encouragement, etc.
3.2 E-learning recommender system framework
We propose a framework of E-learning recommender system. As shown in Figure 1, an E-learning recommender system contains three major components: a user interface, a database server, and a recommendation engine. They are described below.
User interface: It is responsible for generating the requested webpages and handling the user interface logic and events for the learners.
Database server: The database stores all the data of the system, which include mainly the learner database and the item database. The learning database contains the information and behaviors of each user, which will be used for conducting learning profile. The item database contains the information of each learning material/object/activity.
Recommendation engine: The recommendation engine is the core part of the recommender system. It implements the proposed recommendation method and generates recommendations for learners. First, data from various sources need to be processed by data extract, transform, and load (ETL). After data are processed, the engine trains the recommendation methods. The recommendation engine component contains the learner profiling module and item profiling module and implements the recommendation methods. Recommendation results are usually generated as a ranked list of items and are presented to users to help them make decisions. The recommendation methods take those input data to classify, measure, and match users or items; predict a score indicating preferences; and generate recommendation lists. These modules vary depending on the techniques reviewed in
Section 2. For example, if the recommendation technique is content-based, learner and item profiles are established with semantic, query, or attribute information. If the recommendation technique is CF-based, the learner and item profiles contain historical ratings.
Since the data of learners are collected and stored at the database server, it raises user privacy issues that have gained more attention recently. It has been shown that users will have better experience if they trust the system with no concerns of their data privacy. If the privacy issues are considered, the E-learning recommender system frameworks should have the mechanism to assure users that their data are safe. One possible way is to design a federated E-learning recommender system framework that stores user data locally on their own devices and only shares encrypted parameters and gradients with the recommendation server. With this kind of framework, the data of learners are protected from malicious users and even from the recommendation provider. As a result, learners will be more willing to use the recommender system and more willing to be participate in online courses.
Additionally, we present two typical scenarios in E-learning recommender system. The first scenario is a learner who is looking for a course. He/she may know the skills they are targeting. For example, learners aim to gain skills in software engineering. The course recommended may be “Python programming” or “Database fundamentals”. The second scenario is for learners who are already enrolled in a course but are looking for learning materials that help themto advance in the knowledge in this course. For example, learners who are enrolled in “Deep learning” may be interested in learning materials of “Graph neural network”. Different data about learners/items are used by the E-learning recommender system, depending on the recommendation scenario.
4. Recommender systems in E-learning: future directions
E-learning has gained popularity due to the COVID-19 pandemic, which also urges its development to deal with challenges, fulfill new requirements of learners, and provide them better user experience. In this section, we discuss future directions of recommender systems in E-learning.
4.1 Adaptive recommender system in E-learning
In the big data era, learners are suffering from keeping up with the development of science and technology. It is very important for life-long learners to acquire the most cutting-edge skills in the market, which is dynamically changing. Item attributes and item popularity may drift suddenly or gradually and new subjects and skills may emerge. This phenomenon is also known as concept drift. For example, previously, the core course in the area of Artificial Intelligence was “Support vector machine”. With the technology development, the core courses are changing to “Deep neural network”. Thus, adaptive and drift-aware recommender systems in E-learning need to help learners find what benefits them most by adaptively profiling new and existing items and providing real-time recommendations.
On the other hand, the knowledge acquisition of learners is a step-by-step process, so that the interests and requirements of learners are evolving gradually after the learner has read a learning material, participated a learning activity, and finished a course. Existing recommender systems in E-learning usually try to take advantages of the context information and make prediction to provide in-time recommendations, but they cannot adaptively model and respond to the change of the learner profile and provide real-time recommendations. A recent work develops an interactive recommender system by multi-armed bandit to help users improve skills with progressive questions. Adaptive recommender systems enhanced by reinforcement learning techniques are a promising direction, as they interact with learners to track and monitor learner interests and maximize the engagement and satisfaction of learners in the long learning process.
4.2 Privacy-preserved recommender system in E-learning
Big data has brought computational and bandwidth constraints which prevents a centralized recommender system to efficiently and effectively provide services to users. One solution is to build distributed recommender systems that are constituted by multiple service agents or end users to improve the scalability and performance with ever expanding data. When sharing data between these agents/users, privacy is a major concern. Federated recommender systems are proposed as an entirely new and very promising research area because privacy protection is one of the fundamental principles of this paradigm. Instead of using cryptography, which often brings more costs in computation and communication, differential privacy is a strict mathematical definition to protect privacy by data randomization and perturbation. Differential privacy is applied to some typical recommendation techniques, such as matrix factorization on explicit feedback and implicit feedback. Despite the above works, how to protect privacy in E-learning applications should be considered and a general E-learning recommender system framework is needed.
4.3 Explainable recommender system in E-learning
One of the main disadvantages of recommender systems is that they are black boxes, which lack transparency. In the learning process, learners need to make plans and keep themselves motivated to reach their learning goals. If recommender systems cannot provide explanation for learners on how the recommended learning materials/learning courses can help them with skill acquisition, it is easy for learners to lose interest and lose trust in the system. Providing explanations to learners helps them understand the complex recommendation process using visualization or interactive strategies. Initial research has been conducted on whether providing visual and textual explanations can influence learners on achieving their learning goals. Developing explainable recommender systems in E-learning is a potential research direction worth more investigation.
4.4 Scalable recommender system in E-learning
Despite the privacy problem, big data has also brought the the problem of implementing scalable recommender systems in E-learning for a large number of students. Hadoop, Mahout, and Spark are tools that are suitable for building recommender systems for big data, and they are also suitable to be applied in the E-learning area. Recently, a distributed E-learning recommender system is built based on association rule mining techniques with Spark and Hadoop. This system is deployed in High School of Technology of Fez to help students find suitable courses. It has the ability to manipulate big data by massive parallel processing. However, the recommendation method used in this research is relatively simple. There will be a large demand for developing distributed recommender systems with more complex recommendation methods such as CF- and deep learning-based methods for E-learning platforms.
Recommender systems have become an essential part in today’s online services in many application areas. In this position paper, we introduce the specific characteristics which make the E-learning recommender system distinguishable from other areas, elaborate the main techniques used in developing E-learning recommender systems, and discuss challenges and future directions for E-learning recommender systems. Each technique has its advantages and limitations. Compared with content-based methods, CF is easier to implement and takes advantage of collective intelligence, but it suffers from the cold-start or data sparsity problem. It relies heavily on the learner’s feedback or behavior collection. Content-based methods, however, are able to take real-time queries from users but lack serendipity, since they seldom take others’ opinion into consideration. Knowledge-based methods are particularly useful when the ontology is established and does not change, but they rely on domain knowledge, are not easy to implement, and are difficult to adapt. Hybrid recommender systems are more popular than single recommendation method-based systems for avoiding the drawbacks of individual recommendation methods. This paper highlights how recommendation techniques can support learners in universities and life-long learners to gain skills to stay competitive. It aims to provide guidance for researchers and practitioners in developing E-learning recommender systems.
Methodology, Writing- Original draft preparation: Zhang Q
Idea discussion, Supervision, Funding acquisition: Zhang G
Idea discussion, Paper structure design, Supervision, Writing, Reviewing and Editing, Funding acquisition: Lu JAvailability of data and materials
Not applicable.Financial support and sponsorship
This work was supported by the Australian Research Council (ARC) under the Australian Laureate Fellowship (FL190100149).Conflicts of interest
All authors declared that there are no conflicts of interest.Ethical approval and consent to participate
Not applicable.Consent for publication
© The Author(s) 2021.
1. Salehi M, Kamalabadi IN. Hybrid recommendation approach for learning material based on sequential pattern of the accessed material and the learner’s preference tree. Knowledge Based Systems 2013;48:57-69.
2. Castro F, Vellido A, Nebot A, Mugica F. Applying data mining techniques to e-learning problems. In: Evolution of teaching and learning paradigms in intelligent environment. Springer; 2007. pp. 183-221.
3. Ricci F, Rokach L, Shapira B, Kantor PB. Recommender systems handbook 1st ed. Berlin, Heidelberg: Springer-Verlag; 2010.
4. Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 2005;17:734-49.
5. Lu J, Wu D, Mao M, Wang W, Zhang G. Recommender system application developments: A survey. Decision Support Systems 2015;74:12-32.
6. Drachsler H, Hummel HG, Koper R. Personal recommender systems for learners in lifelong learning networks: The requirements, techniques and model. International Journal of Learning Technology 2008;3:404-23.
7. Thai-Nghe N, Drumond L, Krohn-Grimberghe A, Schmidt-Thieme L. Recommender system for predicting student performance. Procedia Computer Science 2010;1:2811-19.
8. Segal A, Gal K, Shani G, Shapira B. A difficulty ranking approach to personalization in e-learning. International Journal of Human Computer Studies 2019;130:261-72.
9. Aher SB, Lobo L. Combination of machine learning algorithms for recommendation of courses in E-Learning System based on historical data. Knowledge Based Systems 2013;51:1-14.
10. Dwivedi P, Kant V, Bharadwaj KK. Learning path recommendation based on modified variable length genetic algorithm. Education and Information Technologies 2018;23:819-36.
11. Bennett J, Lanning S. The netflix prize. In: Proceedings of KDD Cup and Workshop 2007. p. 35.
12. Lu J, Zhang Gq, Zhang Q. Recommender Systems: Advanced Developments World Scientific; 2020.
13. Burke R. Hybrid recommender systems: survey and experiments. User Modeling and User Adapted Interaction 2002;12:331-70.
14. Lops P, Jannach D, Musto C, Bogers T, Koolen M. Trends in content-based recommendation. User Modeling and User Adapted Interaction 2019;29:239-49.
15. Wang D, Liang Y, Xu D, Feng X, Guan R. A content-based recommender system for computer science publications. Knowledge Based Systems 2018;157:1-9.
16. Pazzani MJ, Billsus D. Content-based recommendation systems. In: The adaptive web; 2007:325-41.
17. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems 2013;26:3111-19.
18. Chen T, Hong L, Shi Y, Sun Y. Joint text embedding for personalized content-based recommendation. arXiv preprint arXiv:170601084 2017.
19. Ye M, Tang Z, Xu J, Jin L. Recommender system for e-learning based on semantic relatedness of concepts. Information 2015;6:443-53.
20. Shu J, Shen X, Liu H, Yi B, Zhang Z. A content-based recommendation algorithm for learning resources. Multimedia Systems 2018;24:163-73.
21. Salehi M, Kmalabadi IN. A hybrid attribute–based recommender system for e-learning material recommendation. Ieri Procedia 2012;2:565-70.
22. Hariadi AI, Nurjanah D. Hybrid attribute and personality based recommender system for book recommendation. In: 2017 International Conference on Data and Software Engineering (ICoDSE); 2017:1-5.
23. Salehi M. Application of implicit and explicit attribute based collaborative filtering and BIDE for learning resource recommendation. Data & Knowledge Engineering 2013;87:130-45.
24. Lu J. A personalized e-learning material recommender system. In: International Conference on Information Technology and Applications 2004.
25. Wu D, Lu J, Zhang G. A fuzzy tree matching-based personalized e-learning recommender system. IEEE Transactions on Fuzzy Systems 2015;23:2412-26.
26. Li L, Yang Z, Liu L, Kitsuregawa M. Query-URL bipartite based approach to personalized query recommendation. In: AAAI 2008;8:1189-94.
27. Meng F, Gao D, Li W, Sun X, Hou Y. A unified graph model for personalized query-oriented reference paper recommendation. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management; 2013:1509-12.
28. Cai X, Han J, Li W, Zhang R, Pan S, et al. A three-layered mutually reinforced model for personalized citation recommendation. IEEE Transactions on Neural Networks and Learning Systems 2018;29:6026-37.
29. Su JH, Hong TP, Li JY, Su JJ. Personalized content-based music retrieval by user-filtering and query-refinement. In: 2018 Conference on Technologies and Applications of Artificial Intelligence (TAAI); 2018:177-80.
30. Baizal ZA, Widyantoro DH, Maulidevi NU. Query refinement in recommender system based on product functional requirements. In: 2016 International Conference on Advanced Computer Science and Information Systems (ICACSIS); 2016:309-14.
31. Mbipom B, Massie S, Craw S. An e-learning recommender that helps learners find the right materials. In: The 32nd AAAI Conference on Artificial Intelligence; 2018:7928-33.
32. Souali K, Rahmaoui O, Ouzzif M. Introducing a traceability based recommendation approach using chatbot for e-learning platforms. In: International Conference on Advanced Intelligent Systems for Sustainable Development; 2018:346-57.
33. Tian F, Gao P, Li L, Zhang W, et al. Recognizing and regulating e-learners’ emotions based on interactive Chinese texts in e-learning systems. Knowledge Based Systems 2014;55:148-64.
34. Koren Y, Bell R, Volinsky C. Matrix factorization techniques for recommender systems. Computer 2009;42:30-37.
35. Bobadilla J, Serradilla F, Hernando A, et al. Collaborative filtering adapted to recommender systems of e-learning. Knowledge Based Systems 2009;22:261-65.
36. Bobadilla J, Ortega F, Hernando A, Gutiérrez A. Recommender systems survey. Knowledge Based Systems 2013;46:109-32.
37. Shani G, Shapira B. Edurank: A collaborative filtering approach to personalization in e-learning. Educational data mining 2014:68-75.
38. Dey AK, Abowd GD, Salber D. A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Human Computer Interaction 2001;16:97-166.
39. Amasha MA, Areed MF, Alkhalaf S, Abougalala RA, Elatawy SM, et al. The future of using Internet of Things (IoTs) and context-aware technology in e-learning. In: Proceedings of the 2020 9th International Conference on Educational and Information Technology; 2020:114-23.
40. Gallego D, Barra E, Aguirre S, Huecas G. A model for generating proactive context-aware recommendations in e-learning systems. In: 2012 Frontiers in Education Conference Proceedings; 2012:1-6.
41. Moore P, Zhao Z, Van Pham H. Towards cloud-based personalised student-centric context-aware e-learning pedagogic systems. In: Conference on Complex, Intelligent, and Software Intensive Systems; 2019:331-42.
42. Shen L, Wang M, Shen R. Affective e-learning: Using “emotional” data to improve learning in pervasive learning environment. Journal of Educational Technology & Society 2009;12:176-89.
43. Takano K, Li KF. An adaptive e-learning recommender based on user’s web-browsing behavior. In: 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing; 2010:123-31.
44. De Meo P, Messina F, Rosaci D, Sarné GM. Combining trust and skills evaluation to form e-Learning classes in online social networks. Information Sciences 2017;405:107-22.
45. Hu J, Zhang W. Community collaborative filtering for e-Learning. In: 2008 International Conference on Computer and Electrical Engineering; 2008:593-97.
46. Rafaeli S, Dan-Gur Y, Barak M. Social recommender systems: Recommendations in support of e-learning. International Journal of Distance Education Technologies (IJDET) 2005;3:30-47.
47. Klašnja-Milićević A, Vesin B, Ivanović M. Social tagging strategy for enhancing e-learning experience. Computers & Education 2018;118:166-81.
48. Zhang S, Yao L, Sun A, Tay Y. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR) 2019;52:1-38.
49. Sun A, Li YJ, Huang YM, Li Q. Using facial expression to detect emotion in e-learning system: A deep learning method. In: International Symposium on Emerging Technologies for Education; 2017:446-55.
50. Zhou Y, Huang C, Hu Q, Zhu J, Tang Y. Personalized learning full-path recommendation model based on LSTM neural networks. Information Sciences 2018;444:135-52.
51. Wang X, Zhang Y, Yu S, Liu X, Yuan Y, et al. E-learning recommendation framework based on deep learning. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC) IEEE; 2017. pp. 455-60.
52. Chanaa A, El Faddouli Ne. Deep learning for a smart e-learning system. In: Proceedings of the 2nd International Conference on Smart Digital Environment; 2018:197-202.
53. Tarus JK, Niu Z, Mustafa G. Knowledge-based recommendation: A review of ontology-based recommender systems for e-learning. Artificial intelligence review 2018;50:21-48.
54. Shen Lp, Shen Rm. Ontology-based learning content recommendation. International Journal of Continuing Engineering Education and Life Long Learning 2005;15:308-17.
55. Fraihat S, Shambour Q. A framework of semantic recommender system for e-learning. Journal of Software 2015;10:317-30.
56. Zeng Q, Zhao Z, Liang Y. Course ontology-based user’s knowledge requirement acquisition from behaviors within e-learning systems. Computers & Education 2009;53:809-18.
57. Tarus JK, Niu Z, Yousif A. A hybrid knowledge-based recommender system for e-learning based on ontology and sequential pattern mining. Future Generation Computer Systems 2017;72:37-48.
58. Bouihi B, Bahaj M. An ontology-based architecture for context recommendation system in e-learning and mobile-learning applications. In: 2017 International Conference on Electrical and Information Technologies (ICEIT) IEEE; 2017. pp. 1-6.
59. Chu HC, Chen TY, Lin CJ, Liao MJ, Chen YM. Development of an adaptive learning case recommendation approach for problem-based e-learning on mathematics teaching for students with mild disabilities. Expert Systems with Applications 2009;36:5456-68.
60. Ferreira-Satler M, Romero FP, Menendez-Dominguez VH, Zapata A, Prieto ME. Fuzzy ontologies-based user profiles applied to enhance e-learning activities. Soft Computing 2012;16:1129-41.
61. Wan S, Niu Z. An e-learning recommendation approach based on the self-organization of learning resource. Knowledge Based Systems 2018;160:71-87.
62. Wan S, Niu Z. A hybrid e-learning recommendation approach based on learners’ influence propagation. IEEE Transactions on Knowledge and Data Engineering 2019;32:827-40.
63. Shi D, Wang T, Xing H, Xu H. A learning path recommendation model based on a multidimensional knowledge graph framework for e-learning. Knowledge Based Systems 2020:105618.
64. Çano E, Morisio M. Hybrid recommender systems: A systematic literature review. Intelligent Data Analysis 2017;21:1487-524.
65. Turnip R, Nurjanah D, Kusumo DS. Hybrid recommender system for learning material using content-based filtering and collaborative filtering with good learners’ rating. In: 2017 IEEE Conference on e-Learning, e-Management and e-Services; 2017:61-66.
66. Khribi MK, Jemni M, Nasraoui O. Toward a hybrid recommender system for e-learning personalization based on web usage mining techniques and information retrieval. In: E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education Association for the Advancement of Computing in Education (AACE); 2007. pp. 6136-45.
67. Tarus JK, Niu Z, Kalui D. A hybrid recommender system for e-learning based on context awareness and sequential pattern mining. Soft Computing 2018;22:2449-61.
68. Shin H, Kim S, Shin J, Xiao X. Privacy enhanced matrix factorization for recommendation with local differential privacy. IEEE Transac tions on Knowledge and Data Engineering 2018;30:1770-82.
69. Yang Q, Liu Y, Chen T, Tong Y. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 2019;10:1-19.
70. Lu J, Liu A, Dong F, Gu F, Gama J, et al. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering 2018;31:2346-63.
71. Ennouamani S, Mahani Z. An overview of adaptive e-learning systems. In: 2017 Eighth International Conference on Intelligent Computing and Information Systems (ICICIS) IEEE; 2017. pp. 342-47.
72. Teng SY, Li J, Ting LPY, Chuang KT, Liu H. Interactive unknowns recommendation in e-learning systems. In: 2018 IEEE International Conference on Data Mining (ICDM) IEEE; 2018. pp. 497-506.
73. Zhou P, Wang K, Guo L, Gong S, Zheng B. A Privacy-Preserving Distributed Contextual Federated Online Learning Framework with Big Data Support in Social Recommender Systems. IEEE Transactions on Knowledge and Data Engineering 2019.
74. Gao C, Huang C, Lin D, Jin D, Li Y. DPLCF: Differentially private local collaborative filtering. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval; 2020:961-70.
75. Wang W, Zhang G, Lu J. Hierarchy visualization for group recommender systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems 2017;49:1152-63.
76. Barria-Pineda J, Akhuseyinoglu K, Brusilovsky P. Explaining need-based educational recommendations using interactive open learner models. In: Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization; 2019:273-77.
77. Dwivedi S, Roshni VK. Recommender system for big data in education. In: 2017 5th National Conference on E-Learning & E-Learning Technologies (ELELTECH) IEEE; 2017. pp. 1-4.
78. Dahdouh K, Dakkak A, Oughdir L, Ibriz A. Large-scale e-learning recommender system based on Spark and Hadoop. Journal of Big Data 2019;6:1-23.
Cite This Article
Zhang Q, Lu J, Zhang G. Recommender Systems in E-learning. J Smart Environ Green Comput 2021;1:76-89. http://dx.doi.org/10.20517/jsegc.2020.06
Zhang Q, Lu J, Zhang G. Recommender Systems in E-learning. Journal of Smart Environments and Green Computing. 2021; 1(2): 76-89. http://dx.doi.org/10.20517/jsegc.2020.06
Zhang, Qian, Jie Lu, Guangquan Zhang. 2021. "Recommender Systems in E-learning" Journal of Smart Environments and Green Computing. 1, no.2: 76-89. http://dx.doi.org/10.20517/jsegc.2020.06
Zhang, Q.; Lu J.; Zhang G. Recommender Systems in E-learning. . J. Smart. Environ. Green. Comput. 2021, 1, 76-89. http://dx.doi.org/10.20517/jsegc.2020.06
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at email@example.com.