Engagement Score Framework

When a person learns using an online system that is new to them, there are several factors that are important to ensure user acceptability of the platform. The engagement score therefore, is a measure of how well the learners are connected to a particular platform and has a direct correlation with platform acceptability, learning trajectory and user satisfaction.

As e-learning expands there are questions on user engagement and why some users stop their online learning after their initial experience. Any good e-learning platform needs to not only provide an interactive, easy to use and effective environment that facilitates the user to be on the platform for a significant amount of time but also to measure these interactions and drive user engagement and subsequent learning.

The engagement score was designed for learners on the iGOT Karmayogi platform. The score is based on an algorithm that considers high engagement behaviour; open, click, course completion etc and low engagement behaviour; boredom, the goal not achieved, drop-out rate etc. The higher the score, the more engaged a user is on the platform. A high score indicates a strong likelihood of the user engaging with the platform in the future as well. 

When users engage with a virtual learning platform, they exhibit certain behaviours on the platform. These behaviours can be categorised under three broader relationships:

• Relationship with self
• Relationship with others
• Relationship with content

Our approach towards engagement scores is through a four-step process, which includes a compilation of the following:

• A list of behaviours through our research that users exhibit on a learning platform
• A list of selective behaviours that incentivise user engagement
• A list of indicators that measure behaviour
• Weightage that is given to behaviours/indicators to arrive at the final Engagement score

For the final step in the process, the weights are determined by the prioritisation. In this approach, behaviours are weighted based on certain principles of prioritisation and the weightages are predetermined.  The four principles of prioritisation include the effort required to do a certain activity, specific skills required to perform certain activities, the amount of time required to do a particular activity and activities which directly impact competency building are given a higher score than activities just enable competency building.

The engagement score will ultimately be able to encourage users to interact on the platform and incentivise beginner level users to increase their confidence on the platform. It aims help content providers enhance their offerings and design better courses and assist state and central governments in engagement performance comparison leading to healthy competition between departments and ministries.