Proctored, independent, authorised assessments

Proctored, independent, authorised assessments (PIAAs) are a critical part of the entire iGOT Karmayogi platform. Using PIAA the goal is to provide a qualitative edge to the platform by creating a baseline mapping of learners for all competencies they require in a given position. It will also establish whether, after completing a course, the learner has actually learnt and moved higher up in competency levels. In doing so, it determines the effectiveness of a particular course and its provider.

Due to the absence of long term assessments, there is a gap in data to indicate the success, or lack of it, of a capacity building programme. Without data on the progress of learners, the uptake of courses, and preferences and behaviours on the platform, we cannot analyse and gauge if learners and course providers are on the right path.

When defined within an educational setting, assessment, evaluation, and testing are all used to measure how much of the assigned materials students are mastering, how well learners are learning, and how well they meet their goals. 

In order to build a repository of information about learner patterns, and digest that data into meaningful intelligence, the types of assessments that need to be created are manifold.

For the purpose of the iGOT Karmayogi platform, the effort is to make government officials more capable through specific trainings. The best way to assess the viability of these trainings is by evaluating the impact they have on officials. It is therefore important that the assessment of the individual is done in the most credible manner. This can be done using proctored, independent, authorised assessments (PIAAs).

With online education, there is a probability that learners use non-permitted tools to aid them through assessments. To maintain honesty and reliability, a growing number of administrators are incorporating online proctoring into their strategy.

Online proctoring helps maintain the integrity and uniformity of assessments, for which these proctoring services need to have features that the iGOT Karmayogi platform prescribes. Course providers who do not have proctoring processes in place can avail of the services from certified proctoring agencies. Other benefits of online proctoring include reduced cost of assessments and diminished need for physical infrastructure. 

Another aspect of PIAAs is working with independent agencies to effectively remove bias while designing, conducting and scoring assessments. Assessment instruments like PIAAs can be designed by anyone who considers themselves competent. The iGOT Karmayogi platform will provide an online course for those who would like to learn about Assessment Instrument Design and Test Item Creation. By authorising test creators the platform builds trust and the knowledge of a competent assessor is one of the core elements of the process. To make sure that the right criteria are met, assessment designers themselves are assessed. Assessment designers need to be:

• Competent in the area of knowledge they are assessing by taking an assessment for the competency they wish to create a PIAA for.
• Certified to design and deploy such instruments.

Just as designers of assessments are vetted, the instruments of assessment must also be assessed. This establishes their validity and reliability. Reliability is the extent to which the assessment would produce the same or similar results on two or more occasions. Validity is the extent to which an instrument measures what it intends to.

Finally, the assessments themselves will be based on the three competency types: behavioural, functional and domain. The assessment providers will be given competency dictionaries to design the instruments. Also, there are many dependencies on the results of these assessments. Therefore, it is important that the quality of the assessments is transparent and credible. To achieve this, it is recommended that a third party expert agency be tasked with establishing these capabilities. This approach will allow for:

• The capabilities required to set up and run PIAAs are specialised services and not easily available.
• The assessment methodology and its interplay with technology is an emerging discipline. By centralising such capabilities, it is more likely to keep pace with rapidly shifting standards and requirements.

Assessments provide the iGOT Karmayogi platform with a clear advantage and have several benefits. In the short term, the advantages that accrue are in terms of cost, and creating a talent inventory for rapid deployment while in the longer term, it will help in the de-siloisation of the bureaucracy, creating a culture of learning.