SETU VINIT GOENKA vs NATIONAL TESTING AGENCY & ANR.
$~79
* IN THE HIGH COURT OF DELHI AT NEW DELHI
+ W.P.(C) 2860/2024
SETU VINIT GOENKA …. Petitioner
Through: Mr. Arun Sharma, Adv.
versus
NATIONAL TESTING AGENCY & ANR. …. Respondents
Through: Mr. Apoorv Kurup, Ms. Nidhi Mittal, Ms. Gauri Gobardhan, Mr. Akhil Hasija and Ms. Muskaan Gupta, Advs. with Ms. Sarika Soam for NTA
Ms. Shiva Lakshmi, SPC with Mr. Archana Kumari, GP for UoI
CORAM:
HON’BLE MR. JUSTICE C. HARI SHANKAR
J U D G M E N T (O R A L)
% 27.02.2024
1. This is a petition by a student who is undertaking the JEE (Main) examination for entrance into the various Indian Institutes of Technology (IITs).
2. The examination was conducted on 27 January 2024 and the result was declared on 12 February 2024. It may be noted that, as per the scheme of IIT JEE Examination, two attempts are provided to every student to attempt the JEE (Main) examination, one held in January, the other held in April 2024, and the better of the two scores is taken.
3. The candidate who manages, on basis of the better of the two scores, to make the grade vis-à-vis her or his peers, has then to attempt the JEE (Advanced) examination for which, too, two attempts are permitted and the better of the two is taken.
4. It is on this basis that the aspirants who seek to enter the hallowed portals of the various IITs obtain entry.
5. In order to ensure that there is complete transparency, it is not possible to provide the same question paper to all candidates in the country, who, I am told, numbered over 12 lakhs this year. As a result, there is a possibility that there may be varying levels of difficulty of question papers. In order to counter-balance this possibility, the JEE follows a normalisation procedure, which is set out in detail in Appendix V to the Information Bulletin applicable to the IIT JEE examination, which may be reproduced in full thus:
Procedure to be adopted for compilation of NTA scores for multi-session Papers (Normalization procedure based on PERCENTILE SCORE)
NTA may conduct examinations on multiple dates, generally in two sessions per day. The candidates will be given different sets of questions per session and it is quite possible that in spite of all efforts to maintain equivalence among various question papers, the difficulty level of these question papers administered in different sessions may not be exactly the same. Some of the candidates may end up attempting a relatively tougher set of questions when compared to other sets. The candidates who attempt the comparatively tougher examination are likely to get lower marks as compared to those who attempt the easier one. In order to overcome such a situation, a Normalization procedure based on Percentile Score will be used to ensure that candidates are neither benefitted nor disadvantaged due to the difficulty level of the examination. With the objective of ensuring that a candidates true merit is identified and that a level playing field is created in the above context, the Normalization Procedure, set out below shall be adopted, for compiling the NTA scores for multi-session papers.
The process of Normalization is an established practice for comparing candidate scores across multi-session papers and is similar to those being adopted in other large educational selection tests conducted in India. For normalization across sections, NTA shall use the percentile equivalence.
Percentile Scores: Percentile scores are scores based on the relative performance of all those who appear for the examination. The marks obtained are transformed into a scale ranging from 100 to 0 for each session of examinees.
The Percentile Score indicates the percentage of candidates that have scored EQUAL TO OR BELOW (same or lower raw scores) that particular Percentile in that examination. Therefore the topper (highest score) of each session will get the same Percentile of 100 which is desirable. The marks obtained in between the highest and lowest scores are also converted to appropriate Percentiles.
The Percentile score will be the Normalized Score for the examination (instead of the raw marks of the candidate) and shall be used for the preparation of the merit lists.
The Percentile Scores will be calculated up to 7 decimal places to avoid the bunching effect and reduce ties.
The Percentile score of a Candidate is calculated as follows:
100 X Number of candidates appeared in the Session with raw score EQUAL TO OR LESS than the candidate
The total number of the candidates who appeared in the Session
Note: The Percentile of the Total shall NOT be an aggregate or average of the Percentile of the individual subject. The percentile score is not the same as the percentage of marks obtained.
Example: Suppose a test was held in 4 sessions of examinees as per details given below:
(Allocation of Days and shifts was done randomly)
(a) The distribution of candidates was as follows:
Session-1: Day-1 Shift-1, Session-2: Day-1 Shift-2, Session-3: Day-2 Shift-1 and Session4: Day-2 Shift-2
Session
Day/Shift
No. of Candidates
Marks
Absent
Appeared
Total
Highest
Lowest
Session-1
Day-1 Shift 1
3974
28012
31986
335
-39
Session-2
Day -1 Shift 2
6189
32541
38730
346
-38
Session-3
Day 2 Shift 1
6036
41326
47362
331
-49
Session-4
Day 2 Shift 2
9074
40603
49677
332
-44
Total (Session-1 to Session-4)
25273
142482
167755
346
-49
In this method of scoring the HIGHEST RAW SCORE in each paper (irrespective of the raw scores) will be the 100 Percentile indicating that 100% of candidates have scored equal to or lesser than the highest scorer/ topper for that session.
Highest Raw Score and Percentile Score: All the highest raw scores will have a normalized Percentile Score of 100 for their respective session.
Session
Total candidate & appeared
Highest Raw Score
Candidates who scored EQUAL OR LESS THAN Highest Raw Score
Percentile Score
Remarks
Session-1
28012
335
28012
100.0000000
[(28012/28012) *100]
i.e. all the highest raw scores would be normalized to 100 percentile Score for their respective session.
Session-2
32541
346
32541
100.0000000
[(32541/32541) *100]
Session-3
41326
331
41326
100.0000000
[(41326/41326) *100]
Session-4
40603
332
40603
100.0000000
[(40603/40603) *100]
Lowest Raw Score and Percentile Score: The percentile Score of all the lowest raw scores will depend on the total number of candidates who have taken the examination for their respective sessions.
Session
Total candidates appeared
Lowest Raw Score
Candidates who scored EQUAL OR LESS THAN Lowest Raw Score
Percentile Score
Remarks
Session-1
28012
-39
1
0.0035699
[(1/28012) *100]
i.e. percentile score of all the lowest raw scores is different i.e. percentile score depends on the total number of candidates who have taken the examination for their respective sessions
Session-2
32541
-38
1
0.0030730
[(1/32541) *100]
Session-3
41326
-49
1
0.0024198
[(1/41326) *100]
Session-4
40603
-44
1
0.0024629
[(1/40603) *100]
The following is a further explanation of the interpretation of the raw scores and Percentile Score in Session-3 (Day-2 and Shift-1) with 41326 candidates who have taken the examination.
Candidate
Percentile Score
No. of candidates
Raw Score
Remark
A
100.0000000
[(41326/41326) *100]
1
331
This indicates that amongst those who appeared, 100% have scored either EQUAL TO LESS THAN candidate A (331 raw scores)
It also indicates that no candidate has scored more than candidate A (331 raw scores)
B
90.1224411
[(37244/41326) *100]
77
121
This indicates that amongst those who appeared, 90.1224411% have scored either EQUAL TO OR LESS THAN candidate B (121 raw scores)
It also indicates that the remaining candidates have scored more than candidate B (121 raw scores)
C
50.4549194
[(20851/41326)*100]
381
41
This indicates that amongst those who appeared, 50.4549194% have scored either EQUAL TO OR LESS THAN candidate C (41 raw scores)
It also indicates that the remaining those who appeared have scored more than candidate C (41 raw scores)
D
31.7040120
[(13102/41326) *100]
789
25
This indicates that amongst those who appeared, 31.7040120% have scored either EQUAL TO OR LESS THAN candidate D (25 raw scores)
It also indicates that the remaining candidates have scored more than candidate D (25 raw scores)
E
1.1034216
[(456/41326) *100]
100
-15
Indicates that amongst those who appeared, 1.1034216% have scored either EQUAL TO OR LESS THAN candidate E (-15 raw score)
It also indicates that the remaining candidates have scored more than candidate E (-15 raw score)
STEP-BY-STEP PROCEDURE FOR NORMALIZATION AND PREPARATION OF RESULT:
Step-1: Distribution of Examinees in two shifts:
Candidates have to be distributed into two sessions randomly so that each session has an approximately equal number of candidates. These two sessions would be as follows:
Session-1: Day-1 Shift-1, Session-2: Day-1 Shift-2
In the event of a more number of days or less number of shifts, the candidates will be divided accordingly.
This will ensure that there is no bias in the distribution of candidates who shall take the examination. Further, with a large population of examinees spread over the entire country, the possibility of such bias becomes remote.
Step 2: Preparation of Results for each Session:
The examination results for each session will be prepared in the form of
> Raw Scores
> Percentiles Scores of Total raw scores.
The Percentiles would be calculated for each candidate in the Session as follows:
Let TP1 be the Percentile Score of the Total Raw Score of that candidate
Total Percentile
(TP1):
100 X
No. of candidates appeared from the session with raw score EQUAL TO OR LESS than the score of the Candidate
_____________________________________
Total No. of candidates who appeared in the session
Step-3: Compilation of NTA score and Preparation of Result:
The Percentile scores for the Total Raw Score for all the sessions (Session-1: Day-1 Shift-1, Session-2: Day-1 Shift-2) as calculated in Step-2 above would be merged and shall be called the NTA scores which will then be used for the compilation of results and further processing for deciding the allocation.
In the event of the percentiles for the multi-shifts being dissimilar/unequal, the lowest will be the eligibility cut-off for that category for all candidates (i.e. all shifts).
For Example: In the examination held in two shifts, if the 40% marks correspond to a Percentile score of 78 in Shift 1 and 79 in Shift 2, then all those equal to or above 78 percentiles (Percentile score of 100 to 78) in both shifts will become eligible in General Category. A similar method will be adopted for the other categories to determine eligibility cut-offs. In case the examination is held in more number of shifts the same principle shall apply.
6. It is clear that the process of normalisation, on percentile basis, is a detailed statistical process.
7. Mr. Kurup, who appears for the NTA, submits that this is a procedure and a formula which is adopted worldwide and was in fact adopted on the basis of the recommendation of a high-powered committee in November 2018, which was subsequently reviewed in October 2020.
8. Needless to say, this procedure applies across the country to every student who undertakes the IIT JEE examinations.
9. The petitioner appeared for the JEE (Main) examination on 27 January 2024. The result of the examination was declared on 12 February 2024 as a percentile, in accordance with the aforenoted normalisation process envisaged in the Information Bulletin applicable to the IIT JEE examination.
10. Paras d to q of para 3 of the present writ petition, which contain the substratum of the challenge being raised herein, read thus:
d. The petitioner had given the exam of JEE (Main) 2024 conducted by the National Testing Agency during the 1st Shift of the exam held on 27th January 2024. The result of the same examination was declared by the National Testing Agency on 12.02.2024 (declared at 3 am on 13 February 2024).
e. Being aggrieved by the method of calculation of result and the result in percentile declared by the National Testing Agency (hereafter referred to as Respondent No. I) of the session I of JEE Main 2024 with regards to the discrepancies in the so-called Normalization process adopted by the Respondent no. 1 while declaring the results of the students/applicants who appeared for the examination. Needless to say that the basis of percentile adopted by the Respondent No 1 is violation of the Right to Equality as the Respondent No 1 (NTA) never shares Approved Procedure/Methodology/ Information/Policy/Factor/Formula of Calculation of Percentile from Raw Marks scored across different Shifts nor they have shared total number of raw marks in the result sheet. Aspirants made to appeared in different shifts by NTA and who scored the same grade/raw marks but at the time of declaring the result were allocated with different percentile.
f. The Respondent No I never shares the process of so-called Normalization, the formula/factor/methodology and the relevant applicable data to calculate the same, including the raw marks, normalized marks, factor/formula of normalization, which ought to be declared for every single exam set/shifts.
g. The Respondent no. 1 had issued the notification dated 01.11.2023 regarding the upcoming entrance exams that Respondent No 1 will be conducting for the Academic year 2024-2025. The said notification has been annexed herein as Annexure-1. That the Respondent No 1 had also issued an information bulletin for all the students who were applying for JEE (Main) 2024. The said information bulletin has been annexed herein as Annexure 2.
h. That the petitioner had filled in the form for the entrance on 4th of November 2023. The prescribed enrollment fee of Rs. 2000 was also duly submitted by the petitioner. Copy of the payment receipt has been annexed herein as Annexure 3.
i. That the hall ticket or admit card were issued by the Respondent No 1 on 25 January 2024. The date of examination and shift was assigned randomly in the hall ticket by NTA and the aspirant has no say in selecting the same. The hall ticket of the petitioner has been annexed herein as Annexure 4.
j. That the petitioner appeared for the JEE (Main) examination on 27th January 2024 during the 1st Shift as per assigned by NTA in hall ticket.
k. On 9th February, 2024, the petitioner also filed for challenging a question/answer key with NTA. There is a defined process of reverification in Chapter I2- Display of Answer Key for Challenge in Information Bulletin available on NTA website.
l. The petitioner paid the duly fee for the same on 9 February 2024. The receipt for payment of the same has been annexed herein as Annexure 5.
m. That the results of the Session 1 of JEE Main examination were declared on 12 February 2024 on the website of Respondent No 1. The result was declared as a percentile. It was this result that is the main cause of concern for the petitioner. The petitioner’s copy of result has been annexed herein as Annexure 6.
n. The results declared by the Respondent No I had discrepancies. The students with different raw marks were allotted similar percentiles. However, the most glaring defect is that the difference between the marks of the students was not negligible but, in few cases, it was between 50-80 marks. However, it is pertinent to mention here that these students were from different shifts of the exam. The same issue was reported in various newspapers and by various media channels.
o. That the Respondent No 1 follows a so-called process of “NORMALIZATION” in granting marks and declaring result. As per the information bulletin issued by the NTA, the so-called process of Normalization is based on percentile score. The NTA states the purpose of Normalization as “an established practice for comparing candidate scores across multi-session papers and is similar to those being adopted in other large educational selection tests conducted in India. For normalization across sections, NTA shall use the percentile equivalence”.
p. That the so-called Normalization process is unjust and unfair for students. The normalization factor is not disclosed by Respondent No 1. This, however, should not be a ground for judging the performance of any student as they all appear for the same exam through different shifts randomly allocated by National testing agency.
q. As a Testing Agency, it is the ownership of the Respondent No 1 to set papers that are at equal difficulty levels for all shifts for the same examination so that it equal opportunity and assessment for all aspirants. Equal opportunity is a state of fairness in which all aspirants for the same examination are treated similarly, unhampered by artificial barriers, prejudices, or preferences.
11. I have heard Mr. Arun Sharma, learned Counsel for the petitioner, and Mr. Apoorv Kurup, learned Counsel for the NTA.
12. The entire challenge of the petitioner appears to be somewhat nebulous.
13. The petitioner was, even while undertaking the examination, well aware of the normalisation procedure followed by the IIT as contained in Appendix V to the Information Bulletin of the JEE Examination. It was in full consciousness of this procedure that the petitioner went ahead and undertook the examination.
14. It is a well-settled position, in administrative law, that a candidate cannot, in awareness of the system of conducting an examination and the marking process adopted in that regard, challenge the examination or the manner in which it is conducted, after the result are declared.
15. Needless to say, had the petitioner obtained a percentile score to his satisfaction, this writ petition would never have been filed.
16. That apart, a reading of the writ petition reveals that the allegations are entirely speculative in nature. The only case that the petitioner has been able to make out in the petition as well as during oral arguments of Mr. Sharma, is that the raw scores of the candidates differ from the percentile scores which they are ultimately assigned.
17. Mr. Sharma sought also to question the logic of basing the normalisation and percentile process on the basis of the number of candidates attempting the examination, as is reflected in the formula contained in Appendix V to the Information Bulletin.
18. Mr. Kurup pointed out that the number of students who have attempted a particular paper has necessarily to be a relevant consideration while working out the percentile to which a candidate would be entitled. The numerator and the denominator, in the formula for working out the percentile, as contained in Appendix V factor in the number of candidates who attempt a particular paper of a particular difficulty level, vis-à-vis the individual candidate whose score is being considered.
19. The performance of a candidate has, he points out, to be assessed vis-à-vis her, or his, peers who have attempted papers of the same difficulty level. It is for that reason that the number of candidates who have attempted the paper becomes relevant.
20. In fact, Mr. Sharma has cited an example which itself defeats his case. He sought to submit that, if there are two question papers, and one candidate gets 150 marks, by application of percentile, he may not qualify for further advancement in the examination whereas another candidate who obtains only 90 marks in another paper for the same examination may qualify.
21. That, in fact, is the very rationale of the percentile system. It is not the absolute marks of the candidate which are taken into account but the relative marks of the candidates vis-à-vis the marks obtained by other candidates who attempt the paper of the same difficulty level. In that process, there is every possibility of a candidate who has obtained lower absolute marks in a qualifying paper qualifying for advancement, whereas a candidate who has obtained higher absolute marks in another paper, for the same examination, but of a possibly different difficulty level, does not qualify.
22. It is in fact to iron out the discrepancy which arises as a result of the unavoidable possibility of different papers being of different difficulty levels that the process of normalisation is adopted.
23. The Court also appreciates the fact that, when the JEE is being conducted across the country with lakhs of students undertaking the examination, it is but inevitable that all the students cannot be given the same paper, or papers which are clinically of the same level of difficulty.
24. It is a well-settled practice, followed in such cases, to provide different papers to different batches of students, normally attempting papers at different locations. In such cases, absolute mathematical accuracy, to ensure that all papers are of identical difficulty level, is impossible.
25. Some play in the joints has necessarily to be allowed in case such examinations are to be conducted successfully. Individual discomfitures are inevitable in such a process, and cannot afford a basis to unseat the entire examination.
26. This Court does not possess the expertise to subjectively go into the intricacies of the normalisation procedure. These are matters of academic policy, in which the Court has to defer to the authorities, unless the procedure is found to be so arbitrary or resulting in constitutionally unsustainable results which the court not uphold at any cost. No such case has been made out in the averments contained in the present writ petition.
27. Though Mr. Sharma prayed that this Court should call for the record and assess for itself whether the normalisation procedure/percentile procedure has resulted in an acceptable outcome, a writ petition cannot be made a basis of such a roving inquiry.
28. Mr. Sharmas submission that the details of the normalization procedure have not been disclosed also fails to impress. The entire process of normalization, including the manner in which the percentile score is worked out, stands exhaustively delineated in the Information Bulletin, as extracted in para 5 supra. There is, therefore, complete transparency in this regard.
29. Courts have, in my considered opinion, to be conscious even while issuing notice in such cases, where lakhs of students are involved. The very fact that an examination such as the IIT JEE, which governs entrance to IITs, NITs and other centrally funded technical institutions, may be subject matter of a Court proceeding, is itself a serious issue. It also creates uncertainty in the minds of students who attempt the papers. Courts have, therefore, to be extremely careful even while issuing notice in such cases. It is only if the procedure being followed is constitutionally completely unacceptable that such cases deserve issuance of notice.
30. The threshold to be met in such cases is very high. This writ petition does not meet that threshold. Besides the fact that the petitioner has sought to challenge the marking process despite having at all times being fully aware thereof, after having attempted the examination and after the marks have been declared, even otherwise, on merits, I am not of the view that the challenge deserves to be entertained.
31. The writ petition is therefore dismissed in limine.
C. HARI SHANKAR, J.
FEBRUARY 27, 2024
dsn
Click here to check corrigendum, if any
WP(C) 2860/2024 Page 16 of 16