The IRT model and four-parameter logistic model

As we know, the IRT models include the Rasch model and the two-parameter Logistic model(2PLM) [5]. The 2PLM is described as

Concerning guessing responses in tests, Birnbaum (1957) proposed a three-parameter Logistic model (3PLM) [5], which was expressed as

Barton and Lord (1981) explored whether changing the upper asymptote could improve standardized test scoring [6]. Barton and Lord (1981) therefore influenced the upper-asymptote model by adding a fourth parameter, γ, dropping the upper asymptote to below 1:

Some researcher use the γ parameter to denote the upper asymptote [7]. Barton and Lord (1981) re-estimated the test scores of thousands of students taking four separate exams to determine the effects of fixing γ at 0.99 and 0.98 and concluded that the changes were too small to be of practical significance [6].

In this article, the 4PLM-based robustified approach, abbreviated as 4PLM-Robust, serves the purpose of robust estimation in the presence of response disturbances including random guessing and carelessness.

The method of the 4PLM-robust

A new approach, a robust estimation based on the 4PLM (4PLM-Robust) is proposed.

At first, the maximum-likelihood estimation of ability is illustrated. Let u_i denote the response of an individual to the i-th item. The probability of a correct response is denoted by P(η_i), where η_i is an expression involving ability and item parameters in 2PLM. The probability of a single response u_i is given by

The logarithm of this probability, l_i =u_i⋅log(P(η_i))+(1-u_i)⋅log((1−P(η_i))), is the log likelihood contribution to the item response. If local independence holds, these contributions can be added. Taking derivatives of ability and setting the result at zero yields

Using the Logistic model, P(η_i) = exp(η_i)/(1+exp(η_i)), and noting that $$ \frac{dP({\eta }_i)}{d\theta }=\frac{dP({\eta }_i)}{d{\eta }_i}\cdot \frac{d{\eta }_i}{d\theta }$$, so Eq 4 simplifies to

Let η_i = r_i = a_i(θ−b_i), and P(η_i) = exp(r_i)/(1+exp(r_i))

Using the 3PLM, that is, P(η_i) = exp(r_i)/(1+exp(r_i)), Eq 5 can be transformed into

Using the 3PLM, that is, $$ P({\eta }_i)=c_i+(1-c_i)\cdot \frac{\exp (r_i)}{(1+\exp (r_i))}$$, Eq 5 can be transformed into

If c_i = 0, Eq 7 will be the same as Eq 6.

Using the 4PLM, that is, $$ P({\eta }_i)=c_i+({\gamma }_i-c_i)\cdot \frac{\exp (r_i)}{(1+\exp (r_i))}$$, Eq 5 can be transformed into

If c_i = 0 and γ_i = 1, Eq 8 will be the same as Eq 6.

In this article, a robust estimation based on the 4PLM, abbreviated as 4PLM-Robust, is proposed, and it is an alternative approach to robustifying ability estimation. The 4PLM-Robust estimation formulated as follows:

where 0≤p*<0.20 and 0≤p**≤0.03.

If the guessing and carelessness response disturbances do not occur simultaneously on the i-th item, then p* = 0 and p** = 0. If p* = 0 and p** = 0, then c_i = 0 and γ_i = 1 in Eq 8. If c_i = 0 and γ_i = 1, 4PLM-Robust estimation become 4PLM. That is, 4PLM-Robust estimation originate in 2PLM and can be simplified as 2PLM. In other words, 2PLM is one special case of 4PLM-Robust estimation.

The subjects only guess when item difficulty extends beyond their abilities to the extent where their knowledge cannot bring them apparent benefit, and in such cases, the subjects most likely choose randomly from among the four or five alternative responses. As demonstrated by Mislevy and Bock (1982), subjects are assumed to have guessed on the item if their expected probability of correct responding [3], P(r_i) = exp(r_i)/(1+exp(r_i)), is below the p* value at 0.05, or 0.10, or 0.15, because the probability of a random choice from five alternatives is equal to 0.20. If the expected probability P(θ) is lower than the critical point p*, the subject is apt to respond by random guessing. Thus, p* denotes a critical point for the probability of random guessing, and is noted as a critical-probability guessing parameter. In the paper, ψ(θ_v−b_i) represents the correct response probability of the subject v using the 2PLM. For example, when p* = 0.10 is deemed as the average probability for guessing on the i-th item, ψ(θ_v−b_i)≤0.10 indicates r_i = a_i(θ−b_i)≤−2.20. ψ(θ_v−b_i)<p* indicates ψ(θ_v−b_i)<0.05 and r_i = a_i(θ−b_i)≤−1.73 when p* is at 0.05, and ψ(θ_v−b_i)<0.15 and r_i = a_i(θ−b_i)≤−2.97 when p* = 0.15. Moreover, a setting of p* =0.15 signals a high level of guessing in a test, and p* = 0.05 indicates low guessing in a test, and p* = 0.00 indicates no guessing.

Mislevy and Bock (1982) also proposed the carelessness parameter p**, signifying the probability of occasional error because of carelessness [3]. Schuster and Yuan (2011) expounded p** as an occasional error probability, which reflected the result of carelessness, transcription errors, test anxiety and exorbitant intention [4]. Furthermore, they noted that error probability p** was not dependent on subject ability or item difficulty. In the present study, given that carelessness is a small probability event in school achievement testing, it is believed that p** is a small probability, i.e., 0≤p**≤0.03. Similarly, the approach of the 4PLM-Robust to robustify ability estimation in the presence of careless responses can be expressed as γ_i = 1−p**. However, defining or identifying the occasional error probability in practice tests is complicated. If p** = 0.01, it can be assumed that one item may be missed on a one hundred-item test by an occasional error, so p** = 0.01 is a low probability of occasional error. If p** = 0.03, it can be assumed that one item may be missed on a thirty-three-item test by an occasional error, so p** = 0.03 is a high probability of occasional error.

In the Eq 8, if p* = 0.15 and p** = 0.03, the 4PLM-Robust is abbreviated to the 4PLM-Robust (p* = 0.15, p** = 0.03).

The Biweight and Huber robust estimation

In the Biweight, Huber robust estimation, two decisions determine the weight attached to each item: (a) the definition of the residual between estimated ability and item difficulty, r_i = a_i(θ−b_i); (b) the selection of the weight function w(r_i).

In the Biweight, and Huber robust estimation, the robust estimate of ability requires a modified likelihood equation solution, that is, Eq 5 is weighted by w(r_i). So Eq 5 becomes as:

In Biweight estimations, the weight function w(r_i) is shown as

The so-called tuning constant B determines the degree to which residuals are downweighted. Large values of B mean little downweighting and small values mean considerable downweighting. Mislevy & Bock (1982) used B = 4.0. Thus, as r_i exceeds 4.0, the weight becomes zero, eliminating the observation from the estimating. If r_i = 0, then w(r_i) = 1 in Eq 10 and Eq 9. And then Eq 9 will become the same as Eq 5. That is, 2PLM is the special case of Biweight estimation.

In Huber estimations, the weight function w(r_i) is shown as

In the Huber estimation, Schuster and Yuan (2011) used H = 1.0 [4]. The weight function H does not downweight responses at all as long as |r_i|≤H, that is, w(r_i) = 1 in Eq 11 and in Eq 9, and then Eq 9 becomes the same as Eq 5. That is, 2PLM is the special case of Huber estimation.

Therefore, Biweight and Huber estimations do downweight the responses by the weight function w(r_i) and the tuning constants B or H. By comparison, the 4PLM-Robust method robustify ability estimation by c, γ parameter in the 4PLM according to r_i = a_i(θ−b_i), so the 4PLM-Robust estimations do not downweight the responses.

Method 1: Maximum likelihood estimation using the 2plm(2PLM-MLE)

The maximum likelihood using the 2PLM estimates is identical for subject #2 to #6, and so is for subject #7 to #11. Thus, in this case, the different response patterns cannot be differentiated by the 2PLM-MLE. Similar results have also been observed by other researchers (Embretson & Reise, 2000) [18].

The item difficulty of the 10-th item (b₁₀ = 4) is greatly larger than the ability of Subjects #6, but Subject #6 gives a correct response on the 10-th item by guessing probably and get the same ability value as Subject #2. So there is a guessing response probably, and the ability estimation of Subject #6 is not robust.

The item difficulty of the1-st item (b₁ = −4) is greatly smaller than the ability of Subject #11, but he or she gives a wrong response on the 10-th item because of carelessness or other error probably and gets the same ability value as Subject #7. So there is a carelessness response probably, and the ability estimation of Subject #11 is not robust.

Method 2: Using the Biweight estimation (BIW estimation)

The BIW estimates for subject #2 to #6 differ from one another, as do the BIW estimates for subject #7 to #11. The score on the 10-th item of Subject #6 can be seen as most seriously deviated from the referenced pattern of subject #1, because on the probability it can be conclude that Subject #6 has a guessing response on the 10-th item. so the BIW estimate disregards his unexpectedly correct response to item 10 and sets the ability estimate to 0.163; i.e., the influence of the guessing response in subject #6 is reduced. Moreover, the responses from Subject #5 and Subject #4 deviate from the referenced pattern, and the BIW estimate assigns them ability estimates of 0.421 and 0.653, respectively. The estimated abilities of subjects #2 and #3, according to the BIW, are close to the estimated abilities using the 2PLM-MLE.

Conversely, subject #11 missed the easiest item, and his response pattern deviates from the referenced pattern of subject #1. Accordingly, the BIW estimate largely disregards his unexpectedly correct response to item 1 and sets the ability estimate to 0.048, which reduces the influence of the carelessness responses of subject #11. The responses of subject #10 and subject #9 also deviate from the referenced pattern. The BIW estimate assigns them ability estimates of -0.160 and -0.617, respectively. The estimated abilities of subjects #7 and #8, using the BIW, are close to their estimated abilities using the 2PLM-MLE.

Method 3: Maximum likelihood estimation using the 3PLM (3PLM-MLE)

First, c is assigned as 0.2 from item 1 to 10. Using the 3PLM-MLE, the estimate for subject #6 is -0.080, which is quite close to the estimate for subject #1 compared with subject #2 to #5. The trend of the estimates for subject #2 to #6, using the 3PLM-MLE, is similar to that of the estimates using the BIW.

All of the subjects’ estimates using the 3PLM-MLE are lower than those which use the 2PLM-MLE. Notably, the estimates for subject #8 to #11 are all underestimated by a large margin.

Method 4: Maximum likelihood estimation using the 4PLM (4PLM-MLE)

In item 1 to 10, c is assigned as 0.2, and γ is assigned as 0.95. The estimates for subject #2 to #6, using the 4PLM-MLE, are close to the estimates using the 3PLM-MLE. The estimates of subject #5 and #6 are close to the estimate of subject #1, using the 4PLM-MLE; i.e., the influences of the guessing responses of subject #5 and #6 have been decreased.

Using the 4PLM-MLE and the BIW, the ascending trends of subject #8 to #11 are similar, and the estimate of subject #11 is close to that of subject #1. The influences of the guessing responses of subject #6 and the carelessness responses of subject #11 are decreased. However, the estimates of subject #8 and #9, using the 4PLM-MLE, are underestimated compared with those using the 2PLM-MLE and the 3PLM-MLE.

Method 5: Maximum likelihood estimation using the 4PLM-Robust (p* = 0.10, p** = 0.02)

If p* = 0.10 and p** = 0.02 in Eq 5, the 4PLM-Robust is abbreviated to the 4PLM-Robust (p* = 0.10, p** = 0.02) in the paper.

The trend of the estimates for subject #2 to #6 using the 4PLM-Robust is similar to the trend derived using the BIW, the 3PLM-MLE and the 4PLM-MLE. In addition, the trend of the estimates for subject #7 to #11 using the 4PLM-Robust is similar to the trend using the BIW. The estimates of subject #5 and #6 are close to the estimate of subject #1 using the 4PLM-Robust; i.e., the influences of the guessing responses of subject #5 and #6 can effectively be decreased.

The ascending trend of the 4PLM-Robust for subject #7 to #11 is identical to that of the BIW. The estimates of subject #10 and subject #11 are close to the estimate of subject #1; i.e., the influences of the carelessness responses on subject #10 and subject #11 can be decreased effectively.

Method 6: The Huber estimation

The decreasing trend of the Huber estimates for subject #2 to #6 is identical to that of the BIW estimates, although there are differences between them. The ascending trends of the Huber estimates and the BIW estimates for subject #7 to #11 are also identical. As Schuster and Yuan (2011) illustrated, Huber can robustify ability estimation for aberrant responses.

In summary, the ’Example’ can show the guessing and carelessness responses, and the authors try to compare the several robust estimation methods.

Simulation Case I: Random guessing without carelessness

The first type of disturbance is considered random guessing. Responses indicating random guessing were generated by the following model (Mislevy & Bock, 1982) [3]:

In Simulation Case I there is no carelessness; the approach using the 4PLM-Robust to robustify ability estimation is as follows:

Table 2 shows the biases between the estimated ability and the true ability value in Simulation Case I, varying combinations of the true ability value and p*. In Table 2, the BIW (B = 4) represents the BIW estimation with the BIW iterations from maximum likelihood estimates [3], with B = 4. Huber (H = 1) represents the Huber estimation with Huber iterations from maximum likelihood estimates, with H = 1 [4].

Table 2

The biases in Case I: Random guessing without carelessness.

Generated model	Method	True ability value
		-2.000	-1.000	0.000	1.000	2.000
p* = 0.00	2PLM-MLE	0.003	-0.009	-0.005	-0.003	0.002
	3PLM-MLE	-0.628	-0.573	-0.558	-0.524	-0.486
	4PLM-MLE	-0.552	-0.495	-0.461	-0.415	-0.354
	4PLM-Robust (p* = 0.00)	-0.001	-0.006	0.004	0.011	0.003
	4PLM-Robust (p* = 0.05)	-0.047	-0.044	-0.025	-0.029	0.014
	4PLM-Robust (p* = 0.10)	-0.078	-0.063	-0.072	-0.071	-0.050
	4PLM-Robust (p* = 0.15)	-0.123	-0.116	-0.097	-0.102	-0.099
	BIW (B = 4)	0.000	-0.010	-0.010	0.000	0.010
	Huber (H = 1)	-0.010	0.000	0.000	0.000	0.010
	4PLM-Robust (p* = 0.10, p** = 0.02)	-0.038	-0.020	-0.006	0.024	0.018
p* = 0.05	2PLM-MLE	0.722	0.483	0.289	0.132	0.021
	3PLM-MLE	-0.579	-0.531	-0.509	-0.510	-0.491
	4PLM-MLE	-0.504	-0.439	-0.417	-0.383	-0.354
	4PLM-Robust (p* = 0.00)	0.710	0.521	0.287	0.098	0.042
	4PLM-Robust (p* = 0.05)	-0.007	0.005	0.010	-0.009	-0.008
	4PLM-Robust (p* = 0.10)	-0.051	-0.048	-0.012	-0.017	-0.015
	4PLM-Robust (p* = 0.15)	-0.111	-0.096	-0.087	-0.082	-0.071
	BIW (B = 4)	0.050	0.042	0.058	0.039	0.047
	Huber (H = 1)	0.180	0.170	0.130	0.050	0.022
	4PLM-Robust (p* = 0.10, p** = 0.02)	0.044	0.046	0.050	0.041	0.052
p* = 0.10	2PLM-MLE	0.777	0.580	0.386	0.180	0.063
	3PLM-MLE	-0.535	-0.500	-0.495	-0.473	-0.476
	4PLM-MLE	-0.448	-0.406	-0.383	-0.350	-0.340
	4PLM-Robust (p* = 0.00)	0.840	0.561	0.437	0.192	0.077
	4PLM-Robust (p* = 0.05)	0.165	0.161	0.142	0.080	0.032
	4PLM-Robust (p* = 0.10)	-0.004	0.003	-0.009	-0.011	0.021
	4PLM-Robust (p* = 0.15)	-0.088	-0.059	-0.089	-0.074	-0.052
	BIW (B = 4)	0.102	0.096	0.095	0.078	0.044
	Huber (H = 1)	0.287	0.236	0.186	0.121	0.042
	4PLM-Robust (p* = 0.10, p** = 0.02)	0.096	0.108	0.119	0.105	0.101
p* = 0.15	2PLM-MLE	0.832	0.623	0.471	0.266	0.083
	3PLM-MLE	-0.530	-0.487	-0.463	-0.472	-0.453
	4PLM-MLE	-0.436	-0.375	-0.354	-0.344	-0.318
	4PLM-Robust (p* = 0.00)	0.811	0.643	0.458	0.232	0.077
	4PLM-Robust (p* = 0.05)	0.228	0.290	0.236	0.120	0.066
	4PLM-Robust (p* = 0.10)	0.063	0.065	0.041	-0.023	-0.032
	4PLM-Robust (p* = 0.15)	-0.031	-0.042	0.012	-0.039	-0.027
	BIW (B = 4)	0.173	0.143	0.122	0.127	0.106
	Huber (H = 1)	0.311	0.270	0.198	0.143	0.153
	4PLM-Robust (p* = 0.10, p** = 0.02)	0.131	0.146	0.130	0.147	0.145

The first section of Table 2 shows the results when the response-generated model is set at p* = 0.00, which indicates no guessing. The absolute values of the biases of the 2PLM-MLE, the BIW estimation, the Huber estimation, and the 4PLM-Robust (p* = 0.00) of these subjects are minimal, less than 0.02. The absolute values of the biases of the 4PLM-Robust (p* = 0.05), the 4PLM-Robust (p* = 0.10), and the 4PLM-Robust (p* = 0.15) of the five subjects become successively larger. Because responses from guessing are generated with p* = 0.00, which indicates no guessing disturbance, the absolute values of the 4PLM-Robust (p* = 0.00) biases are small, close to zero. However, the absolute values of the 4PLM-Robust biases with p* = 0.05, 0.10, and 0.15 are slightly larger. Moreover, the biases, using the 3PLM-MLE and the 4PLM-MLE, are all negative values, and their absolute values are relatively large, as shown in the other sections of Table 2. This result indicates that the 3PLM-MLE and the 4PLM-MLE severely underestimate the subjects.

The second section of Table 2 shows the results when the response-generated model is set at p* = 0.05. The biases of the 2PLM-MLE, the Huber estimation, and the 4PLM-Robust (p* = 0.00) indicate that those biases of the lower-ability subjects are larger than those of the higher-ability subjects. Compared with the results when p* = 0.00, the bias of the 2PLM-MLE is 0.722 at a true ability level of -2.00, as is the other lower-ability subject. Conversely, the absolute values of the biases of the 4PLM-Robust (p* = 0.05) are not larger than 0.01. The absolute values of the biases of the 4PLM-Robust (p* = 0.10) and the 4PLM-Robust (p* = 0.15) also decreased, compared with the biases of the 2PLM-MLE. The biases which use the 3PLM-MLE and the 4PLM-MLE are all negative values, and their absolute values are relatively large.

The third section of Table 2 shows the results when the response-generated model is set at p* = 0.10. Along with the increase of p* value, the absolute values of the biases of the 2PLM-MLE, the BIW estimation, the Huber estimation, and the 4PLM-Robust (p* = 0.00) increase compared with the result at p* = 0.05 and p* = 0.00. Simultaneously, the absolute values of the biases of the 4PLM-Robust (p* = 0.10) are nearly zero; in other words, measure disturbances by guessing are reduced. This is also visible in the fourth section of Table 2, in which the biases of the 4PLM-Robust (p* = 0.15) are nearly zero.

Consequently, if the critical point p* of the 4PLM-Robust is equal to that of the guessing responses in the generated model, the biases of the 4PLM-Robust (p* = critical point) will be nearly zero. The 4PLM-Robust (p* = critical point) is superior to the BIW estimation and the Huber estimation if the guessing disturbance’s probability is equal to or greater than 0.05.

Regarding each examinee’s absolute values of biases in Table 2, 4PLM-Robust (p* = 0.10, p** = 0.02) is a little larger than the Huber and BIW estimation when p* = 0.00. However,4PLM-Robust (p* = 0.10, p** = 0.02) is less than the Huber and the BIW estimation when p* = 0.10 and p* = 0.15.

Simulation Case II: Carelessness without random guessing

The second type of response disturbance is carelessness. The authors of this paper propose a new generating model for the carelessness disturbance:

In Case II, the approach using the 4PLM-Robust to robustify ability estimation is perceived as

Table 3

The biases in Case II: Carelessness without random guessing.

Generated model	Method	True ability value
		-2.000	-1.000	0.000	1.000	2.000
p** = 0.00	2PLM-MLE	-0.005	0.004	-0.006	0.015	0.006
	3PLM-MLE	-0.629	-0.571	-0.557	-0.513	-0.485
	4PLM-MLE	-0.534	-0.489	-0.445	-0.404	-0.363
	4PLM-Robust(p** = 0.00)	0.005	-0.009	-0.001	-0.006	0.003
	4PLM-Robust(p** = 0.01)	0.052	0.088	0.103	0.109	0.112
	4PLM-Robust(p** = 0.03)	0.116	0.129	0.143	0.151	0.159
	BIW (B = 4)	-0.011	0.008	0.017	0.002	0.010
	Huber (H = 1)	-0.007	0.005	-0.008	-0.004	-0.012
	PLM-Robust (p* = 0.10, p** = 0.02)	-0.042	-0.030	0.000	0.026	0.035
p** = 0.01	2PLM-MLE	-0.028	-0.039	-0.034	-0.057	-0.061
	3PLM-MLE	-0.673	-0.611	-0.628	-0.594	-0.580
	4PLM-MLE	-0.635	-0.542	-0.522	-0.446	-0.425
	4PLM-Robust(p** = 0.00)	-0.087	-0.082	-0.078	-0.077	-0.072
	4PLM-Robust(p** = 0.01)	0.012	-0.031	0.005	0.023	0.005
	4PLM-Robust(p** = 0.03)	0.054	0.033	0.024	0.056	0.089
	BIW (B = 4)	-0.043	-0.028	-0.032	-0.029	-0.023
	Huber (H = 1)	-0.034	-0.021	-0.050	-0.042	-0.036
	4PLM-Robust (p* = 0.10, p** = 0.02)	-0.086	-0.049	-0.040	-0.011	0.019
p** = 0.03	2PLM-MLE	-0.059	-0.079	-0.082	-0.110	-0.128
	3PLM-MLE	-0.739	-0.718	-0.731	-0.775	-0.770
	4PLM-MLE	-0.664	-0.616	-0.618	-0.585	-0.543
	4PLM-Robust(p** = 0.00)	-0.062	-0.112	-0.118	-0.126	-0.132
	4PLM-Robust(p** = 0.01)	-0.016	-0.012	-0.010	0.010	0.009
	4PLM-Robust(p** = 0.03)	0.030	0.041	0.047	0.049	0.043
	BIW (B = 4)	-0.052	-0.056	-0.065	-0.051	-0.059
	Huber (H = 1)	-0.077	-0.069	-0.078	-0.062	-0.081
	4PLM-Robust (p* = 0.10, p** = 0.02)	-0.139	-0.115	-0.119	-0.105	-0.084

Note. The biases are obtained by $$ ({\displaystyle \sum {\widehat{\theta }}_k})/5000-\theta $$ using 5,000 replications.

The first section of Table 3 shows the results when p** is set at 0.00 for the response-generated model, indicating no occasional errors. The absolute values of the biases of the 2PLM-MLE, the BIW estimation, the Huber estimation, and the 4PLM-Robust (p** = 0.00) of these subjects are quite small, close to zero. The absolute values of the biases of the 4PLM-Robust (p** = 0.01) and the 4PLM-Robust (p** = 0.03) of the five subjects are slightly larger. Additionally, the 3PLM-MLE and the 4PLM-MLE’s biases are all negative values, whose absolute values are relatively larger, as observed in the other two sections of Table 3.

The second section of Table 3 shows the results when p** = 0.01. The absolute values of the biases of the 4PLM-Robust (p** = 0.01) are nearly zero, lower than the biases of the other methods. The absolute values of the biases of the 2PLM-MLE, the BIW and the Huber estimation are larger than those in the first section of Table 3, and the bias of the 2PLM-MLE is -0.061 at the high-ability level (θ = 2.000).

The third section of Table 3 shows the results when p** = 0.03. The absolute values of the biases of the 4PLM-Robust (p** = 0.03) are nearly zero, lower than the biases of the other methods. All of the absolute values of the biases of the 2PLM-MLE, the 3PLM-MLE, the 4PLM-MLE, the 4PLM-Robust (p** = 0.00), the 4PLM-Robust (p** = 0.01), the BIW and the Huber estimations increase, compared with the second section of Table 3. The bias of the 2PLM-MLE is-0.128 at the high-ability level (θ = 2.000), which is relatively larger.

In terms of each examinee’s absolute values of biases in Table 3, the 4PLM-Robust (p* = 0.10, p** = 0.02) approximates the Huber and BIW estimation when p** = 0.00. However, the 4PLM-Robust (p* = 0.10, p** = 0.02) is slightly larger than the Huber and BIW estimation when p** = 0.01 and p**= 0.03. That is, the 4PLM-Robust (p* = 0.10, p** = 0.02) is inferior to the Huber and BIW estimation. There are no random guessing responses in the simulation, but 4PLM-Robust (p* = 0.10, p** = 0.02) assumes that random guessing responses exist in the test, and the subjects are assumed to have guessed on the item if their expected probability of correct responding is below p* value at 0.10.

Simulation Case III: Carelessness with simultaneous random guessing

In Simulation Case III, two types of response disturbances occur simultaneously, including guessing and carelessness. In fact, they often co-occur on practice tests.

Four combinations of guessing and carelessness in the response-generated model are discussed. Combination I is p* = 0.05 and p** = 0.01 in the generated model, signifying low guessing probability and low occasional error probability in the generated responses. Combination II is p* = 0.05 and p** = 0.03 in the generated model, indicating low guessing probability and high occasional error probability in the generated responses. Combination III is p* = 0.15 and p** = 0.01, indicating high guessing probability and low occasional error probability in the generated responses. Combination IV is p* = 0.15 and p** = 0.03, indicating high guessing probability and high occasional error probability in the generated responses.

In this study, the generated model of two types of response disturbances is presented as

The approach using the 4PLM-Robust to robustify ability estimation is presented as

Table 4 shows the biases of Case III. All of the biases of the 2PLM-MLE in these four combinations at low ability (θ = -2) are larger than 0.500, which is statistically significant. In Combination II, the bias of the 2PLM-MLE is -0.247 at high ability (θ = 2), which signals a relatively large absolute value. In these four combinations, the absolute values of the biases of the BIW (B = 4) and the Huber (H = 1) are lower than those of the 2PLM-MLE. The biases of the 3PLM-MLE and the 4PLM-MLE are all negative values and below -0.350, and their absolute values are relatively larger.

Table 4

The biases in Case III: With simultaneous carelessness and random guessing.

Generated model	Method	True ability value
		-2.000	-1.000	0.000	1.000	2.000
Combinations I	2PLM-MLE	0.675	0.465	0.263	0.053	-0.086
	3PLM-MLE	-0.592	-0.592	-0.566	-0.612	-0.585
	4PLM-MLE	-0.512	-0.498	-0.450	-0.450	-0.426
	4PLM-Robust (p* = 0.05, p** = 0.01)	0.044	0.045	0.049	0.034	-0.004
	4PLM-Robust (p* = 0.05, p** = 0.03)	0.174	0.200	0.142	0.107	0.099
p* = 0.05	4PLM-Robust (p* = 0.15, p** = 0.01)	-0.098	-0.079	-0.073	-0.078	-0.069
	4PLM-Robust (p* = 0.15, p** = 0.03)	-0.040	-0.009	-0.013	-0.004	0.037
	4PLM-Robust (p* = 0.10, p** = 0.02)	-0.018	-0.004	0.012	0.005	0.016
	BIW (B = 4)	0.028	0.010	0.018	0.013	-0.035
p** = 0.01
	Huber (H = 1)	0.183	0.111	0.074	-0.019	-0.038
Combinations Ⅱ	2PLM-MLE	0.630	0.389	0.136	-0.096	-0.247
	3PLM-MLE	-0.677	-0.679	-0.711	-0.739	-0.773
	4PLM-MLE	-0.621	-0.574	-0.563	-0.566	-0.549
	4PLM-Robust (p* = 0.05, p** = 0.01)	-0.023	-0.018	-0.088	-0.091	-0.093
p* = 0.05	4PLM-Robust (p* = 0.05, p** = 0.03)	0.021	-0.014	0.029	0.042	0.002
	4PLM-Robust (p* = 0.15, p** = 0.01)	-0.148	-0.175	-0.155	-0.174	-0.195
p** = 0.03	4PLM-Robust (p* = 0.15, p** = 0.03)	-0.062	-0.051	-0.091	-0.122	-0.083
	4PLM-Robust (p* = 0.10, p** = 0.02)	-0.070	-0.088	-0.085	-0.074	-0.076
	BIW (B = 4)	-0.061	-0.061	-0.070	-0.055	-0.087
	Huber (H = 1)	0.118	0.052	-0.019	-0.077	-0.120
Combinations Ⅲ	2PLM-MLE	0.772	0.563	0.393	0.176	0.014
	3PLM-MLE	-0.532	-0.540	-0.512	-0.552	-0.549
	4PLM-MLE	-0.439	-0.417	-0.387	-0.370	-0.364
	4PLM-Robust (p* = 0.05, p** = 0.01)	0.274	0.221	0.175	0.138	0.083
	4PLM-Robust (p* = 0.05, p** = 0.03)	0.289	0.314	0.288	0.277	0.194
p* = 0.15
	4PLM-Robust (p* = 0.15, p** = 0.01)	-0.045	0.001	-0.003	-0.006	-0.012
	4PLM-Robust (p* = 0.15, p** = 0.03)	0.072	0.079	0.077	0.084	0.075
	4PLM-Robust (p* = 0.10, p** = 0.02)	0.091	0.098	0.104	0.100	0.089
p** = 0.01	BIW (B = 4)	0.105	0.084	0.102	0.086	0.024
	Huber (H = 1)	0.293	0.234	0.171	0.092	0.019
Combinations Ⅳ	2PLM-MLE	0.721	0.512	0.328	0.050	-0.136
	3PLM-MLE	-0.638	-0.644	-0.667	-0.712	-0.756
	4PLM-MLE	-0.549	-0.523	-0.515	-0.517	-0.503
	4PLM-Robust (p* = 0.05, p** = 0.01)	0.159	0.157	0.080	0.059	-0.030
	4PLM-Robust (p* = 0.05, p** = 0.03)	0.274	0.247	0.230	0.164	0.070
p* = 0.15
	4PLM-Robust (p* = 0.15, p** = 0.01)	-0.117	-0.090	-0.113	-0.113	-0.134
	4PLM-Robust (p* = 0.15, p** = 0.03)	-0.010	-0.024	-0.006	-0.005	-0.015
	4PLM-Robust (p* = 0.10, p** = 0.02)	0.017	0.016	0.056	0.015	0.013
p** = 0.03	BIW (B = 4)	0.078	0.053	0.044	0.053	-0.028
	Huber (H = 1)	0.221	0.178	0.092	0.026	-0.057

Note. (1) The biases are obtained by $$ ({\displaystyle \sum {\widehat{\theta }}_k})/5000-\theta $$ using 5,000 replications.

(2) Bold type indicates the biases that are the least of all the estimations.

In these four combinations, the absolute values of the biases of the BIW (B = 4) and the Huber (H = 1) are lower than those of the 2PLM-MLE. Furthermore, when the critical points of p* and p** in the 4PLM-Robust are equal to those in the response-generated model, the absolute values of the biases of the corresponding 4PLM-Robust are the smallest and close to zero among the four forms of the 4PLM-Robust, which also indicates that the 4PLM-Robust (p* = critical point, p** = critical point) is better than the other robust methods in the four combinations.

Especially for the 4PLM-Robust (p* = 0.10, p** = 0.02), p* and p** are also set to median value, that is, p* = 0.10, p** = 0.02. In terms of each examinee’s absolute values of biases in Table 4, 4PLM-Robust (p* = 0.10, p** = 0.02) is less than the Huber estimation and is close to or slightly less than the BIW estimation.