agreement.RdThe default function returns the proportion of overall agreement between two or more raters. Specific agreements can be obtained as well. Specific agreement (averages over discordant cells to correct for random rater combinations). When there are 2 categories, this is equal to the positive/negative agreement. When there are more than two categories, one can either look at the agreement for one category versus the others, for example very satisfied versus rest or for one category versus one specific other category, for example very satisfies versus not satisfied.
A data frame or table with equal number of columns and rows. Or a data frame that contains the scores for each rater in each column.
A character vector indicating the category for which specific
agreement should be calculated. If length(specific) == 1, the named
factor level is compared to all others, if length(specific) == 2 the
specific agreement is of the first category level is compared to the second
category level. If specific = "positive" or
specific = "negative" and scores are dichotomous, the positive or
negative agreement will be returned.
logical vector if confidence interval for agreements are computed.
the confidence interval level
sample size; default n = nrow(data)
number of raters; default k = ncol(data)
number of bootstrap iterations for bootstrapped CI
options for sumtable if data = data.frame
An S3 object containing the proportion of overall agreement.
De confidence intervals are obtained and adjusted with a Fleis continuity correction. For the specific agreements with polytomous data, the confidence intervals are bootstrapped.
#dichotomous
df <- data.frame(r1=factor(c(1,0,1,0,0,1,1,0,0,0,1,1,0,1,1)),
r2=factor(c(1,1,1,1,0,1,1,0,0,0,1,1,0,1,0)),
r3=factor(c(1,1,1,0,0,0,1,1,1,0,0,1,0,1,1)),
r4=factor(c(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"),
levels=c("0","1"))
agreement(table)
#> overall agreement
#> 0.6333333
agreement(table, specific = "1")
#> overall agreement specific agreement: 1
#> 0.6333333 0.7317073
agreement(table, specific = "1", confint = TRUE)
#> p lower upper
#> overall agreement 0.6333333 0.2154740 0.8617275
#> specific agreement: 1 0.7317073 0.6199105 0.8259595
agreement(table, specific = "positive")
#> overall agreement positive agreement
#> 0.6333333 0.4210526
agreement(df)
#> overall agreement
#> 0.6333333
agreement(df, specific = "1")
#> overall agreement specific agreement: 1
#> 0.6333333 0.7317073
agreement(df, specific = "positive")
#> overall agreement positive agreement
#> 0.6333333 0.4210526
agreement(df, specific = "negative")
#> overall agreement negative agreement
#> 0.6333333 0.7317073
#polytomous
df <- data.frame(r1=factor(c(1,2,2,0,3,3,1,0,3,0,2,2,0,3,1)),
r2=factor(c(1,1,1,0,3,3,1,0,1,0,2,2,0,2,1)),
r3=factor(c(1,1,1,3,3,2,1,0,1,0,2,2,0,3,1)),
r4=factor(c(1,2,1,0,3,3,1,0,3,0,2,2,0,2,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"),
levels=c("0","1", "2", "3"))
agreement(table, specific = c("3", "1"))
#> overall agreement specific agreement: 3 vs 1
#> 0.7666667 0.8461538
agreement(table, specific = c("3", "1", "2"))
#> Warning: The specific agreement cannot be obtained for more than two categories; only the first argument is used and compared to all other category levels available. To avoid this warning use: specific = 3
#> overall agreement specific agreement: 3
#> 0.7666667 0.6111111
agreement(table, specific = c(3))
#> overall agreement specific agreement: 3
#> 0.7666667 0.6666667
agreement(table, specific = c("3", "1"), confint = TRUE)
#> Warning: input should be a data.frame to obtain confidence intervals for specific agreement with more than 2 factor levels; confinterval are not returned
#> overall agreement specific agreement: 3 vs 1
#> 0.7666667 0.8461538
agreement(df, specific = c("3", "1"), confint = TRUE)
#> p lower upper
#> overall agreement 0.7666667 0.6232767 0.8747149
#> specific agreement: 3 vs 1 0.8461538 0.4285714 1.0000000