Kendalls W by juliuspfadt · Pull Request #319 · jasp-stats/jaspReliability

juliuspfadt · 2026-05-22T16:02:52Z

adds Kendalls coefficient of concordance with the irr::kendall function.
also restructures the agreement analysis a bit
fixes jasp-stats/jasp-issues#2151

Copilot

Pull request overview

Adds Kendall’s coefficient of concordance (W) to the Reliability module’s Rater Agreement analysis (using irr::kendall()), and refactors the Rater Agreement UI/options around a shared bootstrap/CI + seed “Advanced Options” section. Also adds extensive info tooltips across multiple analyses and updates tests + versioning.

Changes:

Add Kendall’s W computation + bootstrap CI support to raterAgreement (R + QML) and extend unit tests with a published reference dataset.
Restructure Rater Agreement options (new bootstrapSamples, coefficient checkboxes default off) + add upgrade mapping for renamed option.
Add info metadata to many tables/plots/controls; bump version + update renv.lock.

Reviewed changes

Copilot reviewed 27 out of 64 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
R/raterAgreement.R	Adds Kendall’s W + placeholder table; renames bootstrap option usage; adds `info` strings.
inst/qml/RaterAgreement.qml	Adds Kendall’s W UI + “Advanced Options” section for CI/bootstrap/seed; expands allowed variable types.
tests/testthat/test-raterAgreement.R	Updates options for new defaults; adds Kendall’s W tests + reference check.
tests/testthat/anxietyRatings.csv	New reference dataset for Kendall’s W verification test.
inst/Upgrades.qml	Adds rename upgrade: `krippendorffsAlphaBootstrapSamplesForCI` → `bootstrapSamples`.
R/standardErrorOfMeasurement.R	Adds `info` metadata; adjusts `$dependOn()` wiring for CI tables/plots.
inst/qml/StandardErrorOfMeasurement.qml	Adds `info` tooltips across SEM UI controls.
R/unidimensionalReliabilityFrequentist.R	Adds `info` metadata to tables.
inst/qml/UnidimensionalReliabilityFrequentist.qml	Adds `info` tooltips and minor UI restructuring (e.g., Samples section).
R/unidimensionalReliabilityBayesian.R	Adds `info` metadata to tables/containers/plots.
inst/qml/UnidimensionalReliabilityBayesian.qml	Adds `info` tooltips across Bayesian UI controls.
R/intraclassCorrelation.R	Adds `info` metadata to main ICC table.
inst/qml/IntraclassCorrelation.qml	Adds `info` tooltips to ICC controls.
R/blandAltman.R	Adds `info` metadata to Bland–Altman containers/plots/tables.
inst/qml/BlandAltman.qml	Adds `info` tooltips to Bland–Altman controls.
NEWS.md	Adds release notes for 0.97.1 (Kendall’s W + UI restructuring).
DESCRIPTION	Bumps package version/date.
renv.lock	Updates pinned dependency metadata (incl. `jaspTools` entry) + updates JASP GitHub SHAs.
.Rbuildignore	Ignores additional dev directories (`.positai`, `.claude`).
.gitignore	Ignores `.positai`.
.vscode/settings.json	Adds workspace R LSP setting.
inst/Description.qml	Minor formatting change (blank line).
inst/help/unidimensionalReliabilityFrequentist.md	Deleted help page.
inst/help/unidimensionalReliabilityBayesian.md	Deleted help page.
inst/help/standardErrorOfMeasurement.md	Deleted help page.
inst/help/raterAgreement.md	Deleted help page.
inst/help/intraclassCorrelation.md	Deleted help page.
inst/help/toolTip/sampleSavingFreq.md	Deleted tooltip help page (replaced by `info`).
inst/help/toolTip/sampleSavingBayes.md	Deleted tooltip help page (replaced by `info`).

  jaspTable <- createJaspTable(title = "Krippendorff's alpha")
+  jaspTable$info <- gettext("Krippendorff's alpha: reliability coefficient applicable to any number of raters, any scale level (nominal/ordinal/interval/ratio), and incomplete data.")
  jaspTable$addColumnInfo(name = "method", title = gettext("Method"), type = "string")
  jaspTable$addColumnInfo(name = "kAlpha", title = "Krippendorff's alpha", type = "number")
  jaspTable$position <- 2


+.kendallWBootRA <- function(jaspResults, dataset, options, ready) {
+  if (!ready || !is.null(jaspResults[["kendallWBootstrapSamples"]]$object))
+    return()
+


+  jaspTable$addColumnInfo(name = "W",     title = gettext("W"),          type = "number")
+  jaspTable$addColumnInfo(name = "chisq", title = gettext("Chi-square"), type = "number")
+  jaspTable$addColumnInfo(name = "df",    title = gettext("df"),         type = "integer")
+  jaspTable$addColumnInfo(name = "p",     title = gettext("p"),          type = "pvalue")


+^\.positai$
+^\.claude$


FBartos · 2026-06-16T14:28:39Z

 		{
-			name:   	"krippendorffsAlpha"
-			label:  	qsTr("Krippendorff's alpha")
-			checked: 	true
-
-			DropDown
-			{
-				name:	"krippendorffsAlphaMethod"
-				label:	qsTr("Method")
-				values:
-				[
-					{label: qsTr("Nominal"),		value: "nominal"},
-					{label: qsTr("Ordinal"),		value: "ordinal"},
-					{label: qsTr("Interval"),		value: "interval"},
-					{label: qsTr("Ratio"),			value: "ratio"}
-				]
-			}
-			IntegerField
+			name:              "ci"
+			label:             qsTr("Confidence interval")
+			checked:           true
+			childrenOnSameRow: true


@juliuspfadt maybe good points?

FBartos · 2026-06-16T14:28:28Z

+			info:         qsTr("Number of bootstrap replications used to compute confidence intervals. Higher values give more stable estimates.")
 		}
+
+		SetSeed {}


@juliuspfadt maybe good points?

-  ciTable$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                   options = c("ciLevelTable", "sumScoreCiTable"))
+  ciTable$info <- gettext("Confidence intervals for the true score at each sum score level, computed from the conditional SEM.")
+  ciTable$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]])


-    plot$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                  options = "combinedPointPlot")
+    plot$info <- gettext("Overlay of all selected methods' conditional SEM across the full range of sum scores.")
+    plot$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]])


-  ciPlotsContainer$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                            options = c("sumScoreCiPlots", "sumScoreCiPlotsCutoff", "sumScoreCiPlotsCutoffValue", "ciLevelPlots"))
+  ciPlotsContainer$info <- gettext("True-score confidence interval plotted against sum score for each selected method.")
+  ciPlotsContainer$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]])



    g <- jaspGraphs::themeJasp(g)

    plot <- createJaspPlot(plot = g, title = "Posterior Predictive Check Omega", width = 350)


Copilot

Pull request overview

Copilot reviewed 27 out of 64 changed files in this pull request and generated 10 comments.

  ciTable <- createJaspTable(gettext("Sum Score CI Table"))
-  ciTable$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                   options = c("ciLevelTable", "sumScoreCiTable"))
+  ciTable$info <- gettext("Confidence intervals for the true score at each sum score level, computed from the conditional SEM.")
+  ciTable$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]])
  ciTable$position <- 2


-    plot$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                  options = "combinedPointPlot")
+    plot$info <- gettext("Overlay of all selected methods' conditional SEM across the full range of sum scores.")
+    plot$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]])


-  ciPlotsContainer$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                            options = c("sumScoreCiPlots", "sumScoreCiPlotsCutoff", "sumScoreCiPlotsCutoffValue", "ciLevelPlots"))
+  ciPlotsContainer$info <- gettext("True-score confidence interval plotted against sum score for each selected method.")
+  ciPlotsContainer$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]])


 .semPointPlots <- function(jaspResults, dataset, options, ready) {

  if (!is.null(jaspResults[["semMainContainer"]][["pointPlots"]]) || !options[["pointPlots"]]
      || !ready || jaspResults[["semMainContainer"]]$getError()) {return()}

  pointPlotsContainer <- createJaspContainer(title = gettext("Point Plots"))
+  pointPlotsContainer$info <- gettext("Conditional SEM plotted as a function of sum score for each selected method.")
  pointPlotsContainer$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]], options = "pointPlots")
  jaspResults[["semMainContainer"]][["pointPlotsContainer"]] <- pointPlotsContainer


 .semSumScoreCiPlots <- function(jaspResults, dataset, options, ready) {

  if (!is.null(jaspResults[["semMainContainer"]][["ciPlots"]]) || !options[["sumScoreCiPlots"]]
      || !ready || jaspResults[["semMainContainer"]]$getError()) {return()}

  nc <- length(unique(c(as.matrix(dataset)))) # may be needed for IRT

  ciPlotsContainer <- createJaspContainer(title = gettext("Sum Score CI Plots"))
-  ciPlotsContainer$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                            options = c("sumScoreCiPlots", "sumScoreCiPlotsCutoff", "sumScoreCiPlotsCutoffValue", "ciLevelPlots"))
+  ciPlotsContainer$info <- gettext("True-score confidence interval plotted against sum score for each selected method.")
+  ciPlotsContainer$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]])
  jaspResults[["semMainContainer"]][["ciPlotsContainer"]] <- ciPlotsContainer


  jaspTable <- createJaspTable(title = "Krippendorff's alpha")
+  jaspTable$info <- gettext("Krippendorff's alpha: reliability coefficient applicable to any number of raters, any scale level (nominal/ordinal/interval/ratio), and incomplete data.")
  jaspTable$addColumnInfo(name = "method", title = gettext("Method"), type = "string")
  jaspTable$addColumnInfo(name = "kAlpha", title = "Krippendorff's alpha", type = "number")


+.kendallWBootRA <- function(jaspResults, dataset, options, ready) {
+  if (!ready || !is.null(jaspResults[["kendallWBootstrapSamples"]]$object))
+    return()
+
+  bootstrapSamples <- createJaspState()
+  bootstrapSamples$dependOn(options = c(
+    "variables", "kendallW", "ci", "bootstrapSamples",
+    "correctForTies", "dataStructure", "setSeed", "seed"
+  ))
+  jaspResults[["kendallWBootstrapSamples"]] <- bootstrapSamples
+


+			checked:           true
+			childrenOnSameRow: true
+			id: 							 ciOpt
+			info:              qsTr("Report a bootstrap confidence interval for each agreement coefficient.")


+			label:             qsTr("Confidence interval")
+			checked:           true
+			childrenOnSameRow: true
+			id: 							 ciOpt


 ^\.github$
 ^tests/upgrades$
 ^examples$
 ^jasp_dev_work_dir$
+^\.positai$
+^\.claude$


CRAN binary for dcurver on aarch64-apple-darwin20 links against libiomp5 (Intel OpenMP), unavailable on CI (fopenmp stripped). Source install compiles without OpenMP. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jaspTools now uses getFromNamespace instead of sourcing R files, requiring jaspReliability to be loaded before analysisOptions() is called. test-blandAltman and test-intraclassCorrelation ran before library tests that incidentally loaded the package, causing only those two to fail. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…mpat)" This reverts commit ed50a8e.

FBartos · 2026-06-15T06:57:22Z

build builder bot!

FBartos · 2026-06-16T14:28:28Z

+			info:         qsTr("Number of bootstrap replications used to compute confidence intervals. Higher values give more stable estimates.")
 		}
+
+		SetSeed {}


@juliuspfadt maybe good points?

FBartos · 2026-06-16T14:28:39Z

 		{
-			name:   	"krippendorffsAlpha"
-			label:  	qsTr("Krippendorff's alpha")
-			checked: 	true
-
-			DropDown
-			{
-				name:	"krippendorffsAlphaMethod"
-				label:	qsTr("Method")
-				values:
-				[
-					{label: qsTr("Nominal"),		value: "nominal"},
-					{label: qsTr("Ordinal"),		value: "ordinal"},
-					{label: qsTr("Interval"),		value: "interval"},
-					{label: qsTr("Ratio"),			value: "ratio"}
-				]
-			}
-			IntegerField
+			name:              "ci"
+			label:             qsTr("Confidence interval")
+			checked:           true
+			childrenOnSameRow: true


@juliuspfadt maybe good points?

FBartos · 2026-06-16T14:47:38Z

build builder bot!

github-actions · 2026-06-16T14:54:41Z

✅ Build Complete!

Your JASP module bundles have been successfully built for macOS (ARM) and Windows.

👉 Click here to download the artifacts

(Scroll to the bottom of the page to find the "Artifacts" section).

Copilot

Pull request overview

Copilot reviewed 29 out of 66 changed files in this pull request and generated 3 comments.


  bootstrapSamples <- createJaspState()


+  if (!ready || !is.null(jaspResults[["kendallWBootstrapSamples"]]$object))
+    return()


  ciTable$dependOn(optionsFromObject = jaspResults[["semMainContainer"]][["coefficientTable"]],
-                   options = c("ciLevelTable", "sumScoreCiTable"))
+                   options           = c("sumScoreCiTable", "ciLevelTable"))


juliuspfadt force-pushed the kendallsW branch from 60ab1e1 to 2be69d7 Compare May 26, 2026 12:01

juliuspfadt added 7 commits June 11, 2026 11:05

stuff for kendalls W

070b1ef

restructure agreement analysis

de83311

adjust tests

d644483

change from md files to info field in qml

23eb4dc

add info to jaspResults objects

90b1535

fix kendall tbale issue

250253a

more tests for kendalls w

5ff7f7c

juliuspfadt force-pushed the kendallsW branch from 173437b to a649b22 Compare June 11, 2026 09:06

juliuspfadt requested a review from Copilot June 11, 2026 09:06

Copilot started reviewing on behalf of juliuspfadt June 11, 2026 09:06 View session

Copilot AI reviewed Jun 11, 2026

View reviewed changes

juliuspfadt force-pushed the kendallsW branch from a649b22 to f81e7c8 Compare June 11, 2026 09:28

fix tests and add more info

cf18504

juliuspfadt force-pushed the kendallsW branch from f81e7c8 to cf18504 Compare June 11, 2026 09:39

juliuspfadt requested a review from Copilot June 11, 2026 09:45

Copilot started reviewing on behalf of juliuspfadt June 11, 2026 09:45 View session

Copilot AI reviewed Jun 11, 2026

View reviewed changes

juliuspfadt force-pushed the kendallsW branch 2 times, most recently from ef4171f to 354a13e Compare June 11, 2026 12:11

fix copilot feedback

a480ed5

juliuspfadt force-pushed the kendallsW branch from 354a13e to a480ed5 Compare June 11, 2026 12:39

juliuspfadt and others added 3 commits June 11, 2026 15:05

fix: force dcurver source install to avoid OpenMP CI failure

35af3c2

CRAN binary for dcurver on aarch64-apple-darwin20 links against libiomp5 (Intel OpenMP), unavailable on CI (fopenmp stripped). Source install compiles without OpenMP. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Revert "fix: load package before tests (jaspTools getFromNamespace co…

baf13bd

…mpat)" This reverts commit ed50a8e.

juliuspfadt requested a review from FBartos June 11, 2026 15:02

update jaspTools

426d03d

FBartos requested changes Jun 16, 2026

View reviewed changes

fix ci id checked stuff

cbb4262

juliuspfadt requested review from FBartos and Copilot June 16, 2026 14:54

Copilot started reviewing on behalf of juliuspfadt June 16, 2026 14:55 View session

Copilot AI reviewed Jun 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Kendalls W#319

Kendalls W#319
juliuspfadt wants to merge 14 commits into
jasp-stats:masterfrom
juliuspfadt:kendallsW

juliuspfadt commented May 22, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

FBartos Jun 16, 2026

Uh oh!

FBartos Jun 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

FBartos commented Jun 15, 2026

Uh oh!

FBartos Jun 16, 2026

Uh oh!

FBartos Jun 16, 2026

Uh oh!

FBartos commented Jun 16, 2026

Uh oh!

github-actions Bot commented Jun 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		g <- jaspGraphs::themeJasp(g)

		plot <- createJaspPlot(plot = g, title = "Posterior Predictive Check Omega", width = 350)

		if (!ready \|\| !is.null(jaspResults[["kendallWBootstrapSamples"]]$object))
		return()

		^\.positai$
		^\.claude$

Uh oh!

Conversation

juliuspfadt commented May 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

FBartos Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

FBartos Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

FBartos commented Jun 15, 2026

Uh oh!

FBartos Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

FBartos Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

FBartos commented Jun 16, 2026

Uh oh!

github-actions Bot commented Jun 16, 2026

✅ Build Complete!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants