TEZ-4688 TEZ-4648: Tez upgrade to Hadoop 3.5.0 and jersey 2.x by maheshrajus · Pull Request #474 · apache/tez

maheshrajus · 2026-04-06T06:35:21Z

What changes were proposed in this PR?

This PR upgrades Tez to support:

Hadoop 3.5.0
Jersey 2.x

As part of the upgrade, several ATS Timeline API response format differences were observed between older Hadoop versions and Hadoop 3.5.0. This PR adds compatibility handling for those schema variations.

Main updates include:

Migrated Jersey client usage from Jersey 1.x APIs to Jersey 2.x APIs
Added handling for ATS field name variations observed across versions
Handled ATS compatibility differences including:
otherinfo ↔ otherInfo
entity ↔ entityId
entitytype ↔ entityType
domain / id ↔ domainId

Additional fixes:

Handled nested Timeline domain JSON structures
Stabilized ATS-related unit tests after Jersey 2.x migration

How was this patch tested?

with existed UT and all unit tests are green.

jersey upgrade changes taken from PR : #429 by @abstractdog as they need for hadoop 3.5.0 upgrade.

kokila-19 · 2026-04-06T06:44:38Z

FYI,
https://issues.apache.org/jira/browse/TEZ-4648 Jersey upgrade is also required for hadoop upgrade

maheshrajus · 2026-04-14T05:27:26Z

@abstractdog @ayushtkn Can you please review and provide comments if any. Thanks !

Aggarwal-Raghav · 2026-04-14T15:53:13Z

@maheshrajus , can you please rebase this PR and resolve merge conflicts?

maheshrajus · 2026-04-14T15:57:13Z

@Aggarwal-Raghav Hi, i rebased and pushed the commit just now. Could you please take a look at this? Thanks !

Aggarwal-Raghav · 2026-04-14T18:11:57Z

The explicit dependency of org.codehaus.jettison should be removed from tez and moved to use fasterxml IMO, the question is whether it should be done in this JIRA scope or a separate JIRA.

maheshrajus · 2026-04-22T06:08:58Z

The explicit dependency of org.codehaus.jettison should be removed from tez and moved to use fasterxml IMO, the question is whether it should be done in this JIRA scope or a separate JIRA.

@Aggarwal-Raghav Better this we will handle separately. thanks !

tez-yetus · 2026-04-22T21:31:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 41s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 36s		master passed
+1 💚	compile	7m 42s		master passed
+1 💚	checkstyle	5m 45s		master passed
+1 💚	javadoc	6m 13s		master passed
+0 🆗	spotbugs	1m 32s		tez-api in master has 545 extant spotbugs warnings.
+0 🆗	spotbugs	1m 47s		tez-dag in master has 749 extant spotbugs warnings.
+0 🆗	spotbugs	0m 50s		tez-tests in master has 6 extant spotbugs warnings.
+0 🆗	spotbugs	0m 47s		tez-ext-service-tests in master has 62 extant spotbugs warnings.
+0 🆗	spotbugs	0m 46s		tez-plugins/tez-protobuf-history-plugin in master has 26 extant spotbugs warnings.
+0 🆗	spotbugs	0m 43s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 39s		tez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 41s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 45s		tez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗	spotbugs	0m 44s		tez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	6m 58s		root in master has 1935 extant spotbugs warnings.
+0 🆗	spotbugs	0m 43s		tez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 18s	/patch-mvninstall-tez-tests.txt	tez-tests in the patch failed.
+1 💚	codespell	1m 48s		No new issues.
+1 💚	compile	7m 45s		the patch passed
+1 💚	javac	7m 45s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 27s		tez-api: The patch generated 0 new + 34 unchanged - 4 fixed = 34 total (was 38)
+1 💚	checkstyle	0m 37s		The patch passed checkstyle in tez-dag
+1 💚	checkstyle	0m 26s		The patch passed checkstyle in tez-tests
+1 💚	checkstyle	0m 23s		The patch passed checkstyle in tez-ext-service-tests
+1 💚	checkstyle	0m 23s		The patch passed checkstyle in tez-protobuf-history-plugin
+1 💚	checkstyle	0m 24s		The patch passed checkstyle in tez-yarn-timeline-history
+1 💚	checkstyle	0m 23s		The patch passed checkstyle in tez-yarn-timeline-history-with-acls
+1 💚	checkstyle	0m 23s		The patch passed checkstyle in tez-yarn-timeline-history-with-fs
-0 ⚠️	checkstyle	0m 25s	/results-checkstyle-tez-plugins_tez-history-parser.txt	tez-plugins/tez-history-parser: The patch generated 1 new + 186 unchanged - 1 fixed = 187 total (was 187)
+1 💚	checkstyle	0m 25s		The patch passed checkstyle in tez-aux-services
+1 💚	checkstyle	0m 25s		The patch passed checkstyle in job-analyzer
-0 ⚠️	checkstyle	1m 0s	/results-checkstyle-root.txt	root: The patch generated 1 new + 309 unchanged - 5 fixed = 310 total (was 314)
+1 💚	javadoc	6m 14s		the patch passed
+1 💚	spotbugs	1m 44s		tez-api generated 0 new + 544 unchanged - 1 fixed = 544 total (was 545)
+1 💚	spotbugs	2m 4s		tez-dag in the patch passed.
+1 💚	spotbugs	1m 3s		tez-tests in the patch passed.
+1 💚	spotbugs	1m 1s		tez-ext-service-tests in the patch passed.
+1 💚	spotbugs	0m 59s		tez-protobuf-history-plugin in the patch passed.
+1 💚	spotbugs	0m 57s		tez-yarn-timeline-history in the patch passed.
+1 💚	spotbugs	0m 51s		tez-yarn-timeline-history-with-acls in the patch passed.
+1 💚	spotbugs	0m 53s		tez-yarn-timeline-history-with-fs in the patch passed.
+1 💚	spotbugs	0m 58s		tez-history-parser in the patch passed.
+1 💚	spotbugs	0m 58s		tez-aux-services in the patch passed.
+1 💚	spotbugs	1m 3s		job-analyzer in the patch passed.
+1 💚	spotbugs	8m 6s		root generated 0 new + 1934 unchanged - 1 fixed = 1934 total (was 1935)
			_ Other Tests _
+1 💚	unit	2m 28s		tez-api in the patch passed.
+1 💚	unit	5m 46s		tez-dag in the patch passed.
+1 💚	unit	40m 59s		tez-tests in the patch passed.
+1 💚	unit	4m 30s		tez-ext-service-tests in the patch passed.
+1 💚	unit	0m 40s		tez-protobuf-history-plugin in the patch passed.
+1 💚	unit	1m 49s		tez-yarn-timeline-history in the patch passed.
+1 💚	unit	1m 57s		tez-yarn-timeline-history-with-acls in the patch passed.
+1 💚	unit	1m 46s		tez-yarn-timeline-history-with-fs in the patch passed.
+1 💚	unit	2m 34s		tez-history-parser in the patch passed.
+1 💚	unit	3m 2s		tez-aux-services in the patch passed.
+1 💚	unit	2m 54s		job-analyzer in the patch passed.
+1 💚	unit	72m 35s		root in the patch passed.
+1 💚	asflicense	5m 32s		The patch does not generate ASF License warnings.
		245m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/7/artifact/out/Dockerfile
GITHUB PR	#474
Optional Tests	dupname asflicense javac javadoc unit codespell detsecrets xmllint compile spotbugs checkstyle
uname	Linux 79a8c2db6e7b 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/tez-multibranch_PR-474/src/.yetus/personality.sh
git revision	master / `76a32cb`
Default Java	Ubuntu-21.0.10+7-Ubuntu-124.04
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/7/testReport/
Max. process+thread count	1443 (vs. ulimit of 5500)
modules	C: tez-api tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-plugins/tez-aux-services tez-tools/analyzers/job-analyzer . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/7/console
versions	git=2.43.0 maven=3.9.14 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

maheshrajus · 2026-04-27T05:23:44Z

@abstractdog @ayushtkn @Aggarwal-Raghav
Could you please review and approve the PR at your convenience?
Thank you !

abstractdog · 2026-06-08T09:05:39Z

the latest job (https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/9/console) failed with:

09:23:54  ERROR: failed to build: failed to solve: process "/bin/ash -o pipefail -c apk add --no-cache curl     && curl -sSL [https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64](https://github.com/hadolint/hadolint/releases/download/v$%7BHADOLINT_VERSION%7D/hadolint-Linux-x86_64) -o /bin/hadolint     && chmod a+rx /bin/hadolint     && echo \"${HADOLINT_SHA}  /bin/hadolint\" | sha512sum -c -" did not complete successfully: exit code: 1
...
09:23:56  #2 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

which is strange as my job is currently successfully passing this point (https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/6/)

@Aggarwal-Raghav: you had similar failure recently, can you recall if it was a flaky thing or if it was resolved with some magic, rebase, etc.?

Aggarwal-Raghav · 2026-06-08T10:00:02Z

the latest job (https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/9/console) failed with:
09:23:54  ERROR: failed to build: failed to solve: process "/bin/ash -o pipefail -c apk add --no-cache curl     && curl -sSL [https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64](https://github.com/hadolint/hadolint/releases/download/v$%7BHADOLINT_VERSION%7D/hadolint-Linux-x86_64) -o /bin/hadolint     && chmod a+rx /bin/hadolint     && echo \"${HADOLINT_SHA}  /bin/hadolint\" | sha512sum -c -" did not complete successfully: exit code: 1
...
09:23:56  #2 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
which is strange as my job is currently successfully passing this point (https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/6/)

@Aggarwal-Raghav: you had similar failure recently, can you recall if it was a flaky thing or if it was resolved with some magic, rebase, etc.?

my last run also passed (https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-507/1/console), @maheshrajus , please try to force push again.
@abstractdog , issue i faced, maybe you are referring to #490 (comment), it was because of maven version moving from dlcdn -> archive i guess.

maheshrajus · 2026-06-08T12:23:46Z

@abstractdog @Aggarwal-Raghav After force push the CI pipeline is running fine. Thanks !
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/10/

tez-yetus · 2026-06-08T14:26:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 7s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 42s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 51s		master passed
+1 💚	compile	3m 59s		master passed
+1 💚	checkstyle	4m 22s		master passed
+1 💚	javadoc	4m 50s		master passed
+0 🆗	spotbugs	1m 22s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 40s		tez-dag in master has 537 extant spotbugs warnings.
+0 🆗	spotbugs	0m 47s		tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗	spotbugs	0m 42s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 38s		tez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 43s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 45s		tez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗	spotbugs	6m 53s		root in master has 1054 extant spotbugs warnings.
+0 🆗	spotbugs	0m 46s		tez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 7s		the patch passed
+1 💚	codespell	1m 47s		No new issues.
+1 💚	compile	4m 0s		the patch passed
-1 ❌	javac	4m 0s	/results-compile-javac-root.txt	root generated 1 new + 738 unchanged - 4 fixed = 739 total (was 742)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 27s	/results-checkstyle-tez-api.txt	tez-api: The patch generated 2 new + 38 unchanged - 0 fixed = 40 total (was 38)
-0 ⚠️	checkstyle	0m 23s	/results-checkstyle-tez-plugins_tez-history-parser.txt	tez-plugins/tez-history-parser: The patch generated 2 new + 186 unchanged - 1 fixed = 188 total (was 187)
-0 ⚠️	checkstyle	1m 0s	/results-checkstyle-root.txt	root: The patch generated 4 new + 313 unchanged - 1 fixed = 317 total (was 314)
+1 💚	javadoc	5m 3s		the patch passed
+1 💚	spotbugs	16m 58s		the patch passed
			_ Other Tests _
+1 💚	unit	71m 21s		root in the patch passed.
+1 💚	asflicense	4m 1s		The patch does not generate ASF License warnings.
		158m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/10/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense javac javadoc codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 7822942470b9 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `330fdc8`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/10/testReport/
Max. process+thread count	1363 (vs. ulimit of 5500)
modules	C: tez-api tez-dag tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-tools/analyzers/job-analyzer . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/10/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

tez-yetus · 2026-06-08T18:26:46Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 41s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 44s		master passed
+1 💚	compile	3m 55s		master passed
+1 💚	checkstyle	4m 28s		master passed
+1 💚	javadoc	4m 59s		master passed
+0 🆗	spotbugs	1m 24s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 41s		tez-dag in master has 537 extant spotbugs warnings.
+0 🆗	spotbugs	0m 47s		tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗	spotbugs	0m 44s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 38s		tez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 41s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 46s		tez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗	spotbugs	6m 51s		root in master has 1054 extant spotbugs warnings.
+0 🆗	spotbugs	0m 45s		tez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 2s		the patch passed
+1 💚	codespell	1m 44s		No new issues.
+1 💚	compile	4m 0s		the patch passed
+1 💚	javac	4m 0s		root generated 0 new + 738 unchanged - 4 fixed = 738 total (was 742)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 28s		The patch passed checkstyle in tez-api
+1 💚	checkstyle	0m 36s		The patch passed checkstyle in tez-dag
+1 💚	checkstyle	0m 24s		The patch passed checkstyle in tez-protobuf-history-plugin
+1 💚	checkstyle	0m 24s		The patch passed checkstyle in tez-yarn-timeline-history
+1 💚	checkstyle	0m 24s		The patch passed checkstyle in tez-yarn-timeline-history-with-acls
+1 💚	checkstyle	0m 24s		The patch passed checkstyle in tez-yarn-timeline-history-with-fs
+1 💚	checkstyle	0m 25s		tez-plugins/tez-history-parser: The patch generated 0 new + 185 unchanged - 2 fixed = 185 total (was 187)
+1 💚	checkstyle	0m 25s		The patch passed checkstyle in job-analyzer
+1 💚	checkstyle	1m 0s		root: The patch generated 0 new + 312 unchanged - 2 fixed = 312 total (was 314)
+1 💚	javadoc	5m 2s		the patch passed
+1 💚	spotbugs	16m 15s		the patch passed
			_ Other Tests _
+1 💚	unit	72m 37s		root in the patch passed.
+1 💚	asflicense	4m 22s		The patch does not generate ASF License warnings.
		160m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/12/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense javac javadoc codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 9201ede3f89c 5.15.0-174-generic #184-Ubuntu SMP Fri Mar 13 18:41:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `cba5689`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/12/testReport/
Max. process+thread count	1446 (vs. ulimit of 5500)
modules	C: tez-api tez-dag tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-tools/analyzers/job-analyzer . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/12/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

maheshrajus · 2026-06-09T06:04:21Z

@abstractdog @Aggarwal-Raghav I addressed all your review comments. Could you please take a look and approve it when you have a moment?
Thank you !

abstractdog

thanks @maheshrajus for the updated PR, left some comments, this is getting close!

maheshrajus · 2026-06-09T15:46:20Z

thanks @maheshrajus for the updated PR, left some comments, this is getting close!

@abstractdog I addressed all your review comments. Could you please take a look and approve it when you have a moment? Thank you !

maheshrajus · 2026-06-09T15:57:31Z

@ayushtkn Could you please review and approve when you have a moment? Thank you !

tez-yetus · 2026-06-09T18:21:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 51s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 58s		master passed
+1 💚	compile	4m 1s		master passed
+1 💚	checkstyle	5m 34s		master passed
+1 💚	javadoc	6m 5s		master passed
+0 🆗	spotbugs	1m 25s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 39s		tez-dag in master has 537 extant spotbugs warnings.
+0 🆗	spotbugs	0m 51s		tez-tests in master has 6 extant spotbugs warnings.
+0 🆗	spotbugs	0m 48s		tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗	spotbugs	0m 43s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 39s		tez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 41s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 46s		tez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗	spotbugs	0m 45s		tez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	6m 53s		root in master has 1054 extant spotbugs warnings.
+0 🆗	spotbugs	0m 44s		tez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 8s		the patch passed
+1 💚	codespell	1m 48s		No new issues.
+1 💚	compile	4m 4s		the patch passed
+1 💚	javac	4m 4s		root generated 0 new + 738 unchanged - 4 fixed = 738 total (was 742)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 24s	/results-checkstyle-tez-plugins_tez-history-parser.txt	tez-plugins/tez-history-parser: The patch generated 1 new + 186 unchanged - 1 fixed = 187 total (was 187)
-0 ⚠️	checkstyle	1m 0s	/results-checkstyle-root.txt	root: The patch generated 1 new + 313 unchanged - 1 fixed = 314 total (was 314)
+1 💚	javadoc	6m 3s		the patch passed
+1 💚	spotbugs	19m 21s		the patch passed
			_ Other Tests _
+1 💚	unit	71m 21s		root in the patch passed.
+1 💚	asflicense	5m 35s		The patch does not generate ASF License warnings.
		163m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/13/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense javac javadoc codespell detsecrets xmllint spotbugs checkstyle
uname	Linux f2f7f53ddfa9 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `cba5689`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/13/testReport/
Max. process+thread count	1468 (vs. ulimit of 5500)
modules	C: tez-api tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-plugins/tez-aux-services tez-tools/analyzers/job-analyzer . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/13/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

maheshrajus · 2026-06-10T05:44:15Z

@abstractdog Look like the checkstyle task is catching incorrect one here. 'If' child has Indentation level 2. In this line i have not added any spaces except entityID string. Anything i am missing over here ?

./tez-plugins/tez-history-parser/src/main/java/org/apache/tez/history/parser/SimpleHistoryParser.java:258: userName = subEntity.getString(Constants.ENTITY_ID);:15: 'if' child has incorrect indentation level 14, expected level should be 16. [Indentation]

Code:

            if (subEntityType != null && subEntityType.equals(Constants.USER)) {
              userName = subEntity.getString(Constants.ENTITY_ID);
              break;
            }

Link: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/13/artifact/out/results-checkstyle-root.txt

abstractdog · 2026-06-10T09:03:30Z

@abstractdog Look like the checkstyle task is catching incorrect one here. 'If' child has Indentation level 2. In this line i have not added any spaces except entityID string. Anything i am missing over here ?

./tez-plugins/tez-history-parser/src/main/java/org/apache/tez/history/parser/SimpleHistoryParser.java:258: userName = subEntity.getString(Constants.ENTITY_ID);:15: 'if' child has incorrect indentation level 14, expected level should be 16. [Indentation]

Code:
            if (subEntityType != null && subEntityType.equals(Constants.USER)) {
              userName = subEntity.getString(Constants.ENTITY_ID);
              break;
            }
Link: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-474/13/artifact/out/results-checkstyle-root.txt

we can ignore that checkstyle warning then, for me it's clearly a bug in checkstyle
we use 2 spaces, and it applies to branches of if/else too

maheshrajus · 2026-06-10T11:13:06Z

we can ignore that checkstyle warning then, for me it's clearly a bug in checkstyle
we use 2 spaces, and it applies to branches of if/else too

@abstractdog Thank you for confirming the Checkstyle-related.
Could you please take a look and approve the PR when you have a moment? Thank you!

abstractdog

LGTM

Aggarwal-Raghav · 2026-06-10T13:06:41Z

@abstractdog , please wait for some time before merging, I'm checking few things. Give me 1 hr.

abstractdog · 2026-06-10T13:07:58Z

@abstractdog , please wait for some time before merging, I'm checking few things. Give me 1 hr.

absolutely! I'm not about to merge this immediately
thanks if you take another look at this

Aggarwal-Raghav · 2026-06-10T13:11:19Z

I might be wrong, I was performing cluster testing of this change, and saw this error. In Hadoop jersery migration they have moved from javax.servlet-api -> jakarta.servlet-api, In tez we have direct dependency on javax.servlet-api.
Just checking if it valid or specific to my setup

Caused by: [CIRCULAR REFERENCE: java.lang.NoSuchMethodError: 'javax.servlet.Filter javax.servlet.ServletContext.createFilter(java.lang.Class)']
2026-06-10 18:24:47,042 [ERROR] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |web.WebUIService|: Tez UI WebService failed to start.
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:504)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:484)
	at org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:115)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:195)
	at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1857)
	at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1878)
Caused by: java.io.IOException: Problem starting http server
	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1489)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:501)
	... 5 more

Attaching complete log.
logs.txt

Aggarwal-Raghav · 2026-06-11T09:58:21Z

I might be wrong, I was performing cluster testing of this change, and saw this error. In Hadoop jersery migration they have moved from javax.servlet-api -> jakarta.servlet-api, In tez we have direct dependency on javax.servlet-api. Just checking if it valid or specific to my setup

Caused by: [CIRCULAR REFERENCE: java.lang.NoSuchMethodError: 'javax.servlet.Filter javax.servlet.ServletContext.createFilter(java.lang.Class)']
2026-06-10 18:24:47,042 [ERROR] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |web.WebUIService|: Tez UI WebService failed to start.
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:504)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:484)
	at org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:115)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:195)
	at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1857)
	at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1878)
Caused by: java.io.IOException: Problem starting http server
	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1489)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:501)
	... 5 more

Attaching complete log. logs.txt

@maheshrajus , did you face this in your env?

maheshrajus · 2026-06-11T09:59:04Z

I might be wrong, I was performing cluster testing of this change, and saw this error. In Hadoop jersery migration they have moved from javax.servlet-api -> jakarta.servlet-api, In tez we have direct dependency on javax.servlet-api. Just checking if it valid or specific to my setup

Caused by: [CIRCULAR REFERENCE: java.lang.NoSuchMethodError: 'javax.servlet.Filter javax.servlet.ServletContext.createFilter(java.lang.Class)']
2026-06-10 18:24:47,042 [ERROR] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |web.WebUIService|: Tez UI WebService failed to start.
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:504)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:484)
	at org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:115)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:195)
	at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1857)
	at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1878)
Caused by: java.io.IOException: Problem starting http server
	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1489)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:501)
	... 5 more

Attaching complete log. logs.txt

@Aggarwal-Raghav Could you please try executing the same steps with my latest changes and confirm? I have fixed the Tez UI-related ATS fields according to the new behavior in Hadoop 3.5.0. With these changes, I am able to submit a DAG and view its status in the Tez UI. I have attached the screenshots above for reference. Thank you !
cc: @abstractdog

maheshrajus · 2026-06-11T11:06:01Z

@maheshrajus , did you face this in your env?

@Aggarwal-Raghav i got the below NoSuchMethodError(BufferRecycler.releaseToPool()) exception while testing tez-ui. Similar stack trace i found in attached logs.txt by you. With Latest code changes i cant find any exceptions and able to see the DAG status in tez UI. You also please verify once and confirm. Thank you !

BufferRecycler.releaseToPool() this method available only in jackson-core 2.18.x. We can find the fix changes in my latest pushed commit.

Stack trace:


Caused by: java.lang.NoSuchMethodError: 'void com.fasterxml.jackson.core.util.BufferRecycler.releaseToPool()'
	at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:4075)
	at org.apache.hadoop.yarn.api.records.timeline.writer.TimelineDomainWriter.writeTo(TimelineDomainWriter.java:56)
	at org.apache.hadoop.yarn.api.records.timeline.writer.TimelineDomainWriter.writeTo(TimelineDomainWriter.java:40)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
	at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
	at org.glassfish.jersey.client.ClientRequest.doWriteEntity(ClientRequest.java:468)
	at org.glassfish.jersey.client.ClientRequest.writeEntity(ClientRequest.java:450)
	at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:444)
	at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:286)
	at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:300)

maheshrajus · 2026-06-11T11:14:21Z

The DAG has been submitted and the status is visible in the Tez UI with the latest fixes.

@Aggarwal-Raghav Please review and confirm the changes so @abstractdog can merge them at his convenience. Thanks!

This comment was marked as outdated.

Sign in to view

maheshrajus changed the title ~~TEZ-4688: Upgrade to Hadoop 3.5.0~~ TEZ-4688 TEZ-4648: [WIP] Tez upgrade to Hadoop 3.5.0 and jersey 2.x Apr 6, 2026

maheshrajus force-pushed the TEZ-4688 branch from 89bff1e to 6c8930e Compare April 6, 2026 16:51

This comment was marked as outdated.

Sign in to view

maheshrajus force-pushed the TEZ-4688 branch from 6c8930e to 2f29572 Compare April 7, 2026 11:09

This comment was marked as outdated.

Sign in to view

maheshrajus force-pushed the TEZ-4688 branch from 2f29572 to 4e5b9ae Compare April 8, 2026 06:46

maheshrajus changed the title ~~TEZ-4688 TEZ-4648: [WIP] Tez upgrade to Hadoop 3.5.0 and jersey 2.x~~ TEZ-4688 TEZ-4648: Tez upgrade to Hadoop 3.5.0 and jersey 2.x Apr 8, 2026

This comment was marked as outdated.

Sign in to view

maheshrajus force-pushed the TEZ-4688 branch from 4e5b9ae to c7c7221 Compare April 8, 2026 11:20

This comment was marked as outdated.

Sign in to view

maheshrajus force-pushed the TEZ-4688 branch from c7c7221 to 8f159f9 Compare April 14, 2026 15:55

Aggarwal-Raghav reviewed Apr 14, 2026

View reviewed changes

Comment thread pom.xml

Aggarwal-Raghav reviewed Apr 14, 2026

View reviewed changes

Comment thread tez-api/src/main/java/org/apache/tez/dag/api/client/TimelineReaderFactory.java

Aggarwal-Raghav reviewed Apr 14, 2026

View reviewed changes

Comment thread tez-plugins/tez-history-parser/src/main/java/org/apache/tez/history/ATSImportTool.java

Aggarwal-Raghav reviewed Apr 14, 2026

View reviewed changes

Comment thread pom.xml Outdated

Aggarwal-Raghav reviewed Apr 14, 2026

View reviewed changes

Comment thread pom.xml

This comment was marked as outdated.

Sign in to view

maheshrajus mentioned this pull request Apr 15, 2026

[WIP] TEZ-4677: JDK25 testing in yetus #482

Open

maheshrajus requested a review from Aggarwal-Raghav April 22, 2026 17:25

abstractdog requested review from abstractdog and removed request for Aggarwal-Raghav May 18, 2026 06:53

maheshrajus force-pushed the TEZ-4688 branch from d6cf892 to f4391fd Compare June 8, 2026 11:47

maheshrajus force-pushed the TEZ-4688 branch 2 times, most recently from ab4b69f to 43765a7 Compare June 8, 2026 15:46

abstractdog requested changes Jun 9, 2026

View reviewed changes

Review comments fix 3

e325ba9

maheshrajus force-pushed the TEZ-4688 branch from 43765a7 to e325ba9 Compare June 9, 2026 15:37

maheshrajus requested a review from abstractdog June 9, 2026 15:40

abstractdog approved these changes Jun 10, 2026

View reviewed changes

Aggarwal-Raghav reviewed Jun 10, 2026

View reviewed changes

Comment thread tez-api/src/main/java/org/apache/tez/common/ATSConstants.java

Fix Tez UI timeline parsing after Hadoop 3.5.0 upgrade

c37ce34

maheshrajus requested a review from Aggarwal-Raghav June 11, 2026 09:44

Conversation

maheshrajus commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this PR?

How was this patch tested?

Uh oh!

kokila-19 commented Apr 6, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

maheshrajus commented Apr 14, 2026

Uh oh!

Aggarwal-Raghav commented Apr 14, 2026

Uh oh!

maheshrajus commented Apr 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Aggarwal-Raghav commented Apr 14, 2026

Uh oh!

Uh oh!

This comment was marked as outdated.

maheshrajus commented Apr 22, 2026

Uh oh!

tez-yetus commented Apr 22, 2026

Uh oh!

maheshrajus commented Apr 27, 2026

Uh oh!

abstractdog commented Jun 8, 2026

Uh oh!

Aggarwal-Raghav commented Jun 8, 2026

Uh oh!

maheshrajus commented Jun 8, 2026

Uh oh!

tez-yetus commented Jun 8, 2026

Uh oh!

tez-yetus commented Jun 8, 2026

Uh oh!

maheshrajus commented Jun 9, 2026

Uh oh!

abstractdog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maheshrajus commented Jun 9, 2026

Uh oh!

maheshrajus commented Jun 9, 2026

Uh oh!

tez-yetus commented Jun 9, 2026

Uh oh!

maheshrajus commented Jun 10, 2026

Uh oh!

abstractdog commented Jun 10, 2026

Uh oh!

maheshrajus commented Jun 10, 2026

Uh oh!

abstractdog left a comment

Choose a reason for hiding this comment

Uh oh!

Aggarwal-Raghav commented Jun 10, 2026

Uh oh!

abstractdog commented Jun 10, 2026

Uh oh!

Aggarwal-Raghav commented Jun 10, 2026

Uh oh!

Uh oh!

Aggarwal-Raghav commented Jun 11, 2026

Uh oh!

maheshrajus commented Jun 11, 2026

Uh oh!

maheshrajus commented Apr 6, 2026 •

edited

Loading

maheshrajus commented Jun 11, 2026 •

edited

Loading