Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-28211: Restore hive-exec-core jar #5209

Merged
merged 1 commit into from
May 3, 2024

Conversation

simhadri-g
Copy link
Member

What changes were proposed in this pull request?

The hive-exec-core jar is used by spark, oozie, hudi and many other pojects. Removal of the hive-exec-core jar has caused the following issues.

Spark : https://lists.apache.org/list?dev@hive.apache.org:lte=1M:joda
Oozie: https://lists.apache.org/thread/yld75ltf9y8d9q3cow3xqlg0fqyj6mkg
Hudi: apache/hudi#8147

Until the we shade & relocate dependencies in hive-exec, we should restore the hive-exec core jar .

Why are the changes needed?

Does this PR introduce any user-facing change?

Is the change a dependency upgrade?

How was this patch tested?

Copy link
Contributor

@zabetak zabetak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @simhadri-g, thanks for taking the initiative.

I would prefer before moving forward with this change to reach consensus in the community. Maybe its removal was a mistake but it did have some advantages. Please start or comment on an existing thread in the dev list and we can continue the discussion there.

Moreover, if we decide to restore the core jar it should be done under a separate JIRA ticket. HIVE-26220 is about shading problems so this is not strictly related. The shading problems will have to be resolved anyways.

@simhadri-g simhadri-g changed the title HIVE-26220: Restore hive-exec-core jar HIVE-28211: Restore hive-exec-core jar Apr 24, 2024
@simhadri-g
Copy link
Member Author

Sure @zabetak , i will start or continue an existing mail thread shortly.
I have created a new jira for this as well: https://issues.apache.org/jira/browse/HIVE-28211

The intention for this jira was to provide a smoother transition from the hive-exec-core to hive-exec jar for down stream projects .

Thanks!

Copy link

sonarcloud bot commented Apr 24, 2024

Quality Gate Passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@pan3793
Copy link
Member

pan3793 commented Apr 25, 2024

+1 for restoring hive-exec-core. currently hive-exec shades classes but does not relocate, which can not be consumed by lots of downstream projects, choosing hive-exec-core and handling transitive deps manually is the only choice.

Handling hive transitive deps is always a challenge for downstream projects.

forward my previous comments on google/guava#6666 (comment)

hive-exec-2.3.7.jar is actually a standalone jar, and hive-exec-2.3.7-core.jar is actually the original normal jar. Unfortunately, hive-exec-2.3.7.jar seems to be abused as a normal dependency widely.

and #4888 (comment)

Seems reducing dependencies is not a mission in the Hive project

@simhadri-g
Copy link
Member Author

Thanks @zabetak and @pan3793

I have started a mail thread on this here: https://lists.apache.org/thread/t2vzf1mydnzsckctcvfl93srrod4w76j

Thanks!

@dengzhhu653
Copy link
Member

+1 from my side, as relocating the class seems not an easy task, it requires careful tests, which may in turn causing some inconvenience and delay of upgrading to the new major release for downstream project.

Copy link
Member

@ayushtkn ayushtkn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ayushtkn
Copy link
Member

ayushtkn commented May 3, 2024

We have enough agreement on the ML as well, merging.

@ayushtkn ayushtkn merged commit 978194b into apache:master May 3, 2024
6 checks passed
dengzhhu653 pushed a commit to dengzhhu653/hive that referenced this pull request Jun 13, 2024
…dappa, reviewed by Ayush Saxena, Zhihua Deng, Cheng Pan)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants