Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump to JDK 11 #360

Merged
merged 2 commits into from
Sep 30, 2020
Merged

Bump to JDK 11 #360

merged 2 commits into from
Sep 30, 2020

Conversation

oguzhanunlu
Copy link
Member

No description provided.

@oguzhanunlu oguzhanunlu self-assigned this Sep 21, 2020
@@ -133,7 +133,7 @@ class KafkaSource private (

private def createConsumer(brokers: String, groupId: String): KafkaConsumer[String, Array[Byte]] = {
val properties = createProperties(brokers, groupId)
properties.putAll(kafkaConfig.consumerConf.getOrElse(Map()).asJava)
kafkaConfig.consumerConf.getOrElse(Map()).foreach { case (k, v) => properties.setProperty(k, v) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it actually work? I tried it a week ago and tests were failng with different errors.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I hit this bug scala/bug#10418 and wanted to work-around it by this line. Now I've hit another error on Class org.apache.kafka.common.serialization.StringSerializer could not be found and trying to solve it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests were failng with different errors.

Do you refer to unit tests? Or custom tests you performed manually? I'd like to learn more about those failing tests. @chuwy

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refer to exactly that error you're getting now. I couldn't fix it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just removed that test and CI looks happy now

@oguzhanunlu
Copy link
Member Author

@benjben @chuwy if you agree with the changes, I'd like to publish rc1 and see it in action

@@ -178,13 +178,4 @@ lazy val beam =
)
.enablePlugins(JavaAppPackaging, DockerPlugin, BuildInfoPlugin)

lazy val integrationTests = project
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we didn't delete the actual module?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit deletes modules/integration-tests, did you mean something else? @chuwy

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that's what I meant - sorry didn't notice somehow.

@chuwy
Copy link
Contributor

chuwy commented Sep 23, 2020

Please create a ticket, @oguzhanunlu explaining changes and force-push the ticket-less commit.

And don't forget to resolve the conflict.

@chuwy chuwy self-requested a review September 23, 2020 17:04
Copy link
Contributor

@benjben benjben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @oguzhanunlu !

I believe that many OSS users use Kafka as a message queue. Would it require a lot of work to get the integration tests to work? If not we might want to invest the time.

@oguzhanunlu
Copy link
Member Author

oguzhanunlu commented Sep 24, 2020

Thanks for the comments @chuwy , I just addressed your feedback -- note that I didn't put a module scope to the issue title. @benjben , it took me several hours and I couldn't make that test compile at jdk 11. Feel free to give it a try, but I doubt it is worth the effort, re-writing from scratch would take shorter, if we'd like to offer tests for kafka module.

Copy link
Contributor

@chuwy chuwy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just gave some feedback over Slack. Otherwise - 👍

Copy link
Contributor

@benjben benjben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but let's see with Emily if we can remove Kafka integration tests.

@benjben benjben self-requested a review September 30, 2020 09:10
Copy link
Contributor

@benjben benjben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry it was my misunderstanding, we are still testing that enrich can read from and write to Kafka.

@chuwy
Copy link
Contributor

chuwy commented Sep 30, 2020

Feel free to merge, @oguzhanunlu

@oguzhanunlu
Copy link
Member Author

Thanks @chuwy , proceeding

@oguzhanunlu oguzhanunlu merged commit d3f3520 into develop Sep 30, 2020
@oguzhanunlu oguzhanunlu deleted the jdk11 branch September 30, 2020 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants