-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump to JDK 11 #360
Bump to JDK 11 #360
Conversation
@@ -133,7 +133,7 @@ class KafkaSource private ( | |||
|
|||
private def createConsumer(brokers: String, groupId: String): KafkaConsumer[String, Array[Byte]] = { | |||
val properties = createProperties(brokers, groupId) | |||
properties.putAll(kafkaConfig.consumerConf.getOrElse(Map()).asJava) | |||
kafkaConfig.consumerConf.getOrElse(Map()).foreach { case (k, v) => properties.setProperty(k, v) } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it actually work? I tried it a week ago and tests were failng with different errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Initially I hit this bug scala/bug#10418 and wanted to work-around it by this line. Now I've hit another error on Class org.apache.kafka.common.serialization.StringSerializer could not be found
and trying to solve it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tests were failng with different errors.
Do you refer to unit tests? Or custom tests you performed manually? I'd like to learn more about those failing tests. @chuwy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I refer to exactly that error you're getting now. I couldn't fix it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just removed that test and CI looks happy now
4f7762f
to
9b06acd
Compare
a65b781
to
e05a0f1
Compare
@@ -178,13 +178,4 @@ lazy val beam = | |||
) | |||
.enablePlugins(JavaAppPackaging, DockerPlugin, BuildInfoPlugin) | |||
|
|||
lazy val integrationTests = project |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But we didn't delete the actual module?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This commit deletes modules/integration-tests
, did you mean something else? @chuwy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, that's what I meant - sorry didn't notice somehow.
Please create a ticket, @oguzhanunlu explaining changes and force-push the ticket-less commit. And don't forget to resolve the conflict. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @oguzhanunlu !
I believe that many OSS users use Kafka as a message queue. Would it require a lot of work to get the integration tests to work? If not we might want to invest the time.
.../src/test/scala/com.snowplowanalytics.snowplow.enrich.common/outputs/EnrichedEventSpec.scala
Outdated
Show resolved
Hide resolved
8d092e5
to
372f77a
Compare
Thanks for the comments @chuwy , I just addressed your feedback -- note that I didn't put a module scope to the issue title. @benjben , it took me several hours and I couldn't make that test compile at jdk 11. Feel free to give it a try, but I doubt it is worth the effort, re-writing from scratch would take shorter, if we'd like to offer tests for kafka module. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just gave some feedback over Slack. Otherwise - 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM but let's see with Emily if we can remove Kafka integration tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry it was my misunderstanding, we are still testing that enrich can read from and write to Kafka.
Feel free to merge, @oguzhanunlu |
Thanks @chuwy , proceeding |
No description provided.