apache-pulsar 2.8.0 (new formula) #83238

zbentley · 2021-08-13T21:30:36Z

This commit adds a formula for the Apache Pulsar message broker
(https://pulsar.apache.org/). The broker will operate in its minimal/
default "standalone" mode when started by this formula, which is
roughly equivalent to the behavior of ZooKeeper and other similar
formulae.

Have you followed the guidelines for contributing?
Have you ensured that your commits follow the commit style guide?
Have you checked that there aren't other open pull requests for the same formula update/change?
Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
Does your build pass brew audit --strict <formula> (after doing brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

This commit adds a formula for the Apache Pulsar message broker (https://pulsar.apache.org/). The broker will operate in its minimal/ default "standalone" mode when started by this formula, which is roughly equivalent to the behavior of ZooKeeper and other similar formulae.

Formula/apache-pulsar.rb

zbentley · 2021-08-14T16:46:04Z

The arm64 CI failure is due to missing support upstream: os72/protoc-jar#93

zbentley · 2021-08-16T21:22:08Z

I'm unsure what the ubuntu-latest CI failure indicates. Does anyone know what this means?

==> FAILED
Error: 1 problem in 1 formula detected
Error: No such file or directory @ realpath_rec - /home/runner
apache-pulsar:
  * Binaries built for a non-native architecture were installed into apache-pulsar's prefix.
    The offending files are:
      /home/linuxbrew/.linuxbrew/Cellar/apache-pulsar/2.8.0/libexec/lib/presto/bin/procname/Linux-ppc64le/libprocname.so	(dunno)

SMillerDev · 2021-08-17T06:31:52Z

That means it shouldn't be installing powerpc binaries for Linux because we only support x86_64 on Linux

zbentley · 2021-08-20T18:03:15Z

After spending some time trying to get this working on M1, it seems like there are multiple significant issues blocking it working on that platform: grpc, protoc-maven, protoc itself, and rocksdb all have open issues for M1 support. Some of those have workarounds, but threading those workarounds down through Pulsar's (complicated) build system seems questionable and would require programmatically editing at least the build system, if not some of Pulsar's code.

Attempting to override the architecture to build x86_64 artifacts on ARM causes other build failures, as it appears some compiled artifacts used by this project do not honor the arch override and thus fail to link/load on M1.

That leads me to the question:
@SMillerDev @dawidd6, Is it acceptable to have this formula indicate that it does not work on M1 macs, at least until support is added upstream? If so, how would I go about doing that?

zbentley · 2021-08-20T18:03:31Z

Sorry, didn't mean to close, just comment.

SMillerDev · 2021-08-29T07:42:10Z

Formula/apache-pulsar.rb

+  def install
+    chmod "+x", "src/rename-netty-native-libs.sh"
+    with_env(
+      "JAVA_HOME" => Formula["openjdk@11"].opt_prefix,


This isn't the usual location for JAVA_HOME so I don't think this does anything.

👍, addressed!

How exactly? The line is still the same.

Unfortunately removing this caused tests to fail; I restored it in the latest commit.

Failures in 1e325fb went away when I put this line back in 0d2389d

Wait, the test block was failing? That means that it's compiling in this value. In that case especially this shouldn't be opt_prefix but overridable_java_home

No, the install step was failing because it was building on a newer, incompatible version of Java. Evidence from the CI failure in the first linked commit:

==> mvn -X clean package -DskipTests -Pcore-modules Picked up _JAVA_OPTIONS: -Duser.home=/Users/brew/Library/Caches/Homebrew/java_cache Apache Maven 3.8.2 (ea98e05a04480131370aa0c110b8c54cf726c06f) Maven home: /usr/local/Cellar/maven/3.8.2/libexec Java version: 16.0.2, vendor: Homebrew, runtime: /usr/local/Cellar/openjdk/16.0.2/libexec/openjdk.jdk/Contents/Home Default locale: en_GB, platform encoding: UTF-8 OS name: "mac os x", version: "11.3.1", arch: "x86_64", family: "mac" [DEBUG] Created new class realm maven.api

Apologies for the confusion; by "tests" I meant "formula CI in general", not brew test.

Formula/apache-pulsar.rb

zbentley · 2021-09-02T12:56:13Z

Are there additional changes that should be made to this formula before it can be merged?

Unrelated to this formula, I have a question: there is another distribution of this project, streamnative pulsar, which leads ahead of the Apache release (though they try to upstream most of their work to Apache eventually) and releases on a more regular cadence. If I wanted to make a formula for that distribution, should I a) make a tap, b) make another formula with a different name (e.g. streamnative-pulsar), or c) make a future PR suggesting modifications to this formula to switch where it gets code from?

zbentley · 2021-09-07T14:31:01Z

@SMillerDev any further changes needed here? Apologies for the nagging.

carlocab

Unrelated to this formula, I have a question: there is another distribution of this project, streamnative pulsar, which leads ahead of the Apache release (though they try to upstream most of their work to Apache eventually) and releases on a more regular cadence.

Is that a fork of this one? If so, I'd guess that that would belong in an external tap, unless it's very popular.

Formula/apache-pulsar.rb

carlocab · 2021-09-08T15:53:28Z

Formula/apache-pulsar.rb

+      system(
+        "mvn",
+        "-X",
+        "clean",
+        "package",
+        "-DskipTests",
+        "-Pcore-modules",
+      )


Can we turn this into one (or maybe two) lines?

👍 fixed!

zbentley · 2021-09-14T13:33:29Z

@carlocab @SMillerDev what additional changes are needed here?

Formula/apache-pulsar.rb

Co-authored-by: Carlo Cabrera <30379873+carlocab@users.noreply.github.com>

carlocab

Thanks, @zbentley.

BrewTestBot · 2021-09-16T02:31:34Z

@carlocab has triggered a merge.

zbentley · 2021-09-16T03:36:56Z

Thanks @carlocab and @SMillerDev for your time and assistance!

carlocab · 2021-09-16T04:00:03Z

No problem. Thanks for your patience here!

zbentley added 2 commits August 13, 2021 16:34

Address linter errors

cc5e606

BrewTestBot added java Java use is a significant feature of the PR or issue new formula PR adds a new formula to Homebrew/homebrew-core labels Aug 13, 2021

dawidd6 added the CI-force-linux [DEPRECATED] Don't pass --skip-unbottled-linux to brew test-bot. label Aug 13, 2021

dawidd6 reviewed Aug 13, 2021

View reviewed changes

Formula/apache-pulsar.rb Outdated Show resolved Hide resolved

SMillerDev reviewed Aug 14, 2021

View reviewed changes

Formula/apache-pulsar.rb Outdated Show resolved Hide resolved

Address reviewer comments

3e22f9e

chenrui333 changed the title ~~New Formula: Apache Pulsar~~ apache-pulsar 2.8.0 (new formula) Aug 14, 2021

carlocab added the CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. label Aug 15, 2021

zbentley added 3 commits August 16, 2021 11:19

Attempt to resolve CI failures: make /etc path

a57ec65

try moving directory creation to before tar command

832b3dd

Remove asterisk for config symlink

d744550

zbentley added 9 commits August 19, 2021 20:12

Remove non-optional cross compiled PPC file

59a250e

Add protobuf as build dependency

f949f6b

Attempt to force x86_64 architecture to get m1 build working

6d74191

Fix linter issues

a764ca7

Try overriding protoc command

918a4ce

Fix trailing new line

b976db0

Try a newer version of protoc

e7fbbf0

Try overriding grpc version as well

517a8d1

Give up on versioning grpc and proto forward

83ea2af

zbentley closed this Aug 20, 2021

zbentley reopened this Aug 20, 2021

Constrain arch

0e6e813

Address reviewer comments

774e4f8

SMillerDev reviewed Aug 29, 2021

View reviewed changes

zbentley mentioned this pull request Aug 29, 2021

Build/packaging: source tarballs on Apache mirror are missing executable mode for some shell scrips (e.g. rename-netty-native-libs.sh). apache/pulsar#11833

Closed

sijie mentioned this pull request Aug 29, 2021

ISSUE-11833: Build/packaging: source tarballs on Apache mirror are missing executable mode for some shell scrips (e.g. rename-netty-native-libs.sh). streamnative/pulsar-archived#2979

Closed

Link to upstream packaging issue and force JVM version

1e325fb

BrewTestBot added the no Linux bottle Formula has no Linux bottle label Aug 29, 2021

Restore JVM version pin for build

0d2389d

SMillerDev reviewed Aug 30, 2021

View reviewed changes

Formula/apache-pulsar.rb Outdated Show resolved Hide resolved

Reviewer suggestion: use utility fucntion for java home

e5133c4

BrewTestBot removed the no Linux bottle Formula has no Linux bottle label Aug 30, 2021

Fix mistaken non-string use of java_home_env

ade8e84

carlocab reviewed Sep 8, 2021

View reviewed changes

Address reviewer comments

33679a0

carlocab reviewed Sep 14, 2021

View reviewed changes

zbentley and others added 3 commits September 14, 2021 11:04

Apply suggestions from code review

0dfe3db

Co-authored-by: Carlo Cabrera <30379873+carlocab@users.noreply.github.com>

Move log creation to postinstall

e1a7780

Update Formula/apache-pulsar.rb

b8f8360

Co-authored-by: Carlo Cabrera <30379873+carlocab@users.noreply.github.com>

carlocab approved these changes Sep 16, 2021

View reviewed changes

BrewTestBot closed this in 9e70f4b Sep 16, 2021

zbentley deleted the new_formula_apache_pulsar branch September 16, 2021 03:38

zbentley restored the new_formula_apache_pulsar branch September 16, 2021 03:39

zbentley deleted the new_formula_apache_pulsar branch December 27, 2021 23:00

github-actions bot added the outdated PR was locked due to age label Jan 27, 2022

github-actions bot locked as resolved and limited conversation to collaborators Jan 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apache-pulsar 2.8.0 (new formula) #83238

apache-pulsar 2.8.0 (new formula) #83238

zbentley commented Aug 13, 2021

zbentley commented Aug 14, 2021

zbentley commented Aug 16, 2021

SMillerDev commented Aug 17, 2021

zbentley commented Aug 20, 2021

zbentley commented Aug 20, 2021

SMillerDev Aug 29, 2021

zbentley Aug 29, 2021

SMillerDev Aug 29, 2021

zbentley Aug 29, 2021

zbentley Aug 29, 2021

SMillerDev Aug 29, 2021

zbentley Aug 29, 2021

zbentley Aug 29, 2021

zbentley commented Sep 2, 2021 •

edited

Loading

zbentley commented Sep 7, 2021

carlocab left a comment

carlocab Sep 8, 2021

zbentley Sep 10, 2021

zbentley commented Sep 14, 2021

carlocab left a comment

BrewTestBot commented Sep 16, 2021

zbentley commented Sep 16, 2021

carlocab commented Sep 16, 2021 •

edited

Loading

apache-pulsar 2.8.0 (new formula) #83238

apache-pulsar 2.8.0 (new formula) #83238

Conversation

zbentley commented Aug 13, 2021

zbentley commented Aug 14, 2021

zbentley commented Aug 16, 2021

SMillerDev commented Aug 17, 2021

zbentley commented Aug 20, 2021

zbentley commented Aug 20, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zbentley commented Sep 2, 2021 • edited Loading

zbentley commented Sep 7, 2021

carlocab left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zbentley commented Sep 14, 2021

carlocab left a comment

Choose a reason for hiding this comment

BrewTestBot commented Sep 16, 2021

zbentley commented Sep 16, 2021

carlocab commented Sep 16, 2021 • edited Loading

zbentley commented Sep 2, 2021 •

edited

Loading

carlocab commented Sep 16, 2021 •

edited

Loading