Support for M1 Macs (osx-aarch_64) #93

cmardini · 2021-01-05T19:05:29Z

% java -jar .m2/repository/com/github/os72/protoc-jar/3.11.4/protoc-jar-3.11.4.jar -pp
------------------------------------------------------------------------
Detecting the operating system and CPU architecture
------------------------------------------------------------------------
os.detected.name: osx
os.detected.arch: aarch_64
os.detected.version: 11.1
os.detected.version.major: 11
os.detected.version.minor: 1
os.detected.classifier: osx-aarch_64
Detected platform: osx-aarch_64 (mac os x/aarch64)

Would it be possible to add to protoc-jar/bin// a protoc--osx-aarch_64.exe? Temporarily, it can probably even be a copy of protoc-*-osx-x86_64.exe as it should run on M1 macs.

I also have a brew installed protoc that seems to be compiled for arm, if you need it.

% file /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc
/opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc: Mach-O 64-bit executable arm64

% ls -l /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc
-r-xr-xr-x  1 cmardini  staff  71872  4 Jan 16:22 /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc

% md5 /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc 
MD5 (/opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc) = 0da061ae62c179f668ff4a544377cec3

The protoc-jar-maven-plugin seems to request this for the embedded protoc.

[INFO] Protoc version: 3.11.0
protoc-jar: protoc version: 3.11.0, detected platform: osx-aarch_64 (mac os x/aarch64)
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/maven-metadata.xml
protoc-jar: saved: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/google/protobuf/protoc/maven-metadata.xml
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/github/os72/protoc/maven-metadata.xml
protoc-jar: saved: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/github/os72/protoc/maven-metadata.xml
protoc-jar: cached: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/google/protobuf/protoc/maven-metadata.xml
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.11.0/protoc-3.11.0-osx-aarch_64.exe
protoc-jar: cached: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/github/os72/protoc/maven-metadata.xml
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/github/os72/protoc/3.11.0/protoc-3.11.0-osx-aarch_64.exe
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.323 s
[INFO] Finished at: 2021-01-05T09:25:11-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.os72:protoc-jar-maven-plugin:3.11.4:run (default) on project webapi-grpc-clients: Error extracting protoc for version 3.11.0: Unsupported platform: protoc-3.11.0-osx-aarch_64.exe -> [Help 1]

To workaround, I've done the following:

mvn install:install-file -DgroupId=com -DartifactId=github -Dversion=os72 -Dclassifier=3.14.0 -Dpackaging=protoc -Dfile=/opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc

and then in the plugin configuration in pom.xml:

                        <protocVersion>3.14.0</protocVersion>
                        <protocArtifact>com:github:os72:protoc:3.14.0</protocArtifact>

The text was updated successfully, but these errors were encountered:

os72 · 2021-01-06T02:55:53Z

Ack. I see. We're saying special handling for osx-aarch_64:

look for native osx-aarch_64 binary
if not found, fallback to osx-x86_64 (cause M1 platform handles compatibility)

Correct?

If you could provide clean native binaries (static build, stripped, etc), good for wide consumption, I could host them on maven central:
https://repo.maven.apache.org/maven2/com/github/os72/protoc/

You could make them available via PR:
https://github.com/os72/protoc-jar/tree/master/bin

That would be a nice contribution. And if you're using the project professionally, I'd appreciate support (sponsorship) for it..

cmardini · 2021-01-06T13:39:21Z

I think if we populate the osx-aarch_64 binary, that would be sufficient.

I've built protoc with static linkage (./configure --disable-shared) and stripped the symbols. I'll make a PR shortly.

cmardini · 2021-01-06T14:22:04Z

Added a PR: #94

wangjia184 · 2021-03-13T00:18:07Z

when can this be released?

hazmeister · 2021-05-14T09:15:44Z

Suggested workaround for the time being:

Use brew to install the protobuf cli locally brew install protobuf https://formulae.brew.sh/formula/protobuf
Set the protocCommand property to protoc

I was able to add the property to a m1 maven build profile

<properties>
    <protocCommand>protoc</protocCommand>
</properties>

nikitadol · 2021-08-02T14:23:39Z

protoc-3.17.3-osx-aarch_64.exe
https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.17.3/

vborbely · 2021-08-04T15:46:49Z

protoc-3.17.3-osx-aarch_64.exe
https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.17.3/

I created this comment for the solution.

protocolbuffers/protobuf#8062 (comment)

cmardini · 2021-08-05T14:09:15Z

As per @hazmeister's comment, I added the following profile to my pom.xml that uses protoc. Maybe it can be shortened.

  <profile>
    <build>
      <plugins>
        <plugin>
          <groupId>com.github.os72</groupId>
          <artifactId>protoc-jar-maven-plugin</artifactId>
          <executions> 
            <execution>
              <configuration>
                <protocCommand>protoc</protocCommand> <!-- brew install protobuf -->
              </configuration>
            </execution> 
          </executions>
        </plugin>
      </plugins>
    </build>
    <activation>
      <os>
        <name>mac os x</name>
        <arch>aarch64</arch>
        <family>mac</family>
      </os>
    </activation>
  </profile>
</profiles>

os72 · 2021-08-31T21:14:29Z

I'm afraid the pace in this project will remain slow until more support/sponsorship can be found (most users are enterprises)..

Rigoberto-Vazquez · 2022-01-20T17:07:29Z

in the meantime, I was able to run the process in a Apple M1 chip using the parameter "-Dos.arch=x86_64" with the maven goals

pratikraj1 · 2022-05-18T16:35:32Z

Can someone provide final step to how to resolve this on M1 mac ?

slachiewicz · 2022-05-31T21:01:12Z

simply do something like

<plugin>
       <groupId>com.github.os72</groupId>
       <artifactId>protoc-jar-maven-plugin</artifactId>
       <version>3.11.4</version>
      <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.21.1</protocArtifact>

BLACKCAT25896 · 2022-06-27T11:35:31Z

Where I can add those codes or suggestion codes?
<properties> <protocCommand>protoc</protocCommand> </properties>

os72 · 2022-06-27T18:14:24Z

Where I can add those codes or suggestion codes? <properties> <protocCommand>protoc</protocCommand> </properties>

"properties" doesn't seem right, it goes under <configuration>. example: #93 (comment)

slee500 · 2022-07-25T14:00:55Z

in the meantime, I was able to run the process in a Apple M1 chip using the parameter "-Dos.arch=x86_64" with the maven goals

@Rigoberto-Vazquez thanks for this -- works for me too.

Another way I found that works is to switch to using a x86_64 JDK instead. Seems to run just fine on my M1 Mac.

Reason94 · 2022-11-02T14:48:38Z

I found that works. Add the following configuration to maven settings.xml.

<profile>
      <id>apple-m1</id>
      <properties>
        <os.detected.classifier>osx-x86_64</os.detected.classifier>
      </properties>
</profile>

Hwloser · 2023-01-03T15:36:55Z

firstly, clone protobuf sourc code: git@github.com:protocolbuffers/protobuf.git
check out your specify version: (e.g. v3.11.4: git checkout v3.11.4 -b v3.11.4)
compile profobuf of specify version
a.(optional) ( in version v3.11.4, that must to generate configure.sh): autoconf
b.(optional) (then generate Makefile): ./configure
c. make -j $(nproc)
then you will find protoc under src directory
change the configuration <-- protoc-jar-maven-plugin -->
a. add protocCommand tag
b. example:

    <configuration>
          <protocCommand>/Users/huanwei/git_files/github/protobuf/src/protoc</protocCommand>
          <protocVersion>${protoc.version}</protocVersion>
          <inputDirectories>
                <include>src/main/resources</include>
          </inputDirectories>
     </configuration>

then re-run maven command, you will success

ennerf · 2023-01-05T22:42:13Z

Don't you only need to specify a <protocVersion> of 3.17.3 or higher?

I'm not using protoc-jar by itself, but at least for the protoc-jar-maven-plugin that is sufficient.

Protobuf 3.7.1 does not support MacOS M1 architecture, increasing the the latest protocol buffer release enables support for building this project on the newer mac models. See protoc-jar issue os72/protoc-jar#93

lassounski · 2023-07-25T08:21:17Z

Make sure that you're running on a JDK that is x86_64.

I was previously running on Zulu JDK with arch64 architecture and having that issue.

After switching to OpenJDK the problem was gone.

YuxiangZhuang · 2023-08-30T08:14:30Z

Make sure that you're running on a JDK that is x86_64.

I was previously running on Zulu JDK with arch64 architecture and having that issue.

After switching to OpenJDK the problem was gone.

it's good for me

cmardini mentioned this issue Jan 13, 2021

Support for M1 Macs (osx-aarch_64) os72/protoc-jar-maven-plugin#99

Closed

zbentley mentioned this issue Aug 14, 2021

apache-pulsar 2.8.0 (new formula) Homebrew/homebrew-core#83238

Closed

6 tasks

maver1ck mentioned this issue Nov 16, 2021

[FLINK-24964] Upgrade frontend-maven-plugin apache/flink#17795

Merged

vpejovic mentioned this issue Aug 22, 2022

Android example compilation fails at app:generateDebugProto task adap/flower#1382

Closed

dmfrey mentioned this issue Oct 28, 2022

object-detection-function tests do not run on M1 macbook pro spring-cloud/stream-applications#401

Closed

GlenKelley mentioned this issue Jan 10, 2023

Bump protobuf dependency version to enable support for M1 Macbooks ocraft/ocraft-s2client#67

Merged

Randgalt mentioned this issue Jun 28, 2023

Update protoc-jar-maven-plugin config so it can build on M1 macs trinodb/trino#18073

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for M1 Macs (osx-aarch_64) #93

Support for M1 Macs (osx-aarch_64) #93

cmardini commented Jan 5, 2021 •

edited

Loading

os72 commented Jan 6, 2021

cmardini commented Jan 6, 2021

cmardini commented Jan 6, 2021

wangjia184 commented Mar 13, 2021

hazmeister commented May 14, 2021

nikitadol commented Aug 2, 2021

vborbely commented Aug 4, 2021

cmardini commented Aug 5, 2021 •

edited

Loading

os72 commented Aug 31, 2021

Rigoberto-Vazquez commented Jan 20, 2022

pratikraj1 commented May 18, 2022

slachiewicz commented May 31, 2022

BLACKCAT25896 commented Jun 27, 2022

os72 commented Jun 27, 2022

slee500 commented Jul 25, 2022

Reason94 commented Nov 2, 2022 •

edited

Loading

Hwloser commented Jan 3, 2023 •

edited

Loading

ennerf commented Jan 5, 2023

lassounski commented Jul 25, 2023

YuxiangZhuang commented Aug 30, 2023

Support for M1 Macs (osx-aarch_64) #93

Support for M1 Macs (osx-aarch_64) #93

Comments

cmardini commented Jan 5, 2021 • edited Loading

os72 commented Jan 6, 2021

cmardini commented Jan 6, 2021

cmardini commented Jan 6, 2021

wangjia184 commented Mar 13, 2021

hazmeister commented May 14, 2021

nikitadol commented Aug 2, 2021

vborbely commented Aug 4, 2021

cmardini commented Aug 5, 2021 • edited Loading

os72 commented Aug 31, 2021

Rigoberto-Vazquez commented Jan 20, 2022

pratikraj1 commented May 18, 2022

slachiewicz commented May 31, 2022

BLACKCAT25896 commented Jun 27, 2022

os72 commented Jun 27, 2022

slee500 commented Jul 25, 2022

Reason94 commented Nov 2, 2022 • edited Loading

Hwloser commented Jan 3, 2023 • edited Loading

ennerf commented Jan 5, 2023

lassounski commented Jul 25, 2023

YuxiangZhuang commented Aug 30, 2023

cmardini commented Jan 5, 2021 •

edited

Loading

cmardini commented Aug 5, 2021 •

edited

Loading

Reason94 commented Nov 2, 2022 •

edited

Loading

Hwloser commented Jan 3, 2023 •

edited

Loading