io.js all v2.3.4 - v3.0.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) #2147

ghost · 2015-07-09T19:41:14Z

I'm building io.sj v2.3.4 on linux/64.

I've built & installed a local instance of openssl

    openssl version
        OpenSSL 1.0.2d 9 Jul 2015

in

    tree -d /usr/local/ssl
        /usr/local/ssl
        ├── bin
        ├── certs
        ├── include
        │   └── openssl
        ├── lib64
        │   ├── engines
        │   └── pkgconfig
        ├── man
        │   ├── man1
        │   ├── man3
        │   ├── man5
        │   └── man7
        ├── misc
        └── private

I want to build io.js against that ^^ instance, ensuring proper RUNTIME-linking against the provided libs, NOT system-defaults in other locations, therefore necessitating rpath.

Currently, for io.js,

    ./configure --help
        ...
            --shared-openssl    link to a shared OpenSSl DLL instead of static linking
            --shared-openssl-includes=SHARED_OPENSSL_INCLUDES
                                directory containing OpenSSL header files
            --shared-openssl-libname=SHARED_OPENSSL_LIBNAME
                                alternative lib name to link to [default: crypto,ssl]
            --shared-openssl-libpath=SHARED_OPENSSL_LIBPATH
                                a directory to search for the shared OpenSSL DLLs
        ...

Configuring with

    ./configure \
    --prefix=/usr/local/io.js \
     --dest-cpu=x64 \
     --dest-os=linux \
    --shared-openssl \
    --shared-openssl-includes=/usr/local/ssl/include \
    --shared-openssl-libpath=/usr/local/ssl/lib64 \
    --shared-openssl-libname=crypto,ssl
    --shared-zlib

returns

    creating  ./icu_config.gypi
    { 'target_defaults': { 'cflags': [],
                           'default_configuration': 'Release',
                           'defines': [],
                           'include_dirs': ['/usr/local/ssl/include'],
                           'libraries': [ '-I/usr/local/ssl/include',
                                          '-lssl',
                                          '-lcrypto']},
      'variables': { 'host_arch': 'x64',
                     'icu_small': 'false',
                     'node_install_npm': 'true',
                     'node_prefix': '/usr/local/io.js',
                     'node_shared_http_parser': 'false',
                     'node_shared_libuv': 'false',
                     'node_shared_openssl': 'true',
                     'node_shared_zlib': 'false',
                     'node_tag': '',
                     'node_use_dtrace': 'false',
                     'node_use_etw': 'false',
                     'node_use_lttng': 'false',
                     'node_use_openssl': 'true',
                     'node_use_perfctr': 'false',
                     'openssl_fips': '',
                     'openssl_no_asm': 0,
                     'python': '/usr/bin/python',
                     'target_arch': 'x64',
                     'uv_parent_path': '/deps/uv/',
                     'uv_use_dtrace': 'false',
                     'v8_enable_gdbjit': 0,
                     'v8_enable_i18n_support': 0,
                     'v8_no_strict_aliasing': 1,
                     'v8_optimized_debug': 0,
                     'v8_random_seed': 0,
                     'v8_use_snapshot': 1,
                     'want_separate_host_toolset': 0}}
    creating  ./config.gypi
    creating  ./config.mk

which appears to

    (1) neglect the "-I" flag for 'include_dirs'

        ...
        'include_dirs': ['/usr/local/ssl/include'],
        ...

and

    (2) incorrectly reference CFLAGS

        '-I/usr/local/ssl/include'

    rather than the correct/typical LDFLAGS addition,

        '-L/usr/local/ssl/lib64'

    as per above at
        ...
        'libraries': [ '-I/usr/local/ssl/include',
                      '-lssl',
                      '-lcrypto']},
        ...

Which, of course, causes a build FAIL

    ./configre ...
    make -C out BUILDTYPE=Release V=1 -j
        ...
        /usr/bin/g++-5 -pthread -rdynamic -m64 -Wl,-z,noexecstack -Wl,--whole-archive /usr/local/src/io.js/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a -Wl,--no-whole-archive -pthread  -o /usr/local/src/io.js/out/Release/iojs -Wl,--start-group /usr/local/src/io.js/out/Release/obj.target/iojs/src/debug-agent.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/async-wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/env.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/fs_event_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/cares_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/handle_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/js_stream.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_buffer.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_constants.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_contextify.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_file.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_http_parser.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_javascript.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_main.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_os.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_v8.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_stat_watcher.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_watchdog.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_zlib.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_i18n.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/pipe_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/signal_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/smalloc.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/spawn_sync.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/string_bytes.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/stream_base.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/stream_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/tcp_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/timer_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/tty_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/process_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/udp_wrap.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/uv.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/util.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_crypto.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_crypto_bio.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_crypto_clienthello.o /usr/local/src/io.js/out/Release/obj.target/iojs/src/tls_wrap.o /usr/local/src/io.js/out/Release/obj.target/deps/cares/libcares.a /usr/local/src/io.js/out/Release/obj.target/deps/v8/tools/gyp/libv8_libplatform.a /usr/local/src/io.js/out/Release/obj.target/deps/zlib/libzlib.a /usr/local/src/io.js/out/Release/obj.target/deps/http_parser/libhttp_parser.a /usr/local/src/io.js/out/Release/obj.target/deps/uv/libuv.a /usr/local/src/io.js/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a /usr/local/src/io.js/out/Release/obj.target/deps/v8/tools/gyp/libv8_libbase.a /usr/local/src/io.js/out/Release/obj.target/deps/v8/tools/gyp/libv8_nosnapshot.a -Wl,--end-group -I/usr/local/ssl/include -lssl -lcrypto -lrt -lm -ldl
        /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_crypto.o: In function `node::crypto::VerifyCallback(int, x509_store_ctx_st*)':
        node_crypto.cc:(.text._ZN4node6crypto14VerifyCallbackEiP17x509_store_ctx_st[_ZN4node6crypto14VerifyCallbackEiP17x509_store_ctx_st]+0x73): undefined reference to `SSL_is_server'
        /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_crypto.o: In function `node::crypto::Connection::New(v8::FunctionCallbackInfo<v8::Value> const&)':
        node_crypto.cc:(.text._ZN4node6crypto10Connection3NewERKN2v820FunctionCallbackInfoINS2_5ValueEEE+0x347): undefined reference to `SSL_set_cert_cb'
        node_crypto.cc:(.text._ZN4node6crypto10Connection3NewERKN2v820FunctionCallbackInfoINS2_5ValueEEE+0x48e): undefined reference to `SSL_set_cert_cb'
        /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_crypto.o: In function `node::crypto::SSLWrap<node::crypto::Connection>::CertCbDone(v8::FunctionCallbackInfo<v8::Value> const&)':
        node_crypto.cc:(.text._ZN4node6crypto7SSLWrapINS0_10ConnectionEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE[_ZN4node6crypto7SSLWrapINS0_10ConnectionEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE]+0xf0): undefined reference to `SSL_CTX_get0_certificate'
        node_crypto.cc:(.text._ZN4node6crypto7SSLWrapINS0_10ConnectionEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE[_ZN4node6crypto7SSLWrapINS0_10ConnectionEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE]+0xfc): undefined reference to `SSL_CTX_get0_privatekey'
        /usr/local/src/io.js/out/Release/obj.target/iojs/src/node_crypto.o: In function `node::crypto::SSLWrap<node::TLSWrap>::CertCbDone(v8::FunctionCallbackInfo<v8::Value> const&)':
        node_crypto.cc:(.text._ZN4node6crypto7SSLWrapINS_7TLSWrapEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE[_ZN4node6crypto7SSLWrapINS_7TLSWrapEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE]+0xf0): undefined reference to `SSL_CTX_get0_certificate'
        node_crypto.cc:(.text._ZN4node6crypto7SSLWrapINS_7TLSWrapEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE[_ZN4node6crypto7SSLWrapINS_7TLSWrapEE10CertCbDoneERKN2v820FunctionCallbackInfoINS4_5ValueEEE]+0xfc): undefined reference to `SSL_CTX_get0_privatekey'
        /usr/local/src/io.js/out/Release/obj.target/iojs/src/tls_wrap.o: In function `node::TLSWrap::InitSSL()':
        tls_wrap.cc:(.text._ZN4node7TLSWrap7InitSSLEv+0xf3): undefined reference to `SSL_set_cert_cb'
        collect2: error: ld returned 1 exit status
        iojs.target.mk:196: recipe for target '/usr/local/src/io.js/out/Release/iojs' failed
        make: *** [/usr/local/src/io.js/out/Release/iojs] Error 1
        make: Leaving directory '/usr/local/src/io.js/out'

Further, there's no obvious mechanism for adding the necessary rpath'ing flags to ensure the aforementioned linking.

For node.js 0.12.x, a simple src patch does the trick

    cat ./nodejs-openssl-rpath.patch
        --- configure.ORIG  2015-03-03 19:37:28.534636561 -0800
        +++ configure   2015-03-03 19:43:53.517334786 -0800
        @@ -695,7 +695,15 @@
               o['include_dirs'] += [options.shared_openssl_includes]
             else:
               o['cflags'] += cflags.split()
        -
        +      o['ldflags'] = [ '-L/usr/local/ssl/lib64',
        +        '-Wl,-rpath,/usr/local/ssl/lib64',
        +        '-lssl',
        +        '-lcrypto',
        +        '-L/usr/local/lib64',
        +        '-Wl,-rpath,/usr/local/lib64',
        +        '-lpcre',
        +      ]
        +

         def configure_fullystatic(o):
           if options.fully_static:

    patch -p0 < ./nodejs-openssl-rpath.patch

so that node's built & properly runtime-linked,

    node -v
        v0.12.6
    ldd `which node` | egrep -i "crypto|ssl"
            libssl.so.1.0.0 => /usr/local/ssl/lib64/libssl.so.1.0.0 (0x00007f688cd14000)
            libcrypto.so.1.0.0 => /usr/local/ssl/lib64/libcrypto.so.1.0.0 (0x00007f688c8ca000)

Looking through io.js' ./configure, it's clearly a departure from node.js', and it's not at all clear, atm, how/where to make a similar patch.

The text was updated successfully, but these errors were encountered:

ghost · 2015-07-09T20:24:37Z

After the aforementioned initial build FAIL

after ./configure,

    ./configure \
    --prefix=/usr/local/io.js \
     --dest-cpu=x64 \
     --dest-os=linux \
    --shared-openssl \
    --shared-openssl-includes=/usr/local/ssl/include \
    --shared-openssl-libpath=/usr/local/ssl/lib64 \
    --shared-openssl-libname=crypto,ssl \
    --shared-zlib

a workaround patch directly to config.gypi,

    ~/iojs-openssl-rpath.patch
        --- config.gypi.ORIG    2015-07-09 13:12:47.676319731 -0700
        +++ config.gypi 2015-07-09 13:14:35.621840956 -0700
        @@ -4,9 +4,13 @@
                                'defines': [],
                                'include_dirs': ['/usr/local/ssl/include'],
                                'libraries': [ '-lz',
        -                                      '-I/usr/local/ssl/include',
        +                                      '-L/usr/local/ssl/lib64',
        +                                      '-Wl,-rpath,/usr/local/ssl/lib64',
                                               '-lssl',
        -                                      '-lcrypto']},
        +                                      '-lcrypto',
        +                                      '-L/usr/local/lib64',
        +                                      '-Wl,-rpath,/usr/local/lib64',
        +                                      '-lpcre']},
           'variables': { 'host_arch': 'x64',
                          'icu_small': 'false',
                          'node_install_npm': 'true',

    patch -p0 < ~/iojs-openssl-rpath.patch
        patching file config.gypi

does the trick. now,

    make -C out BUILDTYPE=Release V=1 -j4
        ...
        make: Leaving directory '/usr/local/src/io.js/out'

    ldd out/Release/iojs | egrep "ssl|crypto|pcre"
        libssl.so.1.0.0 => /usr/local/ssl/lib64/libssl.so.1.0.0 (0x00007f7f5f875000)
        libcrypto.so.1.0.0 => /usr/local/ssl/lib64/libcrypto.so.1.0.0 (0x00007f7f5f42c000)
        libpcre.so.1 => /usr/local/lib64/libpcre.so.1 (0x00007f9db9db7000)

    out/Release/iojs --version
        v2.3.4
    make test
        ...
        Total errors found: 0
        make[1]: Leaving directory '/usr/local/src/io.js'
    make install
    which npm node
        /usr/local/io.js/bin/npm
        /usr/local/io.js/bin/node
    node -v && npm -v
        v2.3.4
        2.12.1
    node -p "process.versions.openssl"
        1.0.2d

Of course, this needs to be generalized to the io.js configure script ...

rvagg · 2015-07-10T00:38:09Z

/cc @nodejs/build @jbergstroem you've been working in this area haven't you?

jbergstroem · 2015-07-10T03:59:33Z

Yeah, this is up my alley. I'll have a look.

ghost · 2015-07-18T20:36:34Z

no new news -- just fyi, same with v2.4.0

jbergstroem · 2015-07-20T08:32:57Z

I swear I posted a diff here last week. Gah - I'll find it.

jbergstroem · 2015-07-20T23:42:42Z

Quick question; @pgnd: do you have pkg-config installed in your system? At the moment iojs prefers that before anything else.

ghost · 2015-07-21T00:01:24Z

@jbergstroem

yes, I do.

Note that 'my' ssl is installed in a NON-system location. I.e.,

    cd /usr/local/ssl
    find . | grep "\.pc"
        ./lib64/pkgconfig/libssl.pc
        ./lib64/pkgconfig/openssl.pc
        ./lib64/pkgconfig/libcrypto.pc

    cat ./lib64/pkgconfig/libssl.pc
        prefix=/usr/local/ssl
        exec_prefix=${prefix}
        libdir=${exec_prefix}/lib64
        includedir=${prefix}/include

        Name: OpenSSL-libssl
        Description: Secure Sockets Layer and cryptography libraries
        Version: 1.0.2d
        Requires.private: libcrypto
        Libs: -L${libdir} -lssl
        Libs.private: -Wl,-z,relro,-z,now -ldl -lz
        Cflags: -I${includedir} 

    cat ./lib64/pkgconfig/openssl.pc
        prefix=/usr/local/ssl
        exec_prefix=${prefix}
        libdir=${exec_prefix}/lib64
        includedir=${prefix}/include

        Name: OpenSSL
        Description: Secure Sockets Layer and cryptography libraries and tools
        Version: 1.0.2d
        Requires: libssl libcrypto

    cat ./lib64/pkgconfig/libcrypto.pc
        prefix=/usr/local/ssl
        exec_prefix=${prefix}
        libdir=${exec_prefix}/lib64
        includedir=${prefix}/include

        Name: OpenSSL-libcrypto
        Description: OpenSSL cryptography library
        Version: 1.0.2d
        Requires: 
        Libs: -L${libdir} -lcrypto
        Libs.private: -Wl,-z,relro,-z,now -ldl -lz
        Cflags: -I${includedir}

and these are NOT, and shouldn't be in the system pkg-config path,

    env | grep PKG
        PKG_CONFIG=/usr/bin/pkg-config
        PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig

rather, in the system path, are the system ssl libs/includes,

    pkg-config libssl --libs
        -lssl -lcrypto 
    pkg-config libcrypto --libs
        -lcrypto 
    pkg-config openssl --libs
        -lssl -lcrypto

which are specifically what I am not linking against, and what these configure-options are supposed enable overriding of pkg-config-derived values.

jbergstroem · 2015-07-21T00:33:19Z

Yeah, just wanted to make it clear that you will have to either unset pkg-config from path or similar ($i_know_what_im_doing) if you want to make this work. I'll post you a diff for testing shortly.

ghost · 2015-07-21T01:17:50Z

either unset pkg-config from path or similar ($i_know_what_im_doing) if you want to make this work

Big 'thumbs up' on NOT trying to to be 'auto-magical' about it; you'll never cover all the possible cases.

In effect a simple SSL_LIBS & SSL_INCLUDES 'manual override', ensuring the option to add the rpath cruft -- and then making sure that that's what configure actually picks up and inserts into the makefiles is all that's really needed.

jbergstroem · 2015-07-21T01:32:44Z

@pgnd we already allow override with flags (so, not 100% autoconf) so that'd be good enough. The behaviour for overridden flags has just changed recently which needs to be adjusted. Finally, I don't think we'll override pkg-config output from flags, rather suggest these selected few just pass /dev/null to PKG_CONFIG or similar. Thanks for your elaborate input, btw.

ghost · 2015-07-21T06:41:04Z

heh, "elaborate". shutting up now.

will test / report when you post. thx.

ghost · 2015-07-22T15:30:09Z

What's changed in the existing 2.4.0 brach?

Now

cd io.js-build
git log | head
    commit a179bcbe9e82791246fda21625455614a0c3f149
    Author: Jeremiah Senkpiel <fishrock123@rocketmail.com>
    Date:   Fri Jul 17 16:19:32 2015 -0700

        2015-07-17 io.js v2.4.0 Release

        Notable changes

        * src: Added a new `--track-heap-objects` flag to track heap object
        allocations for heap snapshots (Bradley Meck)

./configure \
--prefix=/usr/local/io.js \
 --dest-cpu=x64 \
 --dest-os=linux \
--shared-openssl \
--shared-openssl-includes=/usr/local/ssl/include \
--shared-openssl-libpath=/usr/local/ssl/lib64 \
--shared-openssl-libname=crypto,ssl \
--shared-zlib

cat config.gypi
    # Do not edit. Generated by the configure script.
    { 'target_defaults': { 'cflags': [],
                           'default_configuration': 'Release',
                           'defines': [],
                           'include_dirs': [],
                           'libraries': ['-lz', '-lssl', '-lcrypto']},
      'variables': { 'host_arch': 'x64',
                     'icu_small': 'false',
                     'node_install_npm': 'true',
                     'node_prefix': '/usr/local/io.js',
                     'node_shared_http_parser': 'false',
                     'node_shared_libuv': 'false',
                     'node_shared_openssl': 'true',
                     'node_shared_zlib': 'true',
                     'node_tag': '',
                     'node_use_dtrace': 'false',
                     'node_use_etw': 'false',
                     'node_use_lttng': 'false',
                     'node_use_openssl': 'true',
                     'node_use_perfctr': 'false',
                     'openssl_fips': '',
                     'openssl_no_asm': 0,
                     'python': '/usr/bin/python',
                     'target_arch': 'x64',
                     'uv_parent_path': '/deps/uv/',
                     'uv_use_dtrace': 'false',
                     'v8_enable_gdbjit': 0,
                     'v8_enable_i18n_support': 0,
                     'v8_no_strict_aliasing': 1,
                     'v8_optimized_debug': 0,
                     'v8_random_seed': 0,
                     'v8_use_snapshot': 1,
                     'want_separate_host_toolset': 0}}

with no reference at all to my spec'd ssl libs/includes,

and a build with this config now appears to build using the bundled source's openssl in ./deps/openssl/openssl , not my specified libs/includes.

E.g.

...
/usr/bin/gcc-5 '-DL_ENDIAN' '-DPURIFY' '-D_REENTRANT' '-DOPENSSL_NO_SSL2' '-DOPENSSL_NO_SSL3' '-DOPENSSL_NO_HEARTBEATS' '-DENGINESDIR="/dev/null"' '-DTERMIOS' '-DOPENSSLDIR="/etc/ssl"' '-DMONOLITH' -I../deps/openssl -I../deps/openssl/openssl -I../deps/openssl/openssl/crypto -I../deps/openssl/openssl/crypto/asn1 -I../deps/openssl/openssl/crypto/evp -I../deps/openssl/openssl/crypto/md2 -I../deps/openssl/openssl/crypto/modes -I../deps/openssl/openssl/crypto/store -I../deps/openssl/openssl/include  -Wno-missing-field-initializers -pthread -Wall -Wextra -Wno-unused-parameter -m64 -O3 -ffunction-sections -fdata-sections -fno-omit-frame-pointer  -MMD -MF /root/rpmbuild/BUILD/iojs-2.4.0/out/Release/.deps//root/rpmbuild/BUILD/iojs-2.4.0/out/Release/obj.target/openssl-cli/deps/openssl/openssl/apps/rsautl.o.d.raw -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fmessage-length=0 -grecord-gcc-switches -march=x86-64 -mtune=nocona -c -o /root/rpmbuild/BUILD/iojs-2.4.0/out/Release/obj.target/openssl-cli/deps/openssl/openssl/apps/rsautl.o ../deps/openssl/openssl/apps/rsautl.c
...

jbergstroem · 2015-07-22T22:48:25Z

Looking at commit history I don't see anything related to above, but the problem you're experiencing is based on this commit since we no longer pick up the defaults, even if passed.

ghost · 2015-07-22T23:29:06Z

The commit removes the defaults handling, but iiuc should not interfere with passing the flags on the configure line, right?

I'm not clear if a solution has been posted to my OP, or if we're still pending.

If it's supposed to be fixed already, then -- it's not working.

If it's pending, then I just need to wait some more.

jbergstroem · 2015-07-23T00:56:44Z

@pgnd passing flags through configure line would only override the default (through optparse). We need to reintroduce that logic. I'm starting to feel that the only way to really solve this is either moving closer to how autoconf works (i.e, juggling environment variables, pkg-config, default and override flags) or perhaps exposing further configuration through config.mk/config.gypi so people can edit the files post-configuration (we're probably talking <1% of users).

ghost · 2015-07-23T01:14:39Z

@jbergstroem If it were up to me (it's not), if any move were to be made, I'd not recommend autofoo, but rather cmake. That's not a small change, I understand.

If OTOH the "modify config.mk/config.gypi" approach is taken for any necessary options, fine with me TBH. That's allow you to simply rip out the 'find SSL' cruft, and use the bundled openssl. +1 on simpler-is-better.

For 'advanced' config, then, 3 things would be needed

(1) a clear, editable set of FLAGS exposed in config.{mk,gypi} (pretty close already afaict0
(2) ensure that the build system recognizes and uses changes made, rather than using its internal defaults (currently not working)
(3) documentation of the method, and override hierarchy

ghost · 2015-07-29T22:25:46Z

release 2.5.0 returns back to the originally reported fail mode,

./configure \
--prefix=/usr/local/io.js \
 --dest-cpu=x64 \
 --dest-os=linux \
--shared-openssl \
--shared-openssl-includes=/usr/local/ssl/include \
--shared-openssl-libpath=/usr/local/ssl/lib64 \
--shared-openssl-libname=crypto,ssl \
--shared-zlib
    ...
    creating  ./icu_config.gypi
    { 'target_defaults': { 'cflags': [],
                           'default_configuration': 'Release',
                           'defines': [],
                           'include_dirs': ['/usr/local/ssl/include'],
                           'libraries': [ '-lz',
?                                        '-I/usr/local/ssl/include',
                                          '-lssl',
                                          '-lcrypto']},
    ...

rvagg · 2015-07-29T23:21:22Z

Bringing in @bnoordhuis to this discussion as he may have some interest in helping resolve this. I believe @jbergstroem is a little busy with other things atm.

jbergstroem · 2015-07-30T06:03:16Z

Sorry, just travelling at the moment.

@pgnd I didn't have any problems editing config.gypi between compiles (I tried switching shared flags). Did you? If not, I'd suggest thats how we move forward for more exotic combinations which would require better documentation.

ghost · 2015-07-30T12:48:03Z

@jbergstroem

No, no problems editing. Just noting that it changed, again, from 2.4.0 -> 2.5.0

At the very least though, this should be corrected

...
'libraries': [ '-lz',
?                                        '-I/usr/local/ssl/include',
...

It's just plain wrong.

jbergstroem · 2015-07-30T14:40:01Z

Ok, I'll create a PR.

ghost · 2015-08-12T01:35:17Z

fell free to reopen if others' interest continues

fazalmajid · 2015-09-30T00:14:54Z

I am encountering the same issue with 4.0.0 and 4.1.1.

cephasjohnson · 2015-11-11T21:01:33Z

Short version: I'm making node from 5.0.0 source on Ubuntu 14.04.3 LTS (GNU/Linux 3.10.53-1.1.0_ga-wandboard-06034-g13bb184-dirty armv7l). Configure options --shared-openssl-includes, --shared-openssl-libpath and --shared-openssl and the same options for zlib don't work as advertised. This may be due to --start-group and --end-group options used in the command that links node. The same failure occurs when building from 4.2.2 source.

The command that links node in 4.2.2 and 5.0.0 source differs from the command that links node in 0.12.x source.

Longer version:

There are config options that allow building node with locally installed zlib and openssl. I've installed zlib and openssl locally and want node to use the associated libraries so I config with:

./configure --prefix=/usr/local/node-5.0.0 --shared-openssl-includes=/usr/local/ssl/include --shared-openssl-libpath=/usr/local/ssl/lib   --shared-openssl   --shared-zlib-includes=/usr/local/include --shared-zlib-libpath=/usr/local/lib   --shared-zlib

which results in:

creating  ./icu_config.gypi
{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [ '/usr/local/include',
                                         '/usr/local/ssl-1.0.2d/include'],
                       'libraries': [ '-I/usr/local/include',
                                      '-lz',
                                      '-I/usr/local/ssl-1.0.2d/include',
                                      '-lssl',
                                      '-lcrypto']},
  'variables': { 'arm_float_abi': 'hard',
                 'arm_fpu': 'vfpv3',
                 'arm_thumb': 0,
                 'arm_version': '7',
                 'asan': 0,
                 'host_arch': 'arm',
                 'icu_small': 'false',
                 'node_byteorder': 'little',
                 'node_install_npm': 'true',
                 'node_prefix': '/usr/local/node-5.0.0',
                 'node_release_urlbase': '',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_openssl': 'true',
                 'node_shared_zlib': 'true',
                 'node_tag': '',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_perfctr': 'false',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'python': '/usr/bin/python',
                 'target_arch': 'arm',
                 'uv_parent_path': '/deps/uv/',
                 'uv_use_dtrace': 'false',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 0,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_random_seed': 0,
                 'v8_use_snapshot': 1,
                 'want_separate_host_toolset': 0}}
creating  ./config.gypi
creating  ./config.mk

The "libraries" stanza in above is wrong. It should be:

                   'libraries': [ '-L/usr/local/lib',
                                  '-lz',
                                  '-L/usr/local/ssl-1.0.2d/lib',
                                  '-lssl',
                                  '-lcrypto']},

During the subsequent make, the compiler references openssl 1.0.2 header files in -I/usr/local/ssl-1.0.2d/include/. The link command however doesn't reference associated libraries /usr/local/ssl-1.0.2d/lib/ssl.o and crypto.o, possibly because of --start-group ... --end-group options in the link command (see 4. below).

ldd then shows out/Release/node uses these libraries:

ubuntu@wandboard:~/src/nodejs/node-v5.0.0$ ldd out/Release/node
        libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0x76f4d000)
        libssl.so.1.0.0 => /lib/arm-linux-gnueabihf/libssl.so.1.0.0 (0x76f0d000)
        libcrypto.so.1.0.0 => /lib/arm-linux-gnueabihf/libcrypto.so.1.0.0 (0x76e01000)
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x76df6000)
        librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0x76de8000)
        libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0x76d3e000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76cd1000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x76cb0000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76c95000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76bae000)
        /lib/ld-linux-armhf.so.3 (0x76f7d000)
ubuntu@wandboard:~/src/nodejs/node-v5.0.0$

whereas the desired ldd output should show references to locally installed libraries (note I have /usr/local/ssl/lib linked to /usr/local/ssl-1.0.2d/lib):

    libz.so.1 => /usr/local/lib/libz.so.1
libssl.so.1.0.0 => /usr/local/ssl/lib/libssl.so.1.0.0.
libcrypto.so.1.0.0 => /usr/local/ssl/lib/libcrypto.so.1.0.0

Running "make test" generates errors, e.g.:

out/Release/node: symbol lookup error: out/Release/node: undefined symbol: SSL_set_cert_cb

nm shows the undefined symbol isn't in the system version of libssl.so (which was linked into node) but is in the locally installed version of libssl.so (which wasn't, but should have been, linked into node):

~$ /usr/bin/nm -D /lib/arm-linux-gnueabihf/libssl.so.1.0.0|grep  SSL_set_cert_cb                                                               
ubuntu@wandboard:~$ /usr/bin/nm -D /usr/local/ssl/lib/libssl.so.1.0.0|grep SSL_set_cert_cb                                                                     
0002e118 T SSL_set_cert_cb                                                      
ubuntu@wandboard:~$

The ld man page says --start-group ... --end-group means "The specified archives are searched repeatedly until no new undefined references are created." The command that links node is of the form:

g++ -pthread -rdynamic -Wl,-z,noexecstack -Wl,--whole-archive /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a -Wl,--no-whole-archive -pthread -o /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/node -Wl,--start-group /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/obj.target/node/src/debug-agent.o ... <LOTS MORE REFERENCES TO .o AND .a FILES> ... /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/obj.target/deps/v8/tools/gyp/libv8_nosnapshot.a -Wl,--end-group -L/usr/local/lib -lz -L/usr/local/ssl-1.0.2d/lib -lssl -lcrypto -ldl -lrt -lm

which SEEMS to mean -lcrypto -lssl -lz -ldl -lrt -lm that occur after --end-group will never be searched to resolve undefined references in all the archives listed between --start-group and --end-group. In turn the node executable includes undefined symbols (one of which is SSL_set_cert_cb). Subsequently the runtime linker can't resolve the undefineds cause it hasn't been told to look in /usr/local/ssl/lib/.

If -lcrypto -lssl -lz aren't being searched, a POSSIBLE solution is a link command of the form:

g++ -pthread -rdynamic -Wl,-z,noexecstack -Wl,--whole-archive /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a -Wl,--no-whole-archive -pthread -o /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/node -L/usr/local/lib -L/usr/local/ssl-1.0.2d/lib -Wl,--start-group /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/obj.target/node/src/debug-agent.o ... <LOTS MORE REFERENCES TO .o AND .a FILES> ... /home/ubuntu/src/nodejs/node-v5.0.0/out/Release/obj.target/deps/v8/tools/gyp/libv8_nosnapshot.a -lz -lssl -lcrypto -ldl -lrt -lm -Wl,--end-group

A side note: while trying to make node use locally installed openssl, pkg-config couldn't find openssl.pc cause that .pc file isn't in a default pkg-config search path and isn't in a path specified by environmental PGK_CONFIG_PATH. I added /usr/local/ssl/lib/pgkconfig/ to environmental PGK_CONFIG_PATH so pkg-config would find that .pc file.

cephasjohnson · 2015-11-19T04:19:35Z

Making node from 5.0.0 source using locally installed openssl and zlib works if config.gypi and Makefile are edited. For example, configure node with:

./configure --prefix=/usr/local/node-5.0.0 --shared-openssl-includes=/usr/local/ssl/include --shared-openssl-libpath=/usr/local/ssl/lib --shared-openssl --shared-zlib-includes=/usr/local/include --shared-zlib-libpath=/usr/local/lib --shared-zlib

After running above configure command, and with node-v5.0.0 as the current directory, edit:

A. config.gypi

replace:

                   'libraries': [ '-I/usr/local/include',
                                  '-lz',
                                  '-I/usr/local/ssl-1.0.2d/include',
                                  '-lssl',
                                  '-lcrypto']},

with:

                   'libraries': [ '-L/usr/local/lib',
                                  '-lz',
                                  '-L/usr/local/ssl-1.0.2d/lib',
                                  '-lssl',
                                  '-lcrypto']},

Note on this machine I've made /usr/local/ssl/lib a soft link to /usr/local/ssl-1.0.2d/lib.

B. out/Makefile

replace:

cmd_link = $(LINK.$(TOOLSET)) $(GYP_LDFLAGS) $(LDFLAGS.$(TOOLSET)) -o $@ -Wl,--start-group $(LD_INPUTS) -Wl,--end-group $(LIBS)

with:

cmd_link = $(LINK.$(TOOLSET)) $(GYP_LDFLAGS) $(LDFLAGS.$(TOOLSET)) -o $@ -Wl,-rpath=/usr/local/lib -Wl,-rpath=/usr/local/ssl/lib -L/usr/local/lib -L/usr/local/ssl/lib -Wl,--start-group $(LD_INPUTS) $(LIBS) -Wl,--end-group

Subsequently continue to make node from source as usual.

mscdex added build Issues and PRs related to build files or the CI. openssl Issues and PRs related to the OpenSSL dependency. labels Jul 9, 2015

ghost changed the title ~~io.js v2.3.3 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs)~~ io.js v2.3.4 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) Jul 10, 2015

jbergstroem removed the openssl Issues and PRs related to the OpenSSL dependency. label Jul 22, 2015

ghost changed the title ~~io.js v2.3.4 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs)~~ io.js v2.5.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) Jul 29, 2015

ghost changed the title ~~io.js v2.5.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs)~~ io.js all v2.3.4 - v3.0.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) Aug 5, 2015

ghost closed this as completed Aug 12, 2015

snyk-bot mentioned this issue Apr 1, 2020

[Snyk] Upgrade prismjs from 1.17.1 to 1.19.0 O330oei/node#14

Open

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

io.js all v2.3.4 - v3.0.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) #2147

io.js all v2.3.4 - v3.0.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) #2147

ghost commented Jul 9, 2015

ghost commented Jul 9, 2015

rvagg commented Jul 10, 2015

jbergstroem commented Jul 10, 2015

ghost commented Jul 18, 2015

jbergstroem commented Jul 20, 2015

jbergstroem commented Jul 20, 2015

ghost commented Jul 21, 2015

jbergstroem commented Jul 21, 2015

ghost commented Jul 21, 2015

jbergstroem commented Jul 21, 2015

ghost commented Jul 21, 2015

ghost commented Jul 22, 2015

jbergstroem commented Jul 22, 2015

ghost commented Jul 22, 2015

jbergstroem commented Jul 23, 2015

ghost commented Jul 23, 2015

ghost commented Jul 29, 2015

rvagg commented Jul 29, 2015

jbergstroem commented Jul 30, 2015

ghost commented Jul 30, 2015

jbergstroem commented Jul 30, 2015

ghost commented Aug 12, 2015

fazalmajid commented Sep 30, 2015

cephasjohnson commented Nov 11, 2015

cephasjohnson commented Nov 19, 2015

io.js all v2.3.4 - v3.0.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) #2147

io.js all v2.3.4 - v3.0.0 (1) configures incorrect external-openssl flags, (2) FAILs build, (3) lacks rpath mechanism (&/or docs) #2147

Comments

ghost commented Jul 9, 2015

ghost commented Jul 9, 2015

rvagg commented Jul 10, 2015

jbergstroem commented Jul 10, 2015

ghost commented Jul 18, 2015

jbergstroem commented Jul 20, 2015

jbergstroem commented Jul 20, 2015

ghost commented Jul 21, 2015

jbergstroem commented Jul 21, 2015

ghost commented Jul 21, 2015

jbergstroem commented Jul 21, 2015

ghost commented Jul 21, 2015

ghost commented Jul 22, 2015

jbergstroem commented Jul 22, 2015

ghost commented Jul 22, 2015

jbergstroem commented Jul 23, 2015

ghost commented Jul 23, 2015

ghost commented Jul 29, 2015

rvagg commented Jul 29, 2015

jbergstroem commented Jul 30, 2015

ghost commented Jul 30, 2015

jbergstroem commented Jul 30, 2015

ghost commented Aug 12, 2015

fazalmajid commented Sep 30, 2015

cephasjohnson commented Nov 11, 2015

cephasjohnson commented Nov 19, 2015