Segmentation fault upon syntax error

[montchr]

I'm using grunt-sass to compile several files across my project. If there are no syntax errors, everything runs smoothly. If I run into a syntax error or any similar error, I encounter a segmentation fault. I've used node-segfault-handler to get the following log:

PID 25667 received SIGSEGV for address: 0x0
0   segfault-handler.node               0x0000000101ce91c8 _ZL16segfault_handleriP9__siginfoPv + 280
1   libsystem_platform.dylib            0x00007fff882e952a _sigtramp + 26
2   binding.node                        0x000000010893f9d8 _ZN4Sass15names_to_colorsE + 30520
3   binding.node                        0x0000000108825b45 _ZN4Sass13color_to_nameEd + 31
4   binding.node                        0x0000000108893522 _ZN4Sass7InspectclEPNS_5ColorE + 968
5   binding.node                        0x00000001088908fb _ZN4Sass7InspectclEPNS_11DeclarationE + 207
6   binding.node                        0x00000001088a035f _ZN4Sass6OutputclEPNS_7RulesetE + 1185
7   binding.node                        0x0000000108890383 _ZN4Sass7InspectclEPNS_5BlockE + 149
8   binding.node                        0x00000001088357d6 _ZN4Sass7Context6renderEPNS_5BlockE + 50
9   binding.node                        0x00000001088d2a7d sass_compiler_execute + 107
10  binding.node                        0x00000001088d2697 _Z20sass_compile_contextP12Sass_ContextPN4Sass7ContextE + 34
11  node                                0x000000010096e28a worker + 90
12  node                                0x000000010097a494 uv__thread_start + 25
13  libsystem_pthread.dylib             0x00007fff967e999d _pthread_body + 131
14  libsystem_pthread.dylib             0x00007fff967e991a _pthread_body + 0
15  libsystem_pthread.dylib             0x00007fff967e7351 thread_start + 13

The issue also seems connected to how many files or how many @import statements are being processed. If I move all the files out of the watched directories and leave just one file, it fails on syntax error just fine without a segfault. If I add two, three, four files with just a few imports, it's still fine. But then I add another file with a lot of imports, and the segfault appears again. If I leave that last file where the segfault was just introduced, and I remove all the others, there's no issue, so it doesn't appear to be connected to particular files.

I've searched for solutions to this issue for a while but I've picked up on few patterns between the failures and none of the other Sass segfault solutions out there have worked. I've tried:

• rm -rf node_modules && npm cache clear && npm install
• Downgrading grunt-sass to previous versions (which track previous versions of node-sass)
• Downgrading node to previous versions
• Disabling sourcemaps

[saper]

@montchr can you try to prepare some minimal case to reproduce the problem?

[montchr]

Okay I've been able to track it down to @importing partials with color gradients.

https://gist.github.com/montchr/2d725069a91abc25f8db3462342f98f6

The main _colors.scss file in the Gist contains a set of three rules repeated several times throughout the file, originally sourced from the Spectrum project's spectrum.css.

There are some other files:

1. valid.scss: Imports _colors.scss and that's it
2. invalid.scss: introduces invalid syntax to trigger an error, then imports _colors.scss
3. Gruntfile.js

If the set of three rules in _colors.scss only appear once, I don't run into an issue. If they appear three times, then Sass causes a segfault occasionally. After six repetitions of the three rules, segfaults happen 100% of the time.

Two important things to note:

1. I still get segfaults if I paste the contents of _colors.scss into the two other scss files, but here it's in a partial for clarity
2. If there's an error in both valid.scss and invalid.scss, I do not get a segfault at all

I'm running node-sass 3.8.0 and libsass v3.3.6 on macOS 10.11.6

[montchr]

As a side note, I've searched my project and its dependencies for references to gradient and DXImageTransform and I've disabled all those sections of code. However, I'm still experiencing segfaults. Even with the segfault logger enabled, I don't see an error message or any indication of the segfault besides the grunt-sass process hanging and requiring me to close the terminal window.

[saper]

The good news is that with

sassc: 3.3.6-11-g077a
libsass: 3.3.6-92-g014a61
sass2scss: 1.0.6
sass: 3.4

I cannot reproduce this problem anymore. Running sassc invalid.scss produces:

Error: Invalid CSS after "f": expected 1 selector or at-rule, was "fnord"
        on line 1 of invalid.scss
>> fnord
   ^

[saper]

Running node-sass (on FreeBSD):

node-sass   3.8.0   (Wrapper)   [JavaScript]
libsass     3.3.6   (Sass Compiler) [C/C++]

The output is:

{
  "formatted": "Error: Invalid CSS after \"f\": expected 1 selector or at-rule, was \"fnord\"\n        on line 1 of invalid.scss\n>> fnord\n   ^\n",
  "message": "Invalid CSS after \"f\": expected 1 selector or at-rule, was \"fnord\"",
  "column": 1,
  "line": 1,
  "file": "/home/saper/sw/sass-bug-test/2d725069a91abc25f8db3462342f98f6/invalid.scss",
  "status": 1
}

Can you check what are you getting just by running node-sass invalid.scss ?

[montchr]

With node-sass invalid.scss I get the same as you:

{
  "formatted": "Error: Invalid CSS after \"f\": expected 1 selector or at-rule, was \"fnord\"\n        on line 1 of invalid.scss\n>> fnord\n   ^\n",
  "message": "Invalid CSS after \"f\": expected 1 selector or at-rule, was \"fnord\"",
  "column": 1,
  "line": 1,
  "file": "/Users/montchr/Sites/node-sass-issue-1672/invalid.scss",
  "status": 1
}

[saper]

Can you check if grunt is not installing an older version of node-sass by chance?

[saper]

I have used the following package.json:

{
  "name": "2d725069a91abc25f8db3462342f98f6",
  "version": "1.0.0",
  "description": "",
  "main": "Gruntfile.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://gist.github.com/2d725069a91abc25f8db3462342f98f6.git"
  },
  "keywords": [],
  "author": "Marcin Cieślak <saper@saper.info>",
  "license": "ISC",
  "bugs": {
    "url": "https://gist.github.com/2d725069a91abc25f8db3462342f98f6"
  },
  "homepage": "https://gist.github.com/2d725069a91abc25f8db3462342f98f6",
  "dependencies": {
    "grunt": "^1.0.1",
    "grunt-sass": "^1.2.1"
  }
}

and after npm install (with node v6.2.2 and npm 3.9.2) I get the following:

> ./node_modules/.bin/grunt sass
Running "sass:dev" (sass) task
>> Error: Invalid CSS after "f": expected 1 selector or at-rule, was "fnord"
>>         on line 1 of invalid.scss
>> >> fnord
>>    ^
Warning:  Use --force to continue.

Aborted due to warnings.

[montchr]

grunt-sass is installing 3.8.0 as well.

My package.json:

{
  "name": "node-sass-issue-1672",
  "version": "1.0.0",
  "main": "Gruntfile.js",
  "author": "Chris Montgomery",
  "private": true,
  "devDependencies": {
    "grunt": "^1.0.0",
    "grunt-sass": "^1.2.1"
  }
}

I should also mention that a collaborator on my project is working with the same set of files and they're getting segmentation faults as well. This issue only appeared for us in the past couple weeks, but even with the minimal project in my gist and clearing out node_modules there I still get segmentation faults.

Edit: I'm running node v6.4.0 by the way. I switched to your version v6.2.2 with nvm and I'm still having the issue.

[saper]

er, can you take nvm out of the way?

[saper]

Since it looks like an environmental issue to me, can you please follow https://github.com/sass/node-sass/blob/master/TROUBLESHOOTING.md and post the output?

[montchr]

I downgraded to 5.5.12 without nvm and I still get the issue.

Here's the output of some of those commands. If you want me to run anything else let me know.

$ npm -v
3.10.3

$ node -v
v6.4.0

$ node -p process.versions
{ http_parser: '2.7.0',
  node: '6.4.0',
  v8: '5.0.71.60',
  uv: '1.9.1',
  zlib: '1.2.8',
  ares: '1.10.1-DEV',
  icu: '57.1',
  modules: '48',
  openssl: '1.0.2h' }

$ node -p process.platform
darwin

$ node -p process.arch
x64

$ ./node_modules/.bin/node-sass --version
node-sass       3.8.0   (Wrapper)       [JavaScript]
libsass         3.3.6   (Sass Compiler) [C/C++]

$ node -p "console.log(require('node-sass').info)"
node-sass       3.8.0   (Wrapper)       [JavaScript]
libsass         3.3.6   (Sass Compiler) [C/C++]
undefined

[saper]

What happens if you run ./node_modules/.bin/node-sass invalid.scss ?

Did you create a clean workspace in the temporary directory?

[montchr]

That's what I ran above actually:

{
  "formatted": "Error: Invalid CSS after \"f\": expected 1 selector or at-rule, was \"fnord\"\n        on line 1 of invalid.scss\n>> fnord\n   ^\n",
  "message": "Invalid CSS after \"f\": expected 1 selector or at-rule, was \"fnord\"",
  "column": 1,
  "line": 1,
  "file": "/Users/montchr/Sites/node-sass-issue-1672/invalid.scss",
  "status": 1
}

[saper]

Can you post the full output of ./node_modules/.bin/grunt -v sass ?

[montchr]

$ node_modules/.bin/grunt -v sass
Initializing
Command-line options: --verbose

Reading "Gruntfile.js" Gruntfile...OK

Registering Gruntfile tasks.
Initializing config...OK

Registering "grunt-sass" local Npm module tasks.
Reading /Users/montchr/Sites/node-sass-issue-1672/node_modules/grunt-sass/package.json...OK
Parsing /Users/montchr/Sites/node-sass-issue-1672/node_modules/grunt-sass/package.json...OK

node-sass       3.8.0   (Wrapper)       [JavaScript]
libsass         3.3.6   (Sass Compiler) [C/C++]

Loading "sass.js" tasks...OK
+ sass
Loading "Gruntfile.js" tasks...OK
>> No tasks were registered or unregistered.

Running tasks: sass

Running "sass" task

Running "sass:dev" (sass) task
Verifying property sass.dev exists in config...OK
Files: _colors.scss -> dist/css/_colors.css
Files: invalid.scss -> dist/css/invalid.css
Files: valid.scss -> dist/css/valid.css
Options: precision=10, includePaths=["assets/vendor"]
Options: precision=10, includePaths=["assets/vendor"]
Options: precision=10, includePaths=["assets/vendor"]
>> Error: Invalid CSS after "f": expected 1 selector or at-rule, was "fnord"
>>         on line 1 of invalid.scss
>> >> fnord
>>    ^
Warning:  Use --force to continue.

Aborted due to warnings.
[1]    82592 segmentation fault  node_modules/.bin/grunt -v sass

[montchr]

I also created a new user account on my machine and tested there. I still get a segfault, but with a slightly different message Segmentation fault: 11.

[saper]

Thanks. It seems that the node (JavaScript) thread continues normally and we are getting the error message back. The back-end libsass thread crashes though. Looks like a race condition somewhere. Unfortunately, I cannot reproduce this locally on my systems, so I might need access to a Mac to troubleshoot this.

[saper]

@xzyfer can you reproduce this as well?

[xzyfer]

The "race-condition" is probably just grunt-sass running multiple instances of render. It does an async forEach over the set of matched files. It's probably one instance segfault'ing but the loop continues through the rest of the files.

[xzyfer]

I am able to reproduce this locally. I have been able to confirm the issue is only present when colors"` is present. Inline that code into the scss files does not produce the segfault.

[xzyfer]

Reproducible with Node v5.10.1

➜  tmp ./node_modules/.bin/node-sass -v
node-sass   3.8.0   (Wrapper)   [JavaScript]
libsass     3.3.6   (Sass Compiler) [C/C++]

[xzyfer]

Confirmed this issue is present going to back to node-sass@v3.4.2.

[saper]

Thanks. The stacktrace also points to the color name conversion routines. In my repo there are two files that grunt tries to process, valid.scss and invalid.scss. Do we also get a segmentation fault when there is only one file to be processed?

[montchr]

Do we also get a segmentation fault when there is only one file to be processed?

If I change src to process only invalid.scss I do not get a segmentation fault.

[montchr]

I have been able to confirm the issue is only present when colors"` is present. Inline that code into the scss files does not produce the segfault.

Actually I've found that if I copy the contents of _colors.scss into the two other files, I still get a segmentation fault as mentioned towards the end of #1672 (comment). At first I thought it was caused by @importing only but later realized that was a mistake. Should have edited my comment to reflect that.

[xzyfer]

Actually I've found that if I copy the contents of _colors.scss into the two other files, I still get a segmentation fault as mentioned towards the end

@montchr I saw this comment but was not able to reproduce the segfault in that case

Do we also get a segmentation fault when there is only one file to be processed?

I was not able to reproduce the segfault in this case.

---

I also tried executing multiple instances of the node-sass cli in a loop and was not able to reproduce this. Would be good if you could confirm this @saper.

[saper]

Thanks @xzyfer, this is helpful. This is probably related to multiple parallel calls to render() and to reproduce this one needs to run them within one process. Probably it can be reproduced with a simple async loop that invokes render four times in parallel. This is as I understand the way grunt behaves for us here.

I have two theories why it fails, the pessimistic one is that we got bitten by mixing of C++ on MacOS, we use a modern C++11 library when compiling libsass and the stock GNU one to talk to node, as explained in #1110

I can't reproduce this at all on other platforms, unfortunately.

[MartijnHols]

I ran into an extremely similar issue today using node v12.6.0, node v12.13.0 and node v13.1.0, with node-sass v4.13.0. It occurs even after completely fresh npm installs, on my and my colleague's OS X machines and in our Docker based GitLab runners. Coincidentally my stack trace also includes name_to_color even though the error occurs on an import error:

MartijnHols-MacBook-Pro:layout martijnhols$ tsdx build
✓ Creating entry file 3.6 secs
(postcss plugin) Error: File to import not found or unreadable: ../breakpoints.
Error: File to import not found or unreadable: ../breakpoints.
    at options.error (/Users/martijnhols/Workspace/react-components/node_modules/node-sass/lib/index.js:291:26)

PID 7693 received SIGSEGV for address: 0x0
0   segfault-handler.node               0x0000000102ae6066 _ZL16segfault_handleriP9__siginfoPv + 310
1   libsystem_platform.dylib            0x00007fff785d6b5d _sigtramp + 29
2   binding.node                        0x0000000107858efc _ZNKSt3__14lessINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEclERKS6_S9_ + 98
3   binding.node                        0x0000000107837aed _ZNKSt3__16__treeINS_12__value_typeIPKcPKN4Sass5ColorEEENS_19__map_value_compareIS3_S8_NS4_11map_cmp_strELb1EEENS_9allocatorIS8_EEE4findIS3_EENS_21__tree_const_iteratorIS8_PNS_11__tree_nodeIS8_PvEElEERKT_ + 45
4   binding.node                        0x00000001078374ae _ZN4Sass13name_to_colorERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE + 133
5   binding.node                        0x00000001078ed66c _ZNK4Sass6Parser15color_or_stringERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE + 32
6   binding.node                        0x00000001078ec007 _ZN4Sass6Parser11parse_valueEv + 1301
7   binding.node                        0x00000001078e8fef _ZN4Sass6Parser12parse_factorEv + 2379
8   binding.node                        0x00000001078e7eee _ZN4Sass6Parser15parse_operatorsEv + 136
9   binding.node                        0x00000001078e642c _ZN4Sass6Parser16parse_expressionEv + 140
10  binding.node                        0x00000001078f339c _ZN4Sass6Parser22parse_media_expressionEv + 472
11  binding.node                        0x00000001078f2b87 _ZN4Sass6Parser17parse_media_queryEv + 595
12  binding.node                        0x00000001078d9da1 _ZN4Sass6Parser19parse_media_queriesEv + 213
13  binding.node                        0x0000000107870e1b _ZN4Sass6ExpandclEPNS_11Media_BlockE + 583
14  binding.node                        0x000000010786fd4f _ZN4Sass6Expand12append_blockEPNS_5BlockE + 183
15  binding.node                        0x000000010787320b _ZN4Sass6ExpandclEPNS_2IfE + 353
16  binding.node                        0x000000010786fd4f _ZN4Sass6Expand12append_blockEPNS_5BlockE + 183
17  binding.node                        0x00000001078731c5 _ZN4Sass6ExpandclEPNS_2IfE + 283
18  binding.node                        0x0000000107877735 _ZN4Sass6ExpandclEPNS_10Mixin_CallE + 3147
19  binding.node                        0x000000010786fd4f _ZN4Sass6Expand12append_blockEPNS_5BlockE + 183
20  binding.node                        0x0000000107874f35 _ZN4Sass6ExpandclEPNS_4EachE + 3343
21  binding.node                        0x0000000107877735 _ZN4Sass6ExpandclEPNS_10Mixin_CallE + 3147
22  binding.node                        0x000000010786fd4f _ZN4Sass6Expand12append_blockEPNS_5BlockE + 183
23  binding.node                        0x000000010786fc02 _ZN4Sass6ExpandclEPNS_5BlockE + 384
24  binding.node                        0x00000001078491a8 _ZN4Sass7Context7compileEv + 522
25  binding.node                        0x0000000107848e4d _ZN4Sass12Data_Context5parseEv + 833
26  binding.node                        0x00000001079088f3 sass_compiler_parse + 192
27  binding.node                        0x00000001079086b1 _ZL20sass_compile_contextP12Sass_ContextPN4Sass7ContextE + 26
28  node                                0x00000001008650da worker + 196
29  libsystem_pthread.dylib             0x00007fff785df2eb _pthread_body + 126
30  libsystem_pthread.dylib             0x00007fff785e2249 _pthread_start + 66
31  libsystem_pthread.dylib             0x00007fff785de40d thread_start + 13
Segmentation fault: 11

We're using tsdx and rollup-plugin-postcss to build the code. The rollup-plugin-postcss uses 4 threads by default, but setting that to 1 does not fix the issue. I can reproduce this in my pipelines running on a node:12.6 Docker image, so this doesn't appear to be an OS X exclusive issue.

[saper]

Can you produce a minimal reproduction case?

[MartijnHols]

Looks like I was running into 2 issues, and I might have concluded too soon that they're all caused by the segmentation fault. Upon making a reproduction repository I learned that the segmentation fault seems to indeed only be reproducible on macOS.

The other issue completely freezes the build process upon process.exit(x). This appears to only be reproducible when doing multiple builds in parallel and this is the only thing that I could reproduce with GitHub Actions. You can see this here: https://github.com/EmicoEcommerce/node-sass-segfault-repro/runs/311335174 (I set a timeout of 15 minutes, so look for cancelled jobs). I have no idea what causes this, but I think its cause might be similar to the segmentation fault.

I made a reproduction repository with minimal SASS code and minimal build configuration to reproduce these two issues. I also configured GitHub Actions to make debugging easier (see the actions tab on the repository). There is also a readme with steps to reproduce, possible outcomes and my observations. You can find it all here: https://github.com/EmicoEcommerce/node-sass-segfault-repro

[saper]

Great, thanks! Regarding hanging processes - I think this might be the architectural problem we are facing documented in #857. Can you try a workaround as documented there with environment variable and see if that helps?

[saper]

Split out the parallel imports problem to #2789

[saper]

Your segmentation fault could be a stack overflow. I can generate a crash setting ulimit -s 80 on my FreeBSD, don't know if this is the same.

libuv/libuv#669 MacOS X was affected by a small stack size, and it is known that alpine has very small stacks, too. See also libuv/libuv#2310 libuv/libuv#1507

[saper]

I have another theory:

From the backtraces we can see that Exception::NestingLimitError is about to be thrown.

It looks to me that we explicitly turn off exception handling for MacOS X in binding.gyp.

Could anyone with Mac OS X rebuild the binding with 'GCC_ENABLE_CPP_EXCEPTIONS': 'YES' instead and see if this continues to crash? cc @xzyfer

Do we have a test case for the nesting error?

[saper]

Seems like that using catch throw in the gdb and stopping on C++ allows me to finish the program even with parallel builds. On FreeBSD I am getting Sass::Exception::InvalidSyntax exception and not the Sass::Exception::NestingLimitError as suggested by the backtraces on MacOS-X.

Maybe C++ exceptions and multiple libsass threads (#2789) do not go well together, but why nesting limit? The limit should be the same on all platforms.

[saper]

On non-macOS systems we also inherit -fno-exceptions from node:

clang++ '-DNODE_GYP_MODULE_NAME=binding' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-DBUILDING_NODE_EXTENSION' -I/usr/local/include/node -I/usr/local/src -I/usr/local/deps/openssl/config -I/usr/local/deps/openssl/openssl/include -I/usr/local/deps/uv/include -I/usr/local/deps/zlib -I/usr/local/deps/v8/include -I../node_modules/nan -I../src/libsass/include -fPIC -pthread -Wall -Wextra -Wno-unused-parameter -m64 -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++1y -std=c++0x -MMD -MF ./Release/.deps/Release/obj.target/binding/src/sass_types/number.o.d.raw -I/usr/local/include -g -c -o Release/obj.target/binding/src/sass_types/number.o ../src/sass_types/number.cpp

[MartijnHols]

Could anyone with Mac OS X rebuild the binding with 'GCC_ENABLE_CPP_EXCEPTIONS': 'YES' instead and see if this continues to crash? cc @xzyfer

I think I did this right: clone, install, change that value in binding.gyp, build, npm link, verify it uses the correct binary by reading the output of npm rebuild node-sass in the target repo.

I don't see a difference when using that binary.