Missing authentication header when requesting secured elassandra #273

vroyer · 2017-12-13T16:35:01Z

Issue Description
When authentication is enabled on elassandra (5.5.0.8 Help Regarding Output Syntax #146-branch), sentinl does not send any authentication header in one search request, and rejection cause an error. It should at least retry with the appropriate authentication header rather than throws an error. All other requests to elasticsearch are authenticated with the kibana account or the one used to log in kibana, here a watcher account.
Reproducing the Issue step-by-step
I can provide the elassandra enterprise code, license and conf to reproduce it if necessary.
Expected Results vs. Actual Results

Wireshark clear text trace of the sentinl request and response:

Frame 399: 231 bytes on wire (1848 bits), 231 bytes captured (1848 bits) on interface 0
Null/Loopback
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 50158 (50158), Dst Port: 9200 (9200), Seq: 389, Ack: 1688, Len: 175
Secure Sockets Layer
    TLSv1.2 Record Layer: Application Data Protocol: http
Hypertext Transfer Protocol
    POST /watcher/sentinl-watcher/_search?size=50&allow_no_indices=false HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): POST /watcher/sentinl-watcher/_search?size=50&allow_no_indices=false HTTP/1.1\r\n]
        Request Method: POST
        Request URI: /watcher/sentinl-watcher/_search?size=50&allow_no_indices=false
        Request Version: HTTP/1.1
    Host: localhost:9200\r\n
    Content-Length: 0\r\n
    Connection: keep-alive\r\n
    \r\n
    [Full request URI: https://localhost:9200/watcher/sentinl-watcher/_search?size=50&allow_no_indices=false]
    [HTTP request 1/1]
    [Response in frame: 405]


Frame 405: 459 bytes on wire (3672 bits), 459 bytes captured (3672 bits) on interface 0
Null/Loopback
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 9200 (9200), Dst Port: 50158 (50158), Seq: 1688, Ack: 564, Len: 403
Secure Sockets Layer
    TLSv1.2 Record Layer: Application Data Protocol: http
    SSL segment data (166 bytes)
    TLSv1.2 Record Layer: Application Data Protocol: http
    SSL segment data (179 bytes)
[2 Reassembled SSL segments (345 bytes): #405(166), #405(179)]
Hypertext Transfer Protocol
    HTTP/1.1 401 Unauthorized\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 401 Unauthorized\r\n]
        Request Version: HTTP/1.1
        Status Code: 401
        Response Phrase: Unauthorized
    WWW-Authenticate: Basic realm="TestCluster authentication required"\r\n
    content-type: application/json; charset=UTF-8\r\n
    content-length: 179\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 0.001195000 seconds]
    [Request in frame: 399]
JavaScript Object Notation: application/json
    Object
        Member Key: "error"
            Object
                Member Key: "root_cause"
                    Array
                Member Key: "type"
                    String value: authentication_exception
                Member Key: "reason"
                    String value: Authentication required
        Member Key: "status"
            Number value: 401

Elassandra audit trail

[2017-12-13 17:10:18,827 200,PRIVILEGE,kibana,roles/kibana,/127.0.0.1,cluster:monitor/state,[githubstats, twitter, elassandra, twitter2, watcher_alarms, .kibana, gpm, manu, test, fakeit, watcher, tru
c, twitter3, composite, strapcloud, watcher_alarms-2017.12.13, watcher_alarms-2017.12.11, predictor]
2017-12-13 17:10:21,431 200,PRIVILEGE,kibana,roles/kibana,/127.0.0.1,cluster:monitor/main,null
2017-12-13 17:10:21,439 200,PERMISSION,watcher,roles/watcher,/127.0.0.1,indices:data/read/get,[.kibana]
2017-12-13 17:10:21,525 200,PRIVILEGE,kibana,roles/kibana,/127.0.0.1,cluster:monitor/nodes/info,null
2017-12-13 17:10:21,617 200,PRIVILEGE,kibana,roles/kibana,/127.0.0.1,cluster:monitor/nodes/info,null
2017-12-13 17:10:21,708 200,PERMISSION,kibana,roles/kibana,/127.0.0.1,indices:data/read/mget[shard],[.kibana]
2017-12-13 17:10:21,799 200,PRIVILEGE,kibana,roles/kibana,/127.0.0.1,cluster:monitor/health,[.kibana]
2017-12-13 17:10:21,892 200,PERMISSION,kibana,roles/kibana,/127.0.0.1,indices:data/read/search,[.kibana]
2017-12-13 17:10:21,916 403,UNAUTHENTICATED,null,/watcher/sentinl-watcher/_search,/127.0.0.1,null,null
2017-12-13 17:10:22,013 200,PERMISSION,watcher,roles/watcher,/127.0.0.1,indices:data/read/search,[watcher]
2017-12-13 17:10:22,015 200,PERMISSION,watcher,roles/watcher,/127.0.0.1,indices:data/read/search,[watcher]
2017-12-13 17:10:22,090 200,PERMISSION,watcher,roles/watcher,/127.0.0.1,indices:data/read/search,[watcher]](url)

Elassandra error trace

2017-12-13 17:10:21,916 DEBUG [elasticsearch[127.0.0.1][http_server_worker][T#9]] BytesRestResponse.java:131 build path: /watcher/sentinl-watcher/_search, params: {size=50, allow_no_indices=false, i
ndex=watcher, type=sentinl-watcher}
org.apache.cassandra.exceptions.AuthenticationException: Authentication required
        at com.strapdata.elasticsearch.plugin.aaa.CassandraEnforcer.authenticate(CassandraEnforcer.java:246)
        at com.strapdata.elasticsearch.plugin.aaa.rest.AAARestHandlerWrapper$1.handleRequest(AAARestHandlerWrapper.java:64)
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:262)
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:200)
        at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:506)
        at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:72)
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:63)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1273)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1084)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)

Kibana error trace

respons [16:10:21.868]  GET /api/sentinl/time 200 0ms - 9.0B
respons [16:10:21.871]  GET /bundles/0cebf3d61338c454670b1c5bdf5d6d8d.svg 304 0ms - 9.0B
  log   [16:10:21.916] [info][status][Sentinl] Get scripts with type: input
  log   [16:10:21.918] [info][status][Sentinl] Get scripts with type: condition
  log   [16:10:21.920] [info][status][Sentinl] Get scripts with type: transform
Unhandled rejection Error: Cannot provide statusCode or message with boom error
    at Object.exports.assert (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/plugins/sentinl/node_modules/hoek/lib/index.js:736:11)
    at Object.exports.wrap (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/plugins/sentinl/node_modules/boom/lib/index.js:76:10)
    at Object.internals.create (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/plugins/sentinl/node_modules/boom/lib/index.js:94:24)
    at Object.exports.unauthorized (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/plugins/sentinl/node_modules/boom/lib/index.js:164:27)
    at handleESError (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/plugins/sentinl/server/lib/handle_es_error.js:47:21)
    at callWithRequest.then.catch.err (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/plugins/sentinl/server/routes/routes.js:140:29)
    at bound (domain.js:280:14)
    at runBound (domain.js:293:12)
    at tryCatcher (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/bluebird/js/main/promise.js:503:31)
    at Promise._settlePromiseAt (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/bluebird/js/main/promise.js:577:18)
    at Promise._settlePromises (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/bluebird/js/main/promise.js:693:14)
    at Async._drainQueue (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/bluebird/js/main/async.js:123:16)
    at Async._drainQueues (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/bluebird/js/main/async.js:133:10)
    at Immediate.Async.drainQueues (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/bluebird/js/main/async.js:15:14)
    at runCallback (timers.js:666:20)
    at tryOnImmediate (timers.js:639:5)
    at processImmediate [as _immediateCallback] (timers.js:611:5)
respons [16:10:21.935]  GET /bundles/0f3eed75a05071558612ce4dedf76b11.svg 304 1ms - 9.0B
respons [16:10:22.020]  GET /api/sentinl/list/scripts/transform 200 99ms - 9.0B

The text was updated successfully, but these errors were encountered:

vroyer · 2017-12-13T16:36:54Z

Used sentinl version = v5.5.1 (https://github.com/sirensolutions/sentinl/releases/download/tag-5.5/sentinl-v5.5.1.zip)

vroyer · 2017-12-13T16:56:58Z

Same error with sentinl version 5.5.0 for kibana 5.5.0 as shown in the following trace

 log   [16:54:34.108] [debug][Sentinl] [authentication_exception] Authentication required :: {"path":"/watcher/sentinl-script/_search","query":{"size":50,"q":"description:transform"},"statusCode":401,"response":"{\"error\":{\"root_cause\":[{\"type\":\"authentication_exception\",\"reason\":\"Authentication required\"}],\"type\":\"authentication_exception\",\"reason\":\"Authentication required\"},\"status\":401}","wwwAuthenticateDirective":"Basic realm=\"TestCluster authentication required\""}
    at respond (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/elasticsearch/src/lib/transport.js:295:15)
    at checkRespForFailure (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/elasticsearch/src/lib/transport.js:254:7)
    at HttpConnector.<anonymous> (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)
    at IncomingMessage.bound (/Users/vroyer/Dev/kibana-5.5.0-darwin-x86_64/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickDomainCallback (internal/process/next_tick.js:128:9)

lmangani · 2017-12-13T17:21:46Z

Hi @vroyer I need to confirm this with @sergibondarenko but I think this feature set is only available in 5.6.x we'll discuss backporting it as this is not the first request. Will update.

sergibondarenko · 2017-12-14T11:26:05Z

Hi @vroyer
What type of authentication is used?

When authentication is enabled on elassandra ...

vroyer · 2017-12-14T15:37:09Z

Hi Sergii, This is a standard HTTP Basic authentication.

…

Le 14 déc. 2017 à 12:26, Sergii Bondarenko ***@***.***> a écrit : Hi @vroyer <https://github.com/vroyer> What type of authentication is used? When authentication is enabled on elassandra ... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#273 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJzHmWZGYta0mwLx9kf4rPbvVYZwb1rsks5tAQXPgaJpZM4RA1At>.

giulioroggero · 2018-02-12T15:00:36Z

I've a similar error.
Sentinl don't send the authorization header.

I'm using kibana 5.6.5 basic authentication + sentinl 5.6.5 (tag 5.6.2)

I've to change the authentication type of kibana?

Thanks!

vroyer · 2018-05-10T22:30:24Z

I have the same issue with kibana 6.2.3, as describe in kibana issue #9583

It's definitly up to kibana to send the authorization header properly....

sergibondarenko · 2018-05-11T13:21:31Z

Sentinl supports basic authentication since v5.4: https://github.com/sirensolutions/sentinl/blob/branch-5.4/server/lib/get_elasticsearch_client.js#L16

fbaligand · 2018-05-11T13:34:45Z

Concerning this feature, why don’t you use “elasticsearch.username” and “elasticsearch.password” in kibana.yml ?

Globally, it would be great that all options in kibana.yml allowing to customize elasticsearch queries, are useable in sentinl.

lmangani · 2018-05-11T13:41:48Z

I believe if and when those are provided, the ES client in Kibana automatically resolves authentication for SENTINL as well @fbaligand did you have experience with the opposite condition?

fbaligand · 2018-05-11T13:55:28Z

Happy to see it is the expected behavior, and so, that Sentinl use the official elasticsearch Kibana client (with all expected features). I will make a test to check all works as expected and will give you feedback.

vroyer · 2018-05-11T14:02:36Z

Well, kibana use its own account "kibana" to manage its configuration stored in elasticsearch, in index .kibana by default.

For authenticated users, kibana act as a proxy and forward user's credential to elasticsearch, in order to check that the user have access to the requested index.

Elassandra just check that all elasticsearch requests have a user credential (an Authorization header) matching a cassandra user with appropriate authorizations (for exemple having the SELECT permission to get/search in the index). Of course, if no credential is sent, access is refused according to the security policy.

Hope this help.
Vincent.

fbaligand · 2018-05-11T14:14:06Z

@lmangani
When a watcher/report is executed using scheduling, does it use "elasticsearch.username" and "elasticsearch.password" options ? If not, how can I provide username/password in SENTINL for that case ?
Same question when Kibana starts, and so, SENTINL creates its own indexes ?

Fix readme, v5.4.0

lmangani added the AUTH label Dec 14, 2017

sergibondarenko added the enhancement label Dec 14, 2017

lmangani assigned sergibondarenko Mar 2, 2018

lmangani closed this as completed Jul 30, 2020

pull bot pushed a commit to rpatil524/sentinl that referenced this issue Feb 7, 2021

Merge pull request sentinl#273 from sirensolutions/issue-264-master

2b5a176

Fix readme, v5.4.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing authentication header when requesting secured elassandra #273

Missing authentication header when requesting secured elassandra #273

vroyer commented Dec 13, 2017

vroyer commented Dec 13, 2017

vroyer commented Dec 13, 2017

lmangani commented Dec 13, 2017

sergibondarenko commented Dec 14, 2017

vroyer commented Dec 14, 2017 via email

giulioroggero commented Feb 12, 2018

vroyer commented May 10, 2018

sergibondarenko commented May 11, 2018

fbaligand commented May 11, 2018

lmangani commented May 11, 2018

fbaligand commented May 11, 2018

vroyer commented May 11, 2018

fbaligand commented May 11, 2018 •

edited

Loading

Missing authentication header when requesting secured elassandra #273

Missing authentication header when requesting secured elassandra #273

Comments

vroyer commented Dec 13, 2017

vroyer commented Dec 13, 2017

vroyer commented Dec 13, 2017

lmangani commented Dec 13, 2017

sergibondarenko commented Dec 14, 2017

vroyer commented Dec 14, 2017 via email

giulioroggero commented Feb 12, 2018

vroyer commented May 10, 2018

sergibondarenko commented May 11, 2018

fbaligand commented May 11, 2018

lmangani commented May 11, 2018

fbaligand commented May 11, 2018

vroyer commented May 11, 2018

fbaligand commented May 11, 2018 • edited Loading

fbaligand commented May 11, 2018 •

edited

Loading