[BUG] OpenSearch 2.15 with ssl

[DmitriiKuvshinov]

Describe the bug
There are no options to configure client key and certificate for SSL/TLS connection for output endpoint (OpenSearch 2.15)

Host/Environment (please complete the following information):

• OS: Ubuntu 22.04
• Logstash 8.8.2 with opensearch output plugin

I try to create keystore file with certificate and key inside, but it's not working

keytool -importcert -file cert.pem -keystore keystore.jks

And my output config

output {
  stdout {
    codec => json_lines
  }
  opensearch {
        hosts                       => ["rnd-os-node03.devops.nova:9200"]
        ssl                         => true
        ssl_certificate_verification => false
        cacert                      => '/etc/opensearch/ca.pem'
        keystore                    => '/etc/opensearch/my_keystore.jks'
        keystore_password           => 'qwertyuiop'
        user                        => 'admin'
        password                    => '0G1hcd7klgJESuJDzmC'
        index                       => "logstash-kafka-%{+YYYY.MM.dd}"
        template_name               => 'logstash'
  }
}

On node i see this message:

[2024-07-05T14:53:02,253][WARN ][o.o.s.a.BackendRegistry  ] [rnd-os-node03] Authentication finally failed for admin from 172.24.49.6:47508

Ang on Logstash node

[2024-07-05T16:01:05,510][WARN ][logstash.outputs.opensearch] Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"https://admin:xxxxxx@rnd-os-node03.devops.nova:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting OpenSearch at URL 'https://rnd-os-node03.devops.nova:9200/'"}

Are there any solutions?

root@rnd-os-logstash:/etc/opensearch# curl -XGET https://rnd-os-mgr.devops.nova:9200/_cat/nodes?v -u 'admin:0G1hcd7klgJESuJDzmC' --cert /etc/opensearch/cert.pem --key /etc/opensearch/key_pkcs.pem
ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles      cluster_manager name
172.24.49.121           50          92  33    1.17    0.83     0.35 m         cluster_manager *               rnd-os-mgr
172.24.49.111           69          95  36    1.22    0.83     0.37 di        data,ingest     -               rnd-os-node01
172.24.49.230           74          91  36    1.48    0.99     0.43 -         -               -               rnd-os-cr
172.24.49.128           54          92  22    1.05    0.96     0.52 di        data,ingest     -               rnd-os-node03
172.24.49.127           56          90  33    1.15    0.90     0.38 di        data,ingest     -               rnd-os-node02

[dblock]

This will need to be debugged, I suggest walking the SSL setup code and seeing how it tries to use the store/key. If you need a way to code from HEAD locally, #186 (comment) should be helpful (and maybe you/someone could contribute #250 along the way :)).