Java RSA compatibility

[guerrillalg]

Hi. I am trying to implement Javascript encryption and Java decryption system.

Javascript part:

key = [new Uint8Array([0x00, .....]), new Uint8Array([0, 1, 0, 1])]
data = asmCrypto.RSA_OAEP_SHA256.encrypt("abc", key, "");

Java part:

Cipher rsa = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
rsa.init(Cipher.DECRYPT_MODE, entry);
String decrypted = new String(rsa.doFinal(data));

Key exchange: Key was generated and stored in the keystore. Java loads it directly from it.
Next key was extracted to .pem file:

keytool -list -rfc -keystore keystore.jks -alias testkey -storepass 123456
Parsed:
openssl x509 -inform pem -text -in key.pem
And put into a tuple as discussed here #98 (comment)

So, when trying to do so, I get:

Caused by: javax.crypto.BadPaddingException: Decryption error

So the questions are:

1. Is there any code example, how one can make Javascript-java compatible encryption/decryption?
2. In the RSA_OAEP_SHA256 which MGF parameter is used?
3. Any suggestions on how to troubleshoot this issue?

[vibornoff]

1. Is there any code example, how one can make Javascript-java compatible encryption/decryption?
2. Any suggestions on how to troubleshoot this issue?

Your code snippet seems legit. Try to remove leading 0x00 from the RSA modulus part.

1. In the RSA_OAEP_SHA256 which MGF parameter is used?

SHA-256 for both hashing and MGF.

[guerrillalg]

Your code snippet seems legit. Try to remove leading 0x00 from the RSA modulus part.

I've tried with and without it - the same.

I've tried the following:
Java:

Cipher.getInstance("RSA/ECB/OAEPWithSHA-1AndMGF1Padding");
JS:
asmCrypto.RSA_OAEP_SHA1.encrypt("abc", key, "");

And it worked.
So, I guess, the problem is:

SHA-256 for both hashing and MGF.

By default Java supports the following RSA algorithms:

RSA/ECB/PKCS1Padding (1024, 2048)
RSA/ECB/OAEPWithSHA-1AndMGF1Padding (1024, 2048)
RSA/ECB/OAEPWithSHA-256AndMGF1Padding (1024, 2048)

So, now the question is, is there possibility to configure MGF in JS library? And could you point, where is it performed?

[guerrillalg]

SO, proceeding with some tests, I've got it working:
JS code:

enc = asmCrypto.RSA_OAEP_SHA256.encrypt("abc", key, "");

And Java:

Cipher rsa = Cipher.getInstance("RSA/ECB/OAEPPadding");
OAEPParameterSpec oaepParams = new OAEPParameterSpec("SHA-256", "MGF1", new MGF1ParameterSpec("SHA-256"), PSource.PSpecified.DEFAULT);
rsa.init(Cipher.DECRYPT_MODE, key, oaepParams);
String decrypted = new String(rsa.doFinal(data));

The problem is that by default "RSA/ECB/OAEPWithSHA-256AndMGF1Padding" algorithm is using SHA-1 For MGF1, regardless that SHA-256 is provided for hashing.

So, it would be nice to have ability to config Hash for MGF separately.

[bhagyachaudhari]

I want to encrypt using RSA_OAEP_SHA256 on JavaScript side

<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/asmCrypto/0.22.0/asmcrypto.all.js"></script>
var encrypted = asmCrypto.RSA_OAEP_SHA256.encrypt(stringToBeEncrypted, pubkey, "");
getting error: Uncaught TypeError: Cannot read property 'encrypt' of undefined

Do you know the solution or any example that would help?
Thanks