Miscellaneous security hardening fixes

[pushkarnk]

No description provided.

[Copilot]

Pull request overview

This PR is a broad "security hardening" pass across the OpenSSL FIPS Java provider's native and Java layers. It centralizes the OpenSSL library context behind an atomic accessor (jssl_libctx()), enforces the FIPS provider on EVP fetches, tightens JNI error handling and resource cleanup, adds input validation/zeroization across cipher/MAC/PBKDF2/signature/KeyAgreement code paths, removes noisy ERR_print_errors_fp calls, and introduces a brand-new KeyPairGenerator SPI for DH/EC. It also bumps the artifact version 0.6.0 → 0.7.0 and adds compiler/linker hardening flags to the Makefile.

Changes:

• Replace extern OSSL_LIB_CTX *global_libctx direct access with an atomic-loaded jssl_libctx() and enforce provider=fips on EVP_RAND/EVP_MAC fetches.
• Add OpenSSLKeyPairGenerator (DH/EC) with native DER encoding helpers, plus pervasive null-checks, zeroization, and clearer error propagation across DRBG/Cipher/MAC/Signature/PBKDF2/KeyAgreement classes.
• Behavior changes: GMAC now prepends a random 12-byte nonce to its tag; PBKDF2 now requires explicit key length and ≥1000 iterations; Ed25519/Ed448 sv_update accumulates input across calls; engineSetSeed now feeds the user buffer as DRBG additional input rather than seed material.

Reviewed changes

Copilot reviewed 53 out of 54 changed files in this pull request and generated 25 comments.

Show a summary per file

File	Description
Makefile	Adds hardening CFLAGS/LDFLAGS and keypairgenerator source dir.
pom.xml	Version bump 0.6.0 → 0.7.0.
src/main/native/c/init.c	Atomic-managed global_libctx, secure_getenv, EVP_set_default_properties("fips=yes"), removes stderr logging.
src/main/native/c/drbg.c	FIPS-only RAND fetch; reseed/seed APIs return jssl_status; reseed_with_params no longer pulls fresh getrandom entropy.
src/main/native/c/OpenSSLDrbg.c	Hardened error/return paths and JNI cleanup; engineSetSeed arg order changed.
src/main/native/c/OpenSSLCipher.c, cipher.c	Cleansing key/IV on re-init; bounds checks; padding rejection; goto-cleanup refactor.
src/main/native/c/OpenSSLMAC.c, mac.c	FIPS-only MAC fetch with libctx; goto-cleanup; silent set_params failure.
src/main/native/c/OpenSSLSignature.c, signature.c	Sign/verify input validation, accumulating Ed update buffer, sv_context length type widened.
src/main/native/c/OpenSSLPBKDF2.c	New key_length JNI argument, validated against MAX_KEY_SIZE=256.
src/main/native/c/OpenSSLKeyAgreement.c, keyagreement.c	Use d2i_PrivateKey_ex/d2i_PUBKEY_ex; cleanse shared secret.
src/main/native/c/OpenSSLKeyPairGenerator.{c,h}, jni headers	New native key-pair generation via OSSL encoder API.
src/main/native/c/OpenSSLMD.c, md.c	JNI null checks, fix EVP_DigestFinal_ex length type.
src/main/native/c/RSAKEM*.c, KeyConverter.c, evp_utils.c	Migrate from global_libctx to jssl_libctx().
src/main/native/c/jni_utils.c, kdf.c	Add throwIllegalArgument, propagate kdf populate errors.
src/main/native/include/*	Header updates for new APIs and signatures.
src/main/java/.../cipher/OpenSSLCipher.java	AlgorithmParameters/AAD support, IV auto-generation, key-material zeroization, key-unwrap for public/private.
src/main/java/.../drbg/OpenSSLDrbg.java	ProviderException on init failure; engineSetSeed now passes seed as additional input.
src/main/java/.../mac/OpenSSLMAC.java, GMACWithAes128GCM.java	State checks, ByteBuffer fix; GMAC now prepends random nonce to tag.
src/main/java/.../md/OpenSSLMD.java	ByteBuffer engineUpdate fix.
src/main/java/.../signature/OpenSSLSignature.java	PSS parameter round-trip, init-state checks, input validation.
src/main/java/.../kdf/OpenSSLPBKDF2.java	FIPS min iteration count, key-length plumbing, defensive cloning.
src/main/java/.../keyagreement/{OpenSSL,DH,ECDH}KeyAgreement.java	Null-checks, key-bytes zeroization.
src/main/java/.../keyencapsulation/OpenSSLKEMRSA.java	Wipe encoded key after decapsulator init.
src/main/java/.../keypairgenerator/*	New DH/EC KeyPairGeneratorSpi, Encoded{Public,Private}Key holders.
src/main/java/.../provider/OpenSSLFIPSProvider.java	Register new KeyPairGenerator services.
src/main/java/.../util/NativeMemoryCleaner.java	cleaner field made final.
src/main/java/.../key/KeyConverter.java	Wipe encoded key after native call.
src/test/java/{MacTest,KeyAgreementTest}.java, src/test/native/*	Test adaptations for new GMAC output format, libctx parameter, new include path.
src/test/consumer-snap/snapcraft.yaml	Update jar version to 0.7.0.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 62 out of 62 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 62 out of 62 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/main/java/com/canonical/openssl/cipher/OpenSSLCipher.java:469

• engineUpdateAAD(byte[] aad, int offset, int len) forwards user-controlled offset/len to native without validating bounds (or aad for null). Unlike engineUpdate/engineDoFinal, this can lead to ArrayIndexOutOfBoundsException being thrown by JNI mid-call and the native side operating on an uninitialized buffer. Add the same null + offset < 0 || len < 0 || offset > aad.length - len validation here (and ideally return/throw before calling updateAAD0).

    @Override
    protected void engineUpdateAAD(byte[] aad, int offset, int len) {
        if (!firstUpdate) {
            throw new IllegalStateException("An update() method has already been called");
        }

        if (!isAADSupported()) {
            throw new IllegalStateException("Cipher: " + name + "-" + mode + " does not support Additional Authentication Data");
        }

        updateAAD0(aad, offset, len);