Skip to content

Certificates

Debugging Certificate Errors

https://www.netmeister.org/blog/debugging-certificate-errors.html

Usefull OpenSSL Comments

Get the list of certificates from a secrets

1
2
3
4
5
SECRET=letsencrypt-router-certs

openssl crl2pkcs7 -nocrl -certfile \
  <(oc get secrets $SECRET -o go-template='{{ index .data "tls.crt" | base64decode }}' ) \
  | openssl pkcs7 -print_certs  -noout

Check certificate

1
echo -n | openssl s_client -connect q.bohne.io:8443 -servername q.bohne.io 2>/dev/null | openssl x509 -noout -subject -issuer

General: Create a self signed certificate

Create openssl.self-signed-certificate.conf

1
curl -L -O https://examples.openshift.pub/certificate/openssl.self-signed-certificate.conf

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no

[req_distinguished_name]
C = US
ST = VA
L = SomeCity
O = MyCompany
OU = MyDivision
CN = nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com

[v3_req]
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
DNS.1 = nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com
DNS.2 = company.com
DNS.3 = company.net

Create self signed certificate

1
2
3
4
5
openssl req -x509 -nodes -days 730 \
  -newkey rsa:2048 -keyout cert.pem \
  -out cert.pem \
  -config openssl.self-signed-certificate.conf \
  -extensions 'v3_req'

3) Print self signed certificate

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
$ openssl x509 -in cert.pem -noout -text
    Certificate:
        Data:
            Version: 3 (0x2)
            Serial Number: 15062780286100647982 (0xd109bedd3ac9302e)
        Signature Algorithm: sha256WithRSAEncryption
            Issuer: C=US, ST=VA, L=SomeCity, O=MyCompany, OU=MyDivision, CN=nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com
            Validity
                Not Before: Apr  7 13:26:38 2019 GMT
                Not After : Apr  6 13:26:38 2021 GMT
            Subject: C=US, ST=VA, L=SomeCity, O=MyCompany, OU=MyDivision, CN=nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com
    [...snipped...]
            X509v3 extensions:
                X509v3 Key Usage:
                    Digital Signature, Non Repudiation, Key Encipherment
                X509v3 Extended Key Usage:
                    TLS Web Server Authentication
                X509v3 Subject Alternative Name:
                    DNS:nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com, DNS:company.com, DNS:company.net
    [...snipped...]

General: Own root ca and certificate

Create OpenSSL Configuration

Create openssl.root-ca.conf

1
curl -L -O https://examples.openshift.pub/certificate/openssl.root-ca.conf

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
# OpenSSL root CA configuration file.

[ req ]
# Options for the `req` tool (`man req`).
default_bits        = 2048
distinguished_name  = req_distinguished_name
string_mask         = utf8only

# SHA-1 is deprecated, so use SHA-2 instead.
default_md          = sha256

# Extension to add when the -x509 option is used.
x509_extensions     = v3_ca
req_extensions      = v3_req

[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = rootca.example.com
# DNS.2 = *.pass.example.com
# DNS.3 = ...
# DNS.4 = ...


[ req_distinguished_name ]
# See <https://en.wikipedia.org/wiki/Certificate_signing_request>.
countryName                     = Country Name (2 letter code)
stateOrProvinceName             = State or Province Name
localityName                    = Locality Name
0.organizationName              = Organization Name
organizationalUnitName          = Organizational Unit Name
commonName                      = Common Name
emailAddress                    = Email Address

# Optionally, specify some defaults.
countryName_default             = DE
stateOrProvinceName_default     = Bavaria
localityName_default            = Munich
0.organizationName_default      = My Private Root CA
organizationalUnitName_default  = My Private Root CA
emailAddress_default            = email@domain.tld
commonName_default              = rootca.example.com

[ v3_ca ]
# Extensions for a typical CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ v3_intermediate_ca ]
# Extensions for a typical intermediate CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ server_cert ]
# Extensions for server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

[ crl_ext ]
# Extension for CRLs (`man x509v3_config`).
authorityKeyIdentifier=keyid:always

Create openssl.certificate.conf

1
curl -L -O https://examples.openshift.pub/certificate/openssl.certificate.conf

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
# OpenSSL root CA configuration file.

[ req ]
# Options for the `req` tool (`man req`).
default_bits        = 2048
distinguished_name  = req_distinguished_name
string_mask         = utf8only

# SHA-1 is deprecated, so use SHA-2 instead.
default_md          = sha256

# Extension to add when the -x509 option is used.
x509_extensions     = v3_ca
req_extensions      = v3_req

[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = api.example.com
DNS.2 = *.pass.example.com
DNS.3 = nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com
# DNS.4 = ...
# IP.1 = 172.16.0.5

[ req_distinguished_name ]
# See <https://en.wikipedia.org/wiki/Certificate_signing_request>.
countryName                     = Country Name (2 letter code)
stateOrProvinceName             = State or Province Name
localityName                    = Locality Name
0.organizationName              = Organization Name
organizationalUnitName          = Organizational Unit Name
commonName                      = Common Name
emailAddress                    = Email Address

# Optionally, specify some defaults.
countryName_default             = DE
stateOrProvinceName_default     = Bavaria
localityName_default            = Munich
0.organizationName_default      = Private
organizationalUnitName_default  = Private
emailAddress_default            = email@domain.tld
commonName_default              = api.example.com

[ v3_ca ]
# Extensions for a typical CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ v3_intermediate_ca ]
# Extensions for a typical intermediate CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ server_cert ]
# Extensions for server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

[ crl_ext ]
# Extension for CRLs (`man x509v3_config`).
authorityKeyIdentifier=keyid:always

Adjust openssl.certificate.conf

Change commonName, DNS...

Generate the root

1
2
3
4
5
6
openssl genrsa -aes256 -out ca.key -passout pass:openshift 2048

openssl req -config openssl.root-ca.conf \
  -new -x509 -days 7300 -key ca.key -sha256 \
  -extensions v3_ca -out ca.crt \
  -passin pass:openshift

Generate the domain key

1
openssl genrsa -out ssl.key 2048

Generate the certificate signing request

1
2
openssl req -config openssl.certificate.conf \
  -sha256 -new -key ssl.key -out ssl.csr

Sign the request with your root key

1
2
3
4
5
6
openssl x509 -sha256 -req -in ssl.csr \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out yoursite.org.crt -days 7300 \
  -extfile openssl.certificate.conf \
  -extensions v3_req \
  -passin pass:openshift

Checking certificate

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Check your homework:
openssl verify -CAfile ca.crt yoursite.org.crt

# Deployed certificate on openshift online and test it:
$ curl -I --cacert ca.crt  --header 'Host: nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com' https://nginx-ex-ssl-stc-pipeline.6923.rh-us-east-1.openshiftapps.com
HTTP/1.1 200 OK
Server: nginx/1.12.1
Date: Sun, 07 Apr 2019 13:47:08 GMT
Content-Type: text/html
Content-Length: 37451
Last-Modified: Sun, 07 Apr 2019 13:08:57 GMT
ETag: "5ca9f669-924b"
Accept-Ranges: bytes
Set-Cookie: 301fe4e688e3d65605266d24021c9c12=dba6b61bf38d9d364b636a33c0341f4a; path=/; HttpOnly; Secure
Cache-control: private

Trust own root CA on your Linux box

https://access.redhat.com/solutions/1519813

1
2
3
update-ca-trust enable
cp -v ca.crt /etc/pki/ca-trust/source/anchors/
update-ca-trust extract

OpenShift 4 setup own router/ingress certifcate

Official docs:

Replacing the default ingress certificate

Setting a custom default certificate

WARNING: If the default certificate is replaced, it must be signed by a public certificate authority already included in the CA bundle as provided by the container userspace.

Create secret with certificates

1
2
3
4
oc create secret tls router-certs \
  --cert=letsencrypt/fullchain.crt \
  --key=letsencrypt/cert.key \
  -n openshift-ingress

Add secret to ingresscontroller

1
2
3
4
oc patch ingresscontroller default \
  -n openshift-ingress-operator \
  --type=merge \
  --patch='{"spec": { "defaultCertificate": { "name": "router-certs" }}}'

OpenShift 4 - Inject own CA to trusted CA bundle

Quick notes not tested yet:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ oc get cm/trusted-ca-bundle -o yaml -n openshift-console  |grep MyP

$ oc apply -f - <<EOF
apiVersion: v1
data:
  ca-bundle.crt: |
    # MyPrivateCA
    -----BEGIN CERTIFICATE-----
   zzzzz
    -----END CERTIFICATE-----
kind: ConfigMap
metadata:
  name: user-ca-bundle
  namespace: openshift-config
EOF

$ oc patch proxy/cluster \
      --type=merge \
      --patch='{"spec": { "trustedCA": { "name": "user-ca-bundle" }}}'

$ oc get cm/trusted-ca-bundle -o yaml -n openshift-console  |grep MyP
# MyPrivateCA

CONFIGURING A CUSTOM PKI

Some usefull openssl commands

1
oc get svc --all-namespaces  -o=custom-columns="tls:.metadata.annotations.service\.alpha\.openshift\.io/serving-cert-secret-name,namespace:.metadata.namespace"  | grep -v '^<none>' | awk '{ print "oc delete secret/" $1 " -n " $2}'
2023-12-28 2019-11-29 Contributors: Robert Bohne