Dynamic SSL Certificate Loading in OpenResty Based on Upstream Logic

OpenResty®, a powerful web platform built on Nginx and LuaJIT, excels at handling complex, dynamic web serving tasks. One such advanced requirement is the ability to dynamically select and serve SSL/TLS certificates. While OpenResty offers robust mechanisms for dynamic SSL based on Server Name Indication (SNI), a more intricate challenge arises when the choice of certificate needs to be influenced by logic or data derived from an upstream application or service.

This scenario typically means that the information required to select the correct SSL certificate isn’t available directly from the client’s initial request (like SNI alone) but depends on a decision made after consulting a backend system. This article explores how to architect such solutions in OpenResty, clarifying the limitations of Nginx’s processing phases and presenting practical, effective patterns.

We’ll delve into the ssl_certificate_by_lua_block directive, Lua scripting with the ngx.ssl module, and architectural approaches to achieve dynamic SSL certificate loading that aligns with upstream-driven decisions.

The Challenge: Nginx Phases and SSL Handshake Timing

Nginx processes requests through a series of distinct phases. The SSL handshake, where the server presents its certificate, occurs very early in this lifecycle. OpenResty’s primary hook for dynamic SSL certificate configuration is the ssl_certificate_by_lua_block directive. Code within this block executes before the main request processing phases (like rewrite, access, or content) where typical upstream interactions (e.g., via proxy_pass or Lua cosocket calls to application backends) occur.

This timing presents a fundamental challenge: you cannot directly proxy_pass to a main application upstream, get its response, and then use that response to select an SSL certificate for the same, ongoing client SSL handshake. The certificate decision point has already passed.

Therefore, “dynamic SSL based on upstream response” requires careful architectural design to bridge this timing gap.

Approach 1: SNI-Driven Dynamic SSL with an External Metadata Service

This is the foundational pattern for most dynamic SSL setups in OpenResty and a key component of more complex solutions. Here, the “upstream” is a specialized, fast service (like Redis, a custom microservice, or a local database) that provides certificate information based on the SNI hostname sent by the client.

Concept:

Client initiates an SSL connection, providing an SNI hostname.
The ssl_certificate_by_lua_block executes.
Lua code within this block retrieves the SNI using ngx.ssl.server_name().
It then queries an external metadata service (non-blockingly) or a shared memory cache for the SSL certificate and private key corresponding to the SNI.
If found, the certificate and key are loaded using ngx.ssl.set_pem_cert() and ngx.ssl.set_pem_priv_key().

Nginx Configuration (nginx.conf):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# Define a shared memory zone for caching certificate data
# https://github.com/openresty/lua-nginx-module#lua_shared_dict
lua_shared_dict cert_cache 10m; # Adjust size as needed

# Define a shared memory zone for caching OCSP responses
lua_shared_dict ocsp_cache 10m; # Optional, for OCSP stapling

server {
    listen 443 ssl;
    server_name _; # Catch-all or specific managed domains

    # Fallback/default certificate (IMPORTANT for Nginx start-up)
    ssl_certificate /etc/nginx/certs/fallback.crt;
    ssl_certificate_key /etc/nginx/certs/fallback.key;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Dynamic certificate loading via Lua
    ssl_certificate_by_lua_block {
        -- require and execute your Lua script
        -- that handles cert loading
        local cert_loader = require "custom_ssl_loader"
        cert_loader.load_certificate()
    }

    # Optional: Dynamic OCSP Stapling
    # https://github.com/openresty/lua-nginx-module#ssl_stapling_by_lua_block
    ssl_stapling_by_lua_block {
        local cert_loader = require "custom_ssl_loader"
        cert_loader.staple_ocsp()
    }

    location / {
        # Your actual application proxy or content handler
        proxy_pass http://my_application_backend;
        # or root /var/www/html; index index.html;
    }
}

Lua Script (custom_ssl_loader.lua):

This script shows fetching from a cache and conceptually from a metadata service.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
-- custom_ssl_loader.lua
local ngx_ssl = require "ngx.ssl"
local ngx_shared = ngx.shared.cert_cache
local ngx_re_match = ngx.re.match -- For potential regex on SNI

-- For HTTP client (if fetching certs from an API)
-- Example: local http = require "resty.http" (https://github.com/ledgetech/lua-resty-http)
-- local METADATA_SERVICE_URL = "http://127.0.0.1:8081/get-cert"

local M = {}

-- Function to safely get cert/key from an external source
-- In a real scenario, this would involve a non-blocking HTTP call
-- or database lookup if not found in cache.
local function fetch_cert_data_from_source(hostname)
    -- Placeholder: In reality, fetch from a secure API, Vault, etc.
    -- For example, if certs are stored as files named by hostname:
    local cert_path = "/etc/nginx/dynamic_certs/" .. hostname .. ".crt"
    local key_path = "/etc/nginx/dynamic_certs/" .. hostname .. ".key"

    local cert_file, err_cert = io.open(cert_path, "rb")
    if not cert_file then
        ngx.log(ngx.ERR, "Cert file not found: ", cert_path, " ", err_cert)
        return nil, nil
    end
    local cert_data = cert_file:read("*a")
    cert_file:close()

    local key_file, err_key = io.open(key_path, "rb")
    if not key_file then
        ngx.log(ngx.ERR, "Key file not found: ", key_path, " ", err_key)
        return nil, nil
    end
    local key_data = key_file:read("*a")
    key_file:close()

    if cert_data and key_data then
        -- Cache it for next time (e.g., for 1 hour)
        -- Key names should be unique, e.g., prefix with type
        ngx_shared:set("cert:" .. hostname, cert_data, 3600)
        ngx_shared:set("key:" .. hostname, key_data, 3600)
        ngx.log(ngx.INFO, "Fetched and cached cert for: ", hostname)
    end
    return cert_data, key_data
end

function M.load_certificate()
    local server_name = ngx_ssl.server_name()
    if not server_name then
        ngx.log(ngx.ERR, "SNI server_name not provided by client.")
        return ngx.exit(ngx.ERROR) -- Or allow fallback
    end

    ngx.log(ngx.INFO, "Attempting to load cert for SNI: ", server_name)

    -- 1. Try to get from cache
    local cert_pem = ngx_shared:get("cert:" .. server_name)
    local key_pem = ngx_shared:get("key:" .. server_name)

    if not cert_pem or not key_pem then
        ngx.log(ngx.INFO, "Cert for ", server_name, " not in cache. Fetching.")
        -- 2. If not in cache, fetch from source (e.g., files, API)
        -- This call MUST be non-blocking if it's network I/O.
        -- The example fetch_cert_data_from_source is simplified (blocking file I/O).
        -- For a real metadata service, use lua-resty-http with cosockets.
        cert_pem, key_pem = fetch_cert_data_from_source(server_name)
    else
        ngx.log(ngx.INFO, "Loaded cert for ", server_name, " from cache.")
    end

    if not cert_pem or not key_pem then
        ngx.log(ngx.ERR, "Could not find/load certificate for: ", server_name)
        -- Nginx will use the statically configured fallback certificate
        return
    end

    -- Set the certificate
    local ok, err = ngx_ssl.set_pem_cert(cert_pem)
    if not ok then
        ngx.log(ngx.ERR, "Failed to set PEM certificate for ",
                server_name, ": ", err)
        return ngx.exit(ngx.ERROR)
    end

    -- Set the private key
    ok, err = ngx_ssl.set_pem_priv_key(key_pem)
    if not ok then
        ngx.log(ngx.ERR, "Failed to set PEM private key for ",
                server_name, ": ", err)
        return ngx.exit(ngx.ERROR)
    end

    ngx.log(ngx.INFO, "Successfully set dynamic SSL cert for: ", server_name)
end

-- Placeholder for OCSP stapling logic
function M.staple_ocsp()
    -- Similar logic: get cert, fetch OCSP response, cache it,
    -- then use ngx.ssl.set_ocsp_status_resp()
    -- This is a complex topic on its own.
    -- ngx.log(ngx.INFO, "OCSP stapling to be implemented.")
end

return M

This pattern is highly efficient for serving many domains, as certificates are cached and lookups are fast. The “upstream” metadata service is specialized for certificate delivery.

Approach 2: Two-Tier Strategy for True Upstream-Response Influence

To make SSL decisions based on a response from your primary application upstream, a two-tier approach involving a client-side redirect is typically the most robust and manageable solution.

Concept:

Tier 1 (Initial Contact & Decision):
- Client connects to a generic or initial endpoint on OpenResty (e.g., https://initiate.example.com). This endpoint might use a wildcard or default SSL certificate.
- In a content phase handler (e.g., content_by_lua_block), OpenResty makes a subrequest or proxies the request to your main application upstream.
- The application upstream processes the request and its response includes information that dictates which specific hostname (and thus SSL certificate) the client should ultimately use (e.g., a tenant-specific hostname like tenantA.example.com).
- OpenResty’s Lua script receives this information and issues an HTTP redirect (301 or 302) to the client, pointing them to the specific hostname.
Tier 2 (Serving with Correct Certificate):
- The client’s browser follows the redirect and makes a new request to the specific hostname (e.g., https://tenantA.example.com).
- This new request is handled by an OpenResty server block configured for that hostname (or a wildcard that covers it).
- This server block uses the SNI-driven dynamic SSL mechanism (Approach 1) to look up and serve the correct SSL certificate for tenantA.example.com.

Nginx Configuration for Tier 1 (nginx.conf snippet):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
server {
    listen 443 ssl;
    server_name initiate.example.com; # Or a generic entry point

    # Generic/wildcard SSL certificate for this initial endpoint
    ssl_certificate /etc/nginx/certs/generic.example.com.crt;
    ssl_certificate_key /etc/nginx/certs/generic.example.com.key;

    # Standard SSL params (protocols, ciphers)

    location / {
        content_by_lua_block {
            -- Lua script to call upstream and issue redirect
            local upstream_comm = require "upstream_communicator"
            upstream_comm.handle_initial_request_and_redirect()
        }
    }
}

Lua Script for Tier 1 (upstream_communicator.lua):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
-- upstream_communicator.lua
-- Using lua-resty-http: https://github.com/ledgetech/lua-resty-http
local http = require "resty.http"

local M = {}

function M.handle_initial_request_and_redirect()
    -- 1. Create a new HTTP client for the upstream call
    local httpc = http.new()
    local upstream_url = "http://your_application_backend/decide-hostname"

    -- Pass relevant parts of the original request if needed
    -- For example, headers or query parameters
    local res, err = httpc:request_uri(upstream_url, {
        method = "GET", -- Or POST, etc.
        -- body = ngx.var.request_body, -- If forwarding body
        headers = {
            ["X-Original-URI"] = ngx.var.request_uri,
            -- Add other relevant headers
        }
    })

    if not res then
        ngx.log(ngx.ERR, "Failed to connect to upstream: ", err)
        ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
        return
    end

    -- Assuming upstream returns JSON like: {"targetHostname": "tenantA.example.com"}
    if res.status ~= 200 then
        ngx.log(ngx.ERR, "Upstream returned error: ", res.status)
        ngx.say("Error processing request. Upstream status: ", res.status)
        ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
        return
    end

    -- Using lua-cjson: https://github.com/openresty/lua-cjson
    local cjson = require "cjson.safe"
    local body_data, cjson_err = cjson.decode(res.body)
    if not body_data or cjson_err then
        ngx.log(ngx.ERR, "Failed to decode upstream JSON response: ", cjson_err)
        ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
        return
    end

    local target_hostname = body_data.targetHostname
    if not target_hostname then
        ngx.log(ngx.ERR, "targetHostname not found in upstream response.")
        ngx.exit(ngx.HTTP_BAD_REQUEST) -- Or appropriate error
        return
    end

    -- 2. Issue redirect to the target hostname
    -- Preserve original request URI path and query string
    local redirect_url = "https://" .. target_hostname .. ngx.var.request_uri
    ngx.log(ngx.INFO, "Redirecting to: ", redirect_url)
    return ngx.redirect(redirect_url, ngx.HTTP_MOVED_TEMPORARILY) -- Or 301
end

return M

Nginx Configuration for Tier 2: This would be a server block (or multiple) similar to the one shown in Approach 1, configured to handle tenantA.example.com (and other such dynamic hostnames) and use ssl_certificate_by_lua_block with a script like custom_ssl_loader.lua to serve the appropriate certificate based on SNI.

This two-tier approach cleanly separates the concerns: the initial interaction determines the “what” (which hostname/certificate), and the subsequent redirected request handles the “how” (serving that specific certificate via SNI).

Implementing Certificate and Key Handling in Lua

Regardless of the approach, the core Lua logic using ngx.ssl for setting certificates is similar:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
-- Inside ssl_certificate_by_lua_block context

-- Assuming cert_pem_data and key_pem_data are strings
-- containing the PEM-encoded certificate and private key respectively.

-- Example: Loading from variables
local cert_pem_data = "-----BEGIN CERTIFICATE-----\nMIID....\n-----END CERTIFICATE-----"
local key_pem_data = "-----BEGIN PRIVATE KEY-----\nMIIE....\n-----END PRIVATE KEY-----"

-- Set the certificate
-- ngx.ssl.set_pem_cert expects the full PEM content as a Lua string.
local ok, err = ngx.ssl.set_pem_cert(cert_pem_data)
if not ok then
    ngx.log(ngx.ERR, "Failed to set PEM certificate: ", err)
    -- Allow Nginx to use its statically configured fallback certificate
    -- or ngx.exit(ngx.ERROR) if this is a hard failure.
    return
end

-- Set the private key
-- ngx.ssl.set_pem_priv_key expects the full PEM private key
-- content as a Lua string.
ok, err = ngx.ssl.set_pem_priv_key(key_pem_data)
if not ok then
    ngx.log(ngx.ERR, "Failed to set PEM private key: ", err)
    -- Important: If cert was set but key fails, clear the cert
    -- to avoid inconsistent state that might crash worker.
    -- https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#ngxsslclear_certs
    ngx.ssl.clear_certs()
    return -- Or ngx.exit(ngx.ERROR)
end

ngx.log(ngx.INFO, "Successfully applied dynamic certificate and key.")

Ensure that certificate and key data are handled securely, especially when fetched from external sources or stored in variables.

Essential Best Practices

Aggressive Caching: Use lua_shared_dict to cache certificate data (PEM strings) and metadata. This significantly reduces latency for subsequent requests to the same hostname.
Non-Blocking Operations: Critically important. Any I/O within ssl_certificate_by_lua_block (e.g., fetching from a metadata API) must be non-blocking using Lua cosockets (e.g., via lua-resty-http). Blocking calls here will stall SSL handshakes and severely degrade server performance.
Fallback Certificates: Always define static ssl_certificate and ssl_certificate_key directives in your Nginx server block. This ensures Nginx can start and handle requests if Lua logic fails or a dynamic certificate isn’t found.
Security:
- Protect private keys at all costs. Store them securely and transmit them over secure channels if fetched from an API.
- Ensure the metadata service providing certificate information is itself secure and trusted.
- Minimize the exposure of raw key material in logs.
Robust Error Handling: Implement comprehensive error checking in your Lua scripts. Log errors clearly and decide whether to fall back to a default certificate or terminate the connection.
Monitoring and Logging: Use ngx.log extensively to track the certificate loading process, successes, failures, and cache hits/misses. This is invaluable for debugging and operations.
Atomic Caching: When updating cached certificates (e.g., upon renewal), do so atomically to avoid serving a mismatched cert/key pair. Storing them under a versioned key or using temporary keys during update can help.

Common Pitfalls and Considerations

Blocking Calls: The most common and impactful pitfall. Re-iterate: no blocking I/O in ssl_certificate_by_lua_block.
Certificate/Key Formats: ngx.ssl.set_pem_cert/key expect PEM-encoded data. If your source provides DER, you would use ngx.ssl.set_der_cert() / ngx.ssl.set_der_priv_key().
SSL Session Resumption: The ssl_certificate_by_lua_block handler typically runs only for full SSL handshakes, not for resumed sessions (where the original certificate is re-used). This is usually the desired behavior.
Large Number of Certificates: If managing tens of thousands of certificates, ensure your caching strategy and metadata lookup are highly optimized. Memory usage of lua_shared_dict also becomes a consideration.
Testing: Thoroughly test various scenarios: cache miss, cache hit, metadata service down, invalid certificate data, certificate renewal.

Debugging Your Dynamic SSL Setup

Nginx Debug Logs: Enable debug logging for Nginx (error_log /path/to/error.log debug;). This will include detailed messages from the ngx_http_lua_module related to ssl_certificate_by_lua* execution.
ngx.log() in Lua: Sprinkle ngx.log(ngx.INFO, "message") or ngx.log(ngx.DEBUG, "message") throughout your Lua scripts to trace execution flow and variable states.

openssl s_client: Use this command-line tool to test SSL connections and inspect the certificate served by OpenResty for a specific SNI:

1
2
3
openssl s_client -connect your-openresty-server:443 \
                 -servername specific.hostname.com \
                 -tls1_2 # Or -tls1_3

(See openssl-s_client man page for more options.)

lua_code_cache off; (Development ONLY): In your nginx.conf under the http block, set lua_code_cache off; during development to see Lua code changes without needing to reload Nginx. Never use this in production as it severely degrades performance.
Isolate and Test Lua Modules: Test your Lua modules that fetch and process certificate data independently if possible.

Conclusion

Configuring OpenResty for dynamic SSL certificate loading based on upstream application logic is an advanced task that requires a clear understanding of Nginx’s phase model and careful architectural planning. While directly using an application upstream’s response to select a certificate for the same ongoing SSL handshake is generally infeasible, the two-tier redirect strategy provides a robust and practical solution. This approach leverages OpenResty’s strengths in both HTTP request handling (for the initial upstream call and redirect) and dynamic SSL negotiation via SNI (for the subsequent connection).

Combined with aggressive caching, non-blocking I/O, and thorough error handling, these patterns empower you to build highly flexible and scalable systems that can serve unique SSL certificates dynamically, driven by the sophisticated logic of your backend applications. OpenResty’s programmability with Lua truly shines in these complex, customized web infrastructure scenarios.