Stream jobs are different from stateless APIs. They carry progress, they read from an offset, and they have to pick up cleanly after a restart. In a container platform, restarts are routine, not exceptional. If a restart changes your output, the job does not really own its state. Two ideas fix most of this: store progress outside the process, and make every write safe to repeat.

Options and trade-offs

• Keep progress in memory: fast, and all progress is lost on restart.
• Checkpoint after every record: safest, but the overhead can dominate.
• Checkpoint on a bounded interval and make writes idempotent: a crash replays a little work, and the replay changes nothing.

My recommendation

Write idempotent upserts keyed by a stable record ID, store the checkpoint atomically, and on startup resume from the last checkpoint. Treat exactly-once as a goal you approach by combining at-least-once delivery with idempotent effects.

Worked example

The sketch below has no dependencies. It simulates a crash in the middle of a run, then restarts from the checkpoint. The replayed records are written again, but the output does not change.

import java.io.IOException;
import java.nio.file.*;
import java.util.*;

public class IdempotentJob {

    record Event(String id, long offset, String value) { }

    static final Path CHECKPOINT = Path.of("checkpoint.txt");
    static final Map<String, String> sink = new TreeMap<>();   // upsert by id

    static long readCheckpoint() throws IOException {
        return Files.exists(CHECKPOINT) ? Long.parseLong(Files.readString(CHECKPOINT).strip()) : 0L;
    }

    static void writeCheckpoint(long offset) throws IOException {
        Path tmp = Path.of("checkpoint.tmp");
        Files.writeString(tmp, Long.toString(offset));
        Files.move(tmp, CHECKPOINT, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
    }

    static void run(List<Event> source, long crashAfterOffset) throws IOException {
        long start = readCheckpoint();
        int sinceCheckpoint = 0;
        for (Event e : source) {
            if (e.offset() < start) continue;
            sink.put(e.id(), e.value());                 // idempotent: same key, same value
            if (++sinceCheckpoint == 5) {
                writeCheckpoint(e.offset() + 1);
                sinceCheckpoint = 0;
            }
            if (e.offset() == crashAfterOffset) {
                throw new IllegalStateException("simulated crash at offset " + e.offset());
            }
        }
        writeCheckpoint(source.size());
    }

    public static void main(String[] args) throws IOException {
        Files.deleteIfExists(CHECKPOINT);
        List<Event> source = new ArrayList<>();
        for (int i = 0; i < 20; i++) source.add(new Event("evt-" + i, i, "value-" + i));

        try {
            run(source, 7);                              // crashes mid-run
        } catch (IllegalStateException crash) {
            System.out.println(crash.getMessage() + ", resuming from " + readCheckpoint());
        }
        run(source, -1);                                 // restart: replays 5..7, then finishes
        System.out.println("records in sink = " + sink.size());   // 20, not 23
    }
}

In a container, the checkpoint file would live on a volume or, better, in an external store, so it survives the container being replaced. The same idea applies when the source is a message broker with committed offsets and the sink supports upserts or transactions.

Checklist

1. Persist progress outside the container.
2. Write the checkpoint atomically, so a crash never leaves it half-written.
3. Key writes by a stable ID, and make repeating them harmless.
4. On startup, read the checkpoint and resume, and log what you recovered without logging payloads.
5. Pace catch-up after an outage with bounded batches and concurrency.
6. Test by killing the process mid-run and comparing the output with an uninterrupted run.

Where this doesn't apply

This sketch is single-threaded with an in-memory sink, to show the mechanics. Real jobs also need partitioning, ordering guarantees per key, and a sink that is itself durable.

One honest limitation

Idempotency limits what a write can be. Operations like incrementing a counter are not safe to repeat, so they need a composite key, a version, or a transactional sink to be handled correctly.

Sources

• Apache Kafka Documentation: Message delivery semantics
• Oracle Java Documentation: java.nio.file.Files.move (atomic move)
• Docker Docs: Volumes

• Field names, types, and optionality.
• HTTP status codes and which situations produce them.
• The error envelope: its fields and their meaning.
• Batch semantics: partial failures, ordering, and duplicates.
• Behavior when a request omits optional parameters, such as projections.

Be a tolerant reader

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.databind.JsonNode;

@JsonIgnoreProperties(ignoreUnknown = true)
public record ErrorEnvelope(int status, String message, String code, JsonNode errorDetails) {

    public ErrorEnvelope {
        message = message == null ? "" : message;
    }
}

• ignoreUnknown = true protects you from new server fields.
• errorDetails stays a raw JsonNode. If the server's details are free-form, do not force them into a fixed class. Keep them untyped, and let the caller interpret them.
• Missing message becomes an empty string, so downstream code does not need null checks.

Classify errors in one place

public enum ErrorKind { CLIENT_ERROR, NOT_FOUND, UNPROCESSABLE, SERVER_ERROR, UNKNOWN }

public static ErrorKind classify(int status) {
    if (status == 404) return ErrorKind.NOT_FOUND;
    if (status == 422) return ErrorKind.UNPROCESSABLE;
    if (status >= 400 && status < 500) return ErrorKind.CLIENT_ERROR;
    if (status >= 500 && status < 600) return ErrorKind.SERVER_ERROR;
    return ErrorKind.UNKNOWN;
}

Handle batch responses carefully

public static <V> BatchOutcome<V> split(Map<String, V> results, Map<String, ErrorEnvelope> errors) {
    Map<String, V> ok = new LinkedHashMap<>(results);
    Map<String, ErrorKind> failed = new LinkedHashMap<>();
    errors.forEach((id, env) -> {
        ok.remove(id);
        failed.put(id, classify(env.status()));
    });
    return new BatchOutcome<>(ok, failed);
}

public record BatchOutcome<V>(Map<String, V> succeeded, Map<String, ErrorKind> failed) { }

Server-side rules for safe evolution

• Add fields as optional. Never remove or repurpose a field without a deprecation period.
• Do not change status codes for existing situations.
• Do not change the shape of the error envelope. Add to it instead.
• Be careful with requests that send no projection or field selection. Defaults are part of the contract.
• Version deliberately when you must break something, and announce it.

Golden-file tests

import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

class ErrorEnvelopeTest {

    private final ObjectMapper mapper = new ObjectMapper();

    @Test
    void parsesCurrentShape() throws Exception {
        String json = """
            {"status":422,"message":"invalid","code":"VALIDATION","errorDetails":{"field":"email"}}
            """;
        ErrorEnvelope env = mapper.readValue(json, ErrorEnvelope.class);
        assertEquals(422, env.status());
        assertEquals("email", env.errorDetails().get("field").asText());
    }

    @Test
    void ignoresFieldsAddedLater() throws Exception {
        String json = """
            {"status":404,"message":"gone","futureField":{"x":1}}
            """;
        assertEquals(404, mapper.readValue(json, ErrorEnvelope.class).status());
    }

    @Test
    void missingOptionalFieldsDoNotFail() throws Exception {
        ErrorEnvelope env = mapper.readValue("{\"status\":500}", ErrorEnvelope.class);
        assertEquals("", env.message());
        assertNull(env.code());
    }
}

Failure modes

• A status change (for example 400 to 422) with no announcement.
• A strict client parser that rejects new fields.
• Error details forced into a typed structure that drops data.
• Batch handlers that fail the whole batch or hide partial failures.
• Requests without projections returning a different shape than before.

Demo to build

Takeaways

• The contract includes errors, status codes, and batch behavior.
• Clients should be tolerant readers and keep free-form details untyped.
• Golden-file tests turn accidental contract changes into visible diffs.

Compartments split resources and policy scope, and the layout you choose on day one is hard to change quietly. When a compartment moves, the set of policies that apply to it can change with it, and people can gain or lose access without anyone editing a statement.

I wanted a hierarchy that makes moves boring and lets me check the result afterwards.

Options and trade-offs

• A flat design: simple, with few movement cases and limited delegation.
• A hierarchical design: flexible, with inherited access, and it needs careful names and ownership.
• Policies attached at a shared ancestor: they adjust automatically when a descendant moves under the same ancestor, and lower-level policies may need manual edits.

My recommendation

Use a clear hierarchy with named owners, and attach policies at an ancestor that will survive the moves you expect. After any move, verify access from the point of view of the people involved and fix anything that attached lower down.

Worked example

Take a move from A:B:C to A:D. Policies written at A, the common ancestor, keep working and update their path references. A policy attached inside B that named C by path no longer matches its target, and the access it granted disappears until someone rewrites it.

# Attached at A (the shared ancestor): survives the move
Allow group TeamA-Admins to manage all-resources in compartment A:Prod
Allow group TeamA-Users  to use instance-family in compartment A:Dev
 
# Attached inside A:B and naming C by path: must be reviewed after the move
Allow group TeamC to use instance-family in compartment A:B:C

Before a move, list the policies that mention the compartment and note which ancestor each is attached to. After the move, check access with a test user for each group. Quotas and budgets are set per compartment, so confirm them too.

Checklist

1. Define the hierarchy, names and owners before creating compartments.
2. Attach policies at ancestors that will not move.
3. Before a move, list the policies that reference the compartment.
4. After a move, test access for every affected group.
5. Use quotas and budgets at the compartment level as guardrails.
6. Avoid duplicate compartment names under the same parent.

Where this doesn't apply

Security zones have their own rules, and a move that crosses a zone boundary is restricted. Manage those through the Security Zones service.

One honest limitation

Changes in implicit access are easy to miss. Treat a move as a planned change with a validation step afterwards, and do not call it finished until you have checked.

Sources

• OCI Documentation: Managing Compartments (moving a compartment)
• OCI Documentation: Policy Syntax (compartment paths)
• OCI Documentation: Compartment Quotas and Budgets

Issues should track work. Discussions suit open-ended questions, ideas, and announcements. Discussions supports categories with different formats, pinned threads, accepted answers, upvotes, and conversion between issues and discussions. Defining these early keeps the work queue clean and reduces duplicate issues.

Options and trade-offs

• Categories: group conversations by purpose. Each category can have a format such as announcement, Q&A, or open-ended. Check the current category limit in the docs.
• Pinning: highlight canonical threads such as the FAQ or onboarding guide.
• Accepted answers and upvotes: surface solutions and priorities. They need moderator attention to stay useful.
• Conversion: move a thread from issue to discussion, or back, when its nature changes.

My take

Keep a small set of categories, pin the threads newcomers need first, and use a category form for Q&A so questions arrive with the details needed to answer them.

Decision lens: new-contributor friction and review latency.

Example setup: categories for Announcements, Q&A, and Ideas. The FAQ and a first-thirty-minutes guide are pinned. The Q&A category uses a form that asks for version, steps to reproduce, and expected versus actual behavior.

Template

.github/DISCUSSION_TEMPLATE/qna.yml: a category form with prompts for version, steps, expected behavior, and actual behavior.

Welcome post: the purpose of the space, the code of conduct, and the rules for converting between issues and discussions.

Category setup: Announcements restricted to maintainers, and Q&A with accepted answers enabled.

Checklist

1. Enable Discussions and create categories.
2. Pin the FAQ and onboarding threads.
3. Add a Q&A category form.
4. Write down the conversion rules between issues and discussions.
5. Review the category setup each quarter.

Common mistakes

• Running without categories, which makes threads hard to find.
• Not pinning canonical answers, so the same questions keep coming back.
• Never converting issues, so the work queue fills with general chat.

Where this doesn't apply

• Organizations required to use an external forum or ticketing system.

How to get involved

• Good first issue: propose a new pinned FAQ entry.
• Discussion: what should the Q&A form ask first?

Sources

• Managing categories for discussions: https://docs.github.com/en/discussions/managing-discussions-for-your-community/managing-categories-for-discussions. Formats and limits.
• Managing discussions: https://docs.github.com/en/discussions/managing-discussions-for-your-community/managing-discussions. Pinning, permissions, and transfers.
• Collaborating with maintainers using discussions: https://docs.github.com/en/discussions/collaborating-with-your-community-using-discussions/collaborating-with-maintainers-using-discussions. Accepted answers, upvotes, and conversion.
• Category forms: https://docs.github.com/en/discussions/managing-discussions-for-your-community/creating-discussion-category-forms.

---

Support tickets, scanned forms and screenshots all contain personal data, and the people asking for automated triage rarely want to train a model. OCI Language and Vision offer pretrained models behind regional endpoints, which is a good fit when the goal is to route and summarize rather than to research.

The design question is where personal data goes. I wanted the masking to happen before anything gets stored or forwarded, and I wanted a trail showing which call touched which document.

Options and trade-offs

• Language service (entities, sentiment, key phrases, PII detection): good for text pipelines, as long as PII is masked before downstream storage.
• Vision (document and image analysis, text detection, object detection): good for PDFs and images, with batch jobs and custom models available.
• Inline versus asynchronous jobs: inline is simpler for a small payload, and batch jobs backed by Object Storage scale better.

My recommendation

Use inline calls for small, interactive requests and Object Storage backed jobs for volume. Run PII masking first in the text path. Store only the masked output, keep request identifiers for audit, and set a retention period before the first production run.

Worked example

Masking personal data in a piece of text with the Language service. This uses the PII entity detection call with a masking option, and it needs a Python environment with the OCI SDK configured for your tenancy.

import oci
 
config = oci.config.from_file()
language = oci.ai_language.AIServiceLanguageClient(config)
 
doc = oci.ai_language.models.TextDocument(
    key="example-document", text=ticket_text, language_code="en")
 
resp = language.batch_detect_language_pii_entities(
    oci.ai_language.models.BatchDetectLanguagePiiEntitiesDetails(
        documents=[doc], masking={"ALL": {"mode": "MASK"}}))
 
masked = resp.data.documents[0].masked_text   # store this, not ticket_text

Analyzing a scanned document in Object Storage with Vision. The result is text you can pass to the masking step above.

vision = oci.ai_vision.AIServiceVisionClient(config)
 
result = vision.analyze_document(
    oci.ai_vision.models.AnalyzeDocumentDetails(
        compartment_id=compartment_ocid,
        document=oci.ai_vision.models.ObjectStorageDocumentDetails(
            source="OBJECT_STORAGE",
            namespace_name=namespace,
            bucket_name=input_bucket,
            object_name="scan-0007.pdf"),
        features=[oci.ai_vision.models.DocumentTextDetectionFeature(
            feature_type="TEXT_DETECTION")]))
 
print(result.headers.get("opc-request-id"))   # keep for the audit trail

For larger volumes, create a document job through the CLI or SDK with input and output locations in Object Storage, then poll for completion. Run the workload as an instance or resource principal and grant it only the Language, Vision and bucket access it needs.

Checklist

1. Mask or remove PII before storing any text.
2. Use inline calls for small payloads and batch jobs for volume.
3. Keep outputs only as long as you need them.
4. Record request IDs and compartment ownership for audit.
5. Check supported file formats and languages before choosing a feature.
6. Use instance or resource principals instead of keys in code.

Where this doesn't apply

Fully air-gapped environments cannot call these cloud endpoints. They need models that run locally instead.

One honest limitation

Bulk analysis produces a lot of metadata and intermediate output. Decide on redaction and retention rules up front, or the results bucket turns into its own data-protection problem.

Sources

• OCI Documentation: Language, Detecting Personally Identifiable Information
• OCI Documentation: Vision, Document AI Overview
• OCI Python SDK Reference: AIServiceLanguageClient and AIServiceVisionClient

Many apps verify identity attributes with a bespoke token: a signed JWT that only the issuing system can produce and only your app can parse. That locks the user into one issuer's flow and forces every verifier to build a custom integration. Microsoft Entra Verified ID uses open standards (decentralized identifiers, W3C verifiable credentials, OpenID for Verifiable Credentials) so a person can hold a credential in a wallet, such as Microsoft Authenticator, and present it to any verifier that trusts the issuer.

Options and trade-offs

• Issuer setup: bind a decentralized identifier to your domain (did:web), keep signing keys in Key Vault, grant the Request Service the permissions it needs, and define a credential contract (display and rules). The result is portable credentials, at the cost of setup steps and a dependency on a wallet.
• Verifier setup: a separate app, ideally in a separate tenant from the issuer, that creates presentation requests, receives callbacks and validates what comes back. It gets privacy-respecting, user-approved presentations instead of passwords or shared secrets.
• Hybrid designs: where a server must query an issuer continuously, user presentation is a poor fit. Combine it with server-side checks instead of replacing them.

My recommendation

Migrate one low-risk attribute first, such as proof of employment or membership, and keep the existing check running in parallel. Verify the credential's issuer, signature and status, and accept only the claims you need. Expand once the flow is reliable.

Minimal working example

The verifier flow has three parts: get a token for the Request Service, create a presentation request, and handle the callback.

Step 1: get an access token with the client credentials flow, using the Verifiable Credentials Service Request scope (confirm the exact scope value in the current docs):

curl -s -X POST "https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token" \
  -d "client_id=<app-id>" \
  -d "client_secret=<secret-or-use-a-certificate>" \
  -d "grant_type=client_credentials" \
  -d "scope=3db474b9-6a0c-4840-96ac-1fceb342124f/.default"

Step 2: create a presentation request:

POST https://verifiedid.did.msidentity.com/v1.0/verifiableCredentials/createPresentationRequest
Content-Type: application/json
Authorization: Bearer <token>

{
  "authority": "did:web:verifier.example.com",
  "includeQRCode": true,
  "registration": { "clientName": "Example Verifier" },
  "callback": {
    "url": "https://verifier.example.com/api/presentation-callback",
    "state": "<unique-request-id>",
    "headers": { "api-key": "<shared-callback-secret>" }
  },
  "requestedCredentials": [
    {
      "type": "ProofOfEmployment",
      "purpose": "Confirm your employer before granting access",
      "acceptedIssuers": [ "did:web:issuer.example.com" ]
    }
  ]
}

The response includes a QR code and a deep link for the wallet. Step 3: handle the callback and authorize only on a verified presentation:

// Callback handler (framework-agnostic sketch)
void onCallback(String apiKeyHeader, JsonNode body) {
    if (!MessageDigest.isEqual(apiKeyHeader.getBytes(UTF_8), EXPECTED_KEY.getBytes(UTF_8))) {
        throw new SecurityException("bad callback key");
    }
    String status = body.path("requestStatus").asText();
    switch (status) {
        case "request_retrieved"      -> markScanned(body.path("state").asText());
        case "presentation_verified"  -> grantAccess(body.path("state").asText(),
                                                     body.path("verifiedCredentialsData"));
        case "presentation_error"     -> markFailed(body.path("state").asText());
        default                       -> { /* ignore unknown statuses */ }
    }
}

Test it: run the flow with a test credential and confirm access is granted only after presentation_verified. Then present a credential from an issuer that is not in acceptedIssuers and confirm it is rejected.

Checklist

• Use did:web with domain binding, serve it over HTTPS, and avoid redirects on the well-known path.
• Keep signing keys in Key Vault and plan rotation before you need it.
• Check the issuer DID against an allowlist, and confirm the credential status, not just the signature.
• Use a unique state value per request, and authenticate callbacks with a secret you compare in constant time.
• Request only the claims you will use, and explain the purpose to the user.
• Align on the supported standards (VC Data Model, OpenID4VC, Presentation Exchange) so you can work with other wallets and verifiers.

Where this doesn't apply

Scenarios that need continuous server-side queries to an issuer rather than a user presenting a credential. Use a hybrid design there.

Sources

• Verified ID introduction and standards
• Request Service REST API presentation specification
• Configure your tenant for Verified ID

---

Some AI workloads need runtime confidentiality: protect model artifacts and user data from the host OS/kernel and operators. AKS confidential containers (kata-cc) isolate the pod in a confidential utility VM with attestation and policy controls. Trade-offs include startup time, debugging limitations, and preview status constraints.

Options and trade-offs

• AKS confidential VM node pool: Node-level boundary; simpler lift-and-shift; not per-pod sandbox.
• AKS confidential containers (kata-cc): Pod-level confidential sandbox, policy generator and attestation; more operational constraints; supports Azure Linux and specific VM SKUs; preview limitations apply.
• ACI confidential containers: Container group boundary; suitable for event-driven jobs; fewer orchestration features than AKS.

Minimal working example

Prereqs

• Azure CLI 2.44.1+; aks-preview and confcom extensions; Azure Linux; KataCcIsolation on node pool; enable workload identity and OIDC issuer if key release requires identity linkage.

Create cluster (example)

az aks create --resource-group rg --name aks-cc \
  --kubernetes-version 1.25.0 \
  --os-sku AzureLinux \
  --node-vm-size Standard_DC8as_cc_v5 \
  --workload-runtime KataCcIsolation \
  --enable-oidc-issuer --enable-workload-identity

Deploy consumer with attestation sidecar and policy Generate policy with confcom; apply as annotation; tie Key Vault key release to attestation measurement. Test: Remove kata-cc spec and policy; confirm ciphertext (no key release).

Checklist

• Confirm supported SKUs and OS; Azure Linux only for kata-cc at present.
• Enable and maintain policy (do not use :latest tags because layers are measured).
• Expect longer pod startup and limited troubleshooting (exec/stdio require policy change).
• Integrate attestation with Key Vault for secure key release.

Where this doesn't apply

Windows node pools; or workloads requiring frequent interactive debugging incompatible with kata-cc constraints.

Sources

• Confidential containers (Preview)
• Deploy AKS with confidential containers policy
• Confidential containers overview (Azure confidential computing)

---

UCP has dozens of properties, and the defaults work. What I wanted was predictable behavior under stress: callers that fail fast instead of hanging, bad connections that are caught before they reach the application, and a pool that does not quietly use up the database's session budget.

Options and trade-offs

• A small pool with a modest maximum: gentle on the database, and it may add latency while the pool warms up.
• A larger minimum or initial size: faster first requests, and idle sessions you are paying for.
• Validation on borrow: safer, with a small cost on each borrow. A short trust window reduces that cost.

My recommendation

Set the minimum and maximum sizes on purpose, turn on validation on borrow with a short trust window, and give callers a wait timeout. Then measure and adjust. Change one property at a time, so you know which change helped.

Worked example

Start from the database's limit and work backwards. If a database can comfortably serve 200 sessions for this application and you run 8 service instances, a maximum of 20 per instance already reaches 160, which leaves room for failover and administration.

PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setURL("<jdbc_url>");
pds.setUser(System.getenv("DB_USER"));
pds.setPassword(System.getenv("DB_PASSWORD"));
 
// Sizing
pds.setInitialPoolSize(2);
pds.setMinPoolSize(2);
pds.setMaxPoolSize(20);
 
// Resilience
pds.setValidateConnectionOnBorrow(true);
pds.setSecondsToTrustIdleConnection(5);   // skip re-validation for 5 s
pds.setConnectionWaitTimeout(3);          // seconds a caller waits for a connection
pds.setInactiveConnectionTimeout(300);    // release idle connections after 5 min

To watch the pool, log the borrowed and available counts on a timer, or expose them through JMX:

log.info("ucp borrowed={} available={}",
         pds.getBorrowedConnectionsCount(),
         pds.getAvailableConnectionsCount());

If borrowed stays close to the maximum and callers start timing out, the first suspect is a slow query or a leaked connection, not the pool size.

Checklist

1. Derive the maximum from the database's session budget and the number of instances.
2. Turn on validation on borrow, and use a short trust window if the overhead matters.
3. Set a connection wait timeout so callers cannot hang.
4. Log or export the borrowed, available and waiting counts.
5. Test failover and a database restart.
6. Revisit sizes after load testing, in small steps.

Where this doesn't apply

Large Oracle RAC deployments that rely on Fast Application Notification and replay need high-availability features configured, and that is a separate tuning exercise.

One honest limitation

Validation costs a little on every borrow. Only widen the trust window after the basic reliability picture is clear, or you may hide the very failures you wanted to catch.

Sources

• Oracle Universal Connection Pool Developer's Guide: Setting Connection Pool Properties
• Oracle UCP Java API Reference: PoolDataSource
• Oracle UCP Developer's Guide: Monitoring the Pool with JMX

A local demo is easy. A public one raises new questions: which port is exposed, how the platform knows the app is healthy, what the first request does after a quiet night, and how to be sure a traffic spike does not turn into a surprise bill. You do not need a cluster for this. A managed container platform with scale-to-zero gives you a URL that costs almost nothing when idle.

Options and trade-offs

• Run a VM and keep it on: simple, but you pay for the idle hours.
• Run a cluster: powerful, and far more than a demo needs.
• Use a managed container platform with scale-to-zero: you accept a cold start in return for near-zero idle cost.

My recommendation

Make the app platform-friendly first: it listens on a configurable port, has a health endpoint, and redirects the root path somewhere useful. Then deploy a digest-pinned image with a minimum of zero replicas, managed TLS, and budget alerts set well below the ceiling.

Worked example

A tiny Java server with no dependencies. It reads the port from the environment, answers a health check, and redirects the root path to the docs.

import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpServer;

import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;

public class Main {
    public static void main(String[] args) throws IOException {
        int port = Integer.parseInt(System.getenv().getOrDefault("PORT", "8080"));
        HttpServer server = HttpServer.create(new InetSocketAddress(port), 0);

        server.createContext("/health", ex -> respond(ex, 200, "application/json", "{\"status\":\"ok\"}"));
        server.createContext("/docs", ex -> respond(ex, 200, "text/html", "<h1>Docs</h1>"));
        server.createContext("/", ex -> {
            if ("/".equals(ex.getRequestURI().getPath())) {
                ex.getResponseHeaders().add("Location", "/docs");
                ex.sendResponseHeaders(302, -1);
                ex.close();
            } else {
                respond(ex, 404, "text/plain", "not found");
            }
        });
        server.start();
    }

    private static void respond(HttpExchange ex, int status, String type, String body) throws IOException {
        byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
        ex.getResponseHeaders().add("Content-Type", type);
        ex.sendResponseHeaders(status, bytes.length);
        try (OutputStream out = ex.getResponseBody()) {
            out.write(bytes);
        }
    }
}

The deployment description is illustrative, since every platform has its own field names, but the shape is the same everywhere:

{
  "name": "demo-app",
  "image": "registry.example.com/team/app@sha256:<digest>",
  "containerPort": 8080,
  "healthCheckPath": "/health",
  "ingress": { "tls": true },
  "autoscale": { "minReplicas": 0, "maxReplicas": 3 }
}

docker compose up --build
curl -i http://127.0.0.1:8080/
curl -s http://127.0.0.1:8080/health

Checklist

1. Read the port from the environment, and expose the one the platform expects.
2. Add a cheap health endpoint, and point the platform at it.
3. Redirect the root path to something useful.
4. Deploy by digest, with managed TLS.
5. Set minimum replicas to zero, and test the first request after idle.
6. Set a budget alert well under your ceiling, and send it somewhere you will see it.

Where this doesn't apply

If you need steady low latency, scale-to-zero is the wrong choice, because the first request after a quiet period will be slow.

One honest limitation

A cold start on a JVM service can take seconds. You can soften it with a smaller image and class-data sharing, but you cannot remove it, so decide whether a slow first visit is acceptable for the demo.

Sources

• Docker Docs: Compose file reference (ports)
• Docker Docs: Dockerfile HEALTHCHECK
• Oracle Java Documentation: com.sun.net.httpserver package

External roadmap tools fall out of date because they are not connected to issues and pull requests. GitHub Projects sits alongside issues and PRs and gives each team a view suited to its question: a prioritized backlog, the current iteration board, or a timeline roadmap.

Options and trade-offs

• Table view: dense and good for prioritization. It needs consistent field values.
• Board view: a status-based layout for iteration planning. It works best with work-in-progress limits.
• Roadmap view: a timeline across dates or iterations. It suits communicating schedules to others.
• Built-in workflows: add items, set status, and archive without code. They are limited compared with custom automation.
• Insights: charts for status and progress. Useful for a quick read, not a full reporting tool.

My take

Use one organization-level project for the platform roadmap, keep tactical bug work out of it, and save a small set of views that answer the questions people actually ask.

Decision lens: review latency and team workload.

Example setup: a platform roadmap project with Priority, Estimate, Iteration, and Owner fields, auto-add filters for the relevant repositories, and three saved views: backlog by priority, the current iteration board, and the quarter roadmap.

Template

Project fields: a single-select Priority field, a number field for Estimate, and an iteration field.

Views: Table for the backlog, Board for the current iteration, and Roadmap for milestones.

Workflows: auto-add issues with a chosen label, set new items to Todo, and archive items that have been Done for a set period.

Project README: link the saved views and describe the naming and labeling rules.

Checklist

1. Create an organization project and define the fields.
2. Save table, board, and roadmap views.
3. Configure auto-add and status workflows.
4. Add milestone markers to the roadmap view.
5. Document label and ownership conventions.
6. Set work-in-progress limits on the board.

Common mistakes

• Mixing bug triage with strategic planning in one project. Split them.
• Not saving views, so collaborators lose context.
• Skipping automation, which lets status drift from reality.

Where this doesn't apply

• Teams required to use an external project management system. Mirror the data through the API if needed.

How to get involved

• Good first issue: propose a saved view that would improve backlog triage.
• Discussion: what fields should the roadmap track?

Sources

• About Projects: https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects. Views, fields, automation, and insights.
• Quickstart for Projects: https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/quickstart-for-projects. Creating views, fields, and workflows.
• Roadmaps generally available: https://github.blog/changelog/2025-03-22-roadmaps-in-projects-are-now-generally-available/. Roadmap view status and markers.

---

When every team calls the model endpoint directly, governance is a spreadsheet: who has access, how much they use, and what they send. Putting a gateway in front of the model gives you one place to authenticate callers, enforce limits and log usage. Azure API Management (APIM) can authenticate to Azure OpenAI with its own managed identity, so no API keys are stored or shared, and it can validate caller tokens before anything reaches the model.

Options and trade-offs

• Managed identity to the backend: APIM holds the identity, callers hold nothing that works against the model directly. It is simple for service-to-service access, and every caller shares the same backend privileges.
• OAuth 2.0 validation at the gateway: callers present Entra tokens, and APIM checks audience and roles before forwarding. You get per-caller control and audit, with more setup.
• Per-caller limits: rate limits keyed on the caller's identity stop one team's runaway loop from using the shared quota.

My recommendation

Use both layers. Validate the caller's token at the gateway, then use the gateway's managed identity to call the backend. Add a rate limit keyed on the caller. Disable key-based access on the model resource once traffic flows through the gateway, so there is no way around it.

Minimal working example

Enable a managed identity on the APIM instance and grant it access to the model resource:

az role assignment create \
  --assignee <apim-principal-id> \
  --role "Cognitive Services OpenAI User" \
  --scope <azure-openai-resource-id>

Then add the inbound policy. It validates the caller, limits usage, and swaps the caller's token for the gateway's backend token:

<inbound>
  <base />

  <!-- 1. Validate the caller's Entra token -->
  <validate-jwt header-name="Authorization" failed-validation-httpcode="401"
                failed-validation-error-message="Unauthorized">
    <openid-config url="https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration" />
    <audiences>
      <audience><gateway-app-id></audience>
    </audiences>
    <required-claims>
      <claim name="roles" match="any">
        <value>LLM.Invoke</value>
      </claim>
    </required-claims>
  </validate-jwt>

  <!-- 2. Limit usage per caller -->
  <rate-limit-by-key calls="60" renewal-period="60"
      counter-key="@(context.Request.Headers.GetValueOrDefault("Authorization","").AsJwt()?.Subject)" />

  <!-- 3. Authenticate to the backend with the gateway's managed identity -->
  <authentication-managed-identity resource="https://cognitiveservices.azure.com"
      output-token-variable-name="managed-id-access-token" ignore-error="false" />
  <set-header name="Authorization" exists-action="override">
    <value>@("Bearer " + (string)context.Variables["managed-id-access-token"])</value>
  </set-header>
</inbound>

Test it: call the gateway with a valid token and confirm a normal response, call without a token and confirm a 401, and call in a loop to confirm the rate limit returns 429.

Checklist

• Use the model resource's custom subdomain endpoint, and confirm the role assignment scope.
• Validate the audience and a role claim at the gateway, not just that a token exists.
• Key the rate limit on the caller's identity, not on an IP address that may be shared.
• Disable local (key) authentication on the model resource after cutover.
• Log request metadata for audit, and keep prompt and completion content out of default logs.
• Check that your policy ordering does what you expect: validation first, then limits, then backend authentication.

Where this doesn't apply

Backends that do not support Entra authentication or gateway integration. There, the gateway still helps with limits and audit, but backend authentication needs another design.

Sources

• Authenticate and authorize to LLM APIs with Azure API Management
• authentication-managed-identity policy

The failure mode

A verifier calls a central revocation endpoint for every check. The endpoint sees who asked (IP or key), when, for which credential ID, and what outcome occurred. Even if the credential is privacy-preserving, the revocation lookup becomes a “phone home” channel that leaks context. Users and regulators won’t love that.

What I learned

Understand the revocation design space

Status list (offline)

The issuer publishes a privacy-preserving list of credential status values (for example, a bitstring status list) that verifiers fetch periodically. Verification checks against the local list.

Online status check

The verifier queries a status endpoint per credential. Freshness improves; privacy worsens.

Holder stapling

The holder obtains a fresh status attestation and presents it along with the credential. The verifier checks the stapled proof without contacting the issuer per verification.

Short-lived credentials:

Expire credentials quickly so revocation windows are short. This reduces reliance on revocation checks.

Privacy-preserving lists: batch and compress

A status list can represent many credentials without revealing individual identities. Verifiers download the list on a schedule (e.g., every policy-defined freshness window), cache it locally, and check bits/entries offline. Lists can be partitioned or hashed to limit list size.

Stapling shifts “phone home” to holder flow

With stapling, the holder requests a status proof from the issuer (or a delegated service) at a cadence, then presents it to verifiers. This avoids per-verification lookups by verifiers but still requires the holder to contact the issuer on some schedule. The privacy difference is who calls whom and when.

Short-lived credentials reduce time at risk

If credentials expire quickly, revocation needs are less acute. This shifts load from revocation infrastructure to issuance (more frequent renewals). Combine with holder stapling to keep verification fast and private.

Model cache freshness and failure modes

Verifiers should track the freshness of status lists. If a list is too old (e.g., beyond your policy window), treat the result as “unverifiable” rather than “accepted,” and prompt re-fetch or stapling. Distinguish “cache stale” from “revoked.”

Avoid per-ID URLs in logs

If you must hit an online status endpoint, avoid logging per-ID lookups in a way that identifies a particular holder or verifier. Aggregate metrics instead. Keep any necessary identifiers hashed or pseudonymous.

Distinguish revocation from expiry and audience

Revocation says “this specific credential is no longer valid.” Expiry says “credentials of this type are only valid until T.” Audience says “this verifier is permitted to consume the claim.” Don’t overload revocation with other policy failures.

Illustrative status-list freshness check

from dataclasses import dataclass
from datetime import datetime, timezone

@dataclass(frozen=True)
class StatusSnapshot:
    fetched_at: datetime
    bits: bytes

def is_fresh(snapshot: StatusSnapshot, max_age_seconds: int) -> bool:
    age = datetime.now(timezone.utc) - snapshot.fetched_at
    return age.total_seconds() <= max_age_seconds

# Choose the freshness window from policy and operational requirements.
# Stale status should have an explicit outcome, such as "unverifiable".

Trade-offs and failure modes

Trade-offs

Offline lists vs freshness: Lists preserve privacy but can be stale. Address with frequent, batched updates.

Stapling vs issuer load: Stapling shifts calls to holders; issuer still handles periodic status proofs. Load is manageable with caching and rate limits.

Short-lived credentials vs user friction: Frequent renewals add UX overhead. Balance with clear prompts and automated refresh flows.

Failure modes

“Phone home”: Per-verification lookups leak context. Solution: offline lists or stapling.

“Stale cache”: Verifiers use outdated lists. Solution: freshness checks and policy windows that treat stale as “unverifiable.”

“Revocation conflation”: Treating expiry or audience failure as revocation. Solution: separate policy checks.

Try it yourself

Implement an offline status list with a freshness window. Treat stale lists as “unverifiable.”

Add holder stapling to your demo: the holder fetches a status proof and presents it alongside the credential.

Track performance for offline list checks vs online status calls. Compare privacy and tail behavior.

The failure mode

Your aggregator treats null as “0.0,” sums four dimensions, and displays a neat number. A caller misreads that as “complete and low.” In reality, two dimensions were missing, and one was unverifiable. The number masked uncertainty.

What I learned

Keep partial states in the domain layer, not just at the edges

Don’t convert unknowns to defaults at the boundary. Let domain logic operate on sealed types. If you must produce a single number, force policy conversion in a dedicated function so that the decision is auditable.

Policy example

public final class AggregationPolicy {
public static double conservativeAverage(List<ScoreOrUnknown> dims) {
int count = 0;
double sum = 0.0;
for (ScoreOrUnknown d : dims) {
switch (d) {
case Score sc         -> { sum += sc.value(); count++; }
case Missing miss     -> { /* exclude; track missing elsewhere */ }
case Unverifiable unv -> { /* exclude; track unverifiable */ }
}
}
if (count == 0) return Double.NaN; // no numeric evidence
return sum / count;
}
public static boolean okToDisplay(List<ScoreOrUnknown> dims) {
long unknowns = dims.stream().filter(x -> !(x instanceof Score)).count();
return unknowns == 0; // or choose a threshold
}
}

Propagate uncertainty deliberately

Unknowns should travel with the data: into logs, decisions, and UI. For example, if two dimensions are unknown, surface that count alongside any numeric aggregation. The goal is to prevent silent conversions that distort confidence.

Make unknown-handling policies explicit and testable

Write tests that assert what happens when all dimensions are unknown, when some are missing, and when a single unverifiable dimension appears. Pattern matching keeps these tests aligned with the domain states. Don’t assume a “best effort” numeric default; decide and encode.

Resist “default values” in DTOs

If you serialize to JSON, include a type field in each dimension and omit value when the state isn’t Score. A consumer reading {"type":"Missing","reason":"timeout"} can act accordingly. Defaulting to 0.0 or “unknown=false” invites misuse.

Teach callers to branch by type, not numeric checks Documentation should show switch examples, not “if value is 0 then unknown.” This reduces coupling and preserves room for future states (e.g., “Contested,” “Deprecated”).

Audit conversion points with a linelist List every place where ScoreOrUnknown becomes a single number or a boolean. Audit these conversions periodically. If you discover accidental flattening, refactor to preserve states or add proper policy functions.

Accept that aggregation may return “no number” Returning NaN or omitting a field is reasonable when inputs are fully unknown. Treat this as normal, not exceptional. UI can show “insufficient evidence” with counts; APIs can document this outcome clearly.

JSON serialization with a type field

// Using a simple custom serializer sketch

public record Score(double value) implements ScoreOrUnknown {}
public record Missing(String reason) implements ScoreOrUnknown {}
public record Unverifiable(String note) implements ScoreOrUnknown {}
public Map<String, Object> toJson(ScoreOrUnknown s) {
return switch (s) {
case Score sc         -> Map.of("type", "Score", "value", sc.value());
case Missing miss     -> Map.of("type", "Missing", "reason", miss.reason());
case Unverifiable unv -> Map.of("type", "Unverifiable", "note", unv.note());

};

}

Trade-offs and failure modes

Trade-offs

Transparency vs brevity: Returning types and reasons increases payload size but improves clarity. Favor clarity when ambiguity matters.

Strictness vs convenience: Throwing on invalid scores and disallowing defaults catches bugs early but may frustrate callers used to flexible shapes.

NaN vs explicit omission: NaN signals “no numeric value” but can be mishandled by consumers. Omitting a field or providing a separate state field avoids confusion.

Failure modes

“Silent flattening”

A downstream transforms unknowns to zero—likely out of habit. Catch this in contract tests.

“Enum drift”

If reason codes evolve, consumers break. Document codes and provide a safe default handling path.

“Partial measures hiding in averages”: An average of one numeric with three unknowns looks “complete” unless you also surface unknown counts.

Try it yourself

Refactor one DTO to embed a sealed state rather than a nullable numeric field. Add a type field in JSON. Update consumers to switch on type.

Write aggregation policies that return NaN or omit fields when inputs are fully unknown. Test behavior explicitly.

Add metrics that count unknowns and unverifiable states separately from numeric distributions.

A language-model server keeps attention keys and values for every token of every active sequence. That memory grows with context length and concurrency. Unlike model weights, it is not a fixed cost, so a deployment that looks fine in a short test can run short of memory under a different traffic mix.

OpenVINO Model Server lets you set the cache memory with --cache_size, and its documentation examples pass explicit values. Choosing that number deliberately is the heart of governing the cache.

The friction points are picking a limit, observing real usage against it, documenting the number and its basis, and deciding in advance what the service does when a request does not fit.

Options and trade-offs

• Explicit limit: pass --cache_size in gigabytes. The documentation's examples use values such as 20 on a large CPU system and 2 in a GPU example, which shows that the right value depends on the model, precision, and hardware.
• Prefix reuse: with prefix caching enabled, repeated prompt prefixes avoid recomputation. A larger cache can hold more reusable history, so cache size and the benefit of prefix caching are linked.
• Concurrency and context length are the two drivers of cache growth. Limit the maximum prompt size and the number of simultaneous requests at your gateway so the cache you chose is a realistic fit.
• Observation: use operating-system and container tools to watch the server process's memory over time. You do not need special server features to see a trend and compare it with the limit you set.
• Overflow policy: decide whether the service should reject oversize requests with a clear error, shorten context, or queue. Silent failure or a crash is the outcome to avoid.
• Model and precision changes: a different model or precision changes memory per token, so a limit chosen for one is not valid for another.
• Recommendation: start with a conservative value in the style of the documentation's examples, run a realistic mix of context lengths and concurrency, and watch the memory trend. Increase the cache until your largest realistic request fits with headroom, and stop there. Write the limit in the runbook with the model, precision, hardware, and the observed peak that justified it.
• Decision lens: predictable memory versus refusing some requests. Example setup: a single-host service with an explicit --cache_size, a gateway that rejects prompts over a set length with a clear message, and an alert when process memory reaches a chosen fraction of the host.

Minimal working example

Prereqs

• A Model Server container named ovms serving a language model, started with an explicit --cache_size.
• Docker and Python 3 with requests.

memory_watch.sh (sample the container's memory while a load test runs)

while true; do
  docker stats --no-stream --format "{{.Name}} {{.MemUsage}}" ovms
  sleep 5
done

load_profile.py

import requests

URL = "http://localhost:8000/v3/chat/completions"
MODEL = "OpenVINO/Qwen3-8B-int4-ov"  # use the model name you deployed

def send(words):
    prompt = "word " * words
    r = requests.post(URL, json={"model": MODEL, "messages": [{"role": "user", "content": prompt}], "max_tokens": 16})
    return r.status_code, r.elapsed.total_seconds()

for words in [200, 1000, 4000, 8000]:
    print(words, send(words))

Increase the prompt sizes until the server starts refusing or slowing down, and record what it returns.

Adjust file names, model names, and paths to your own setup.

Test: Start memory_watch.sh in one terminal and run load_profile.py in another with increasing prompt sizes. Note the memory trend, the largest prompt that completes, and exactly what the server returns for a prompt that is too large. Change --cache_size, restart, and repeat.

Record: model, precision, Model Server version, and host memory; the --cache_size values tested; the memory trend; the largest prompt that completed for each cache size; the server's response to an oversize prompt; latency as prompt size grows; your chosen limit and the evidence for it.

Checklist

• Set --cache_size explicitly.
• Observe memory over a realistic load and find the largest prompt that fits.
• Test the overflow path on purpose.
• Limit prompt size and concurrency at the gateway.
• Document the limit with hardware, model, and precision, and repeat after any change.
• Avoid: sizing from a single short test.
• Avoid: having no defined behavior for oversize requests.

Where this doesn't apply

Short prompts with low concurrency on a host with ample memory, and services behind a managed platform that sets cache limits for you.

Sources

• Agentic AI with OpenVINO Model Server (example commands)
• Model Server continuous batching demo
• Model Server performance tuning guide
• OpenVINO Model Server parameters

---

Codespaces is fast to start and well integrated with GitHub, but it is metered. Compute is billed by the hour, and storage is billed by the gigabyte-month. Stopped codespaces can keep accruing storage until they are deleted. Personal accounts and organizations have different included quotas, so check the billing page for your account type.

Options and trade-offs

• Devcontainers: make environments reproducible and speed up onboarding. They need ongoing maintenance of the image and its setup scripts.
• Prebuilds: speed up opening large repositories. They consume Actions minutes, so they suit heavy projects.
• Machine size policy: limit the cores and memory contributors can pick. Smaller machines cost less but run slower.
• Retention and idle timeouts: reduce storage and runaway compute. They add some admin overhead.
• Spending limits: cap organization spend. Set them before rollout, not after the first surprise bill.

My take

Ship one default devcontainer sized for typical work, enable prebuilds only on the main branch, and set a short idle timeout. Teach contributors to delete codespaces they no longer need rather than only stopping them.

Decision lens: new-contributor friction and maintenance cost.

Example setup: a service repository with a 4-core default devcontainer, prebuilds on the main branch, a 30-minute idle timeout, automatic deletion after 7 days of inactivity, and a monthly spending cap for the organization.

Template repository

.devcontainer/devcontainer.json: Base image, post-create commands, default machine size, and recommended extensions.

.github/workflows/prebuild.yml: Prebuild configuration limited to main and release branches. Check the Codespaces prebuild docs for the current configuration syntax.

README.md ("Use Codespaces" section): Machine choices, the difference between stopping and deleting, budget notes, and a link to contributor docs.

Checklist

1. Define a devcontainer with a sensible default machine size.
2. Enable prebuilds for large repositories and watch Actions minutes.
3. Set idle timeout, retention, and spending limits at the organization level.
4. Document the differences between personal and organization billing.
5. Tell contributors to delete unused codespaces.

Common mistakes

• Assuming a stopped codespace costs nothing. Storage keeps accruing until deletion.
• Leaving machine sizes unlimited, so contributors default to the largest option.
• Enabling prebuilds without tracking the Actions minutes they consume.

Where this doesn't apply

• Teams that need local-only development or hardware that Codespaces does not support.
• Toolchains that need macOS, iOS, or Windows-only environments.

How to get involved

• Good first issue: add a prebuild for a documentation repository.
• Discussion: share your default machine size and budget caps.

Sources (verify current rates)

• Codespaces billing: https://docs.github.com/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces. Compute and storage billing, included quotas, and organization metering.
• GitHub plans: https://docs.github.com/en/get-started/learning-about-github/githubs-plans. Personal and organization plan differences.

---

A container fails and all you get is an exit code and perhaps a log line. The tempting response is to add a shell, curl, a debugger and a package manager to the image so that next time you can look around. That is the wrong trade. Production images should be small and boring, and your tools should arrive only when needed and leave no trace.

Options and trade-offs

• Bake debugging tools into the image: convenient, but it enlarges the attack surface and the image.
• Add an admin endpoint: useful for version and uptime, but it is one more thing to secure.
• Use the platform: logs, exit codes, signals, and a separate short-lived container that shares the target's network.

My recommendation

Start with the exit code and the last logs. Use signals the JVM already understands, such as a thread dump on SIGQUIT, before reaching for tools. Make crashes leave evidence, and attach any tooling from outside.

Worked example

The first look should cost nothing. Exit code 137 usually means the process was killed, often for memory, and 143 means it was asked to stop:

docker ps -a --filter name=api
docker inspect --format '{{.State.ExitCode}} oom={{.State.OOMKilled}}' api
docker logs --since 15m api

The JVM prints a full thread dump to standard output when it receives SIGQUIT, so you need no tools inside the container:

docker kill --signal=SIGQUIT api
docker logs --tail 200 api

Make a crash leave something behind. These flags write a heap dump on an out-of-memory error to a mounted volume:

services:
  api:
    image: demo/api:1.4.2
    environment:
      JAVA_TOOL_OPTIONS: >-
        -XX:+HeapDumpOnOutOfMemoryError
        -XX:HeapDumpPath=/dumps
        -XX:+ExitOnOutOfMemoryError
    volumes:
      - ./dumps:/dumps

And have the application log one structured line when it dies, so the exit code has context:

public class Main {
    public static void main(String[] args) {
        Thread.setDefaultUncaughtExceptionHandler((thread, error) -> {
            System.err.printf("{\"level\":\"FATAL\",\"thread\":\"%s\",\"error\":\"%s\"}%n",
                    thread.getName(), error.getClass().getName());
            error.printStackTrace();
        });
        Runtime.getRuntime().addShutdownHook(new Thread(() ->
                System.err.println("{\"level\":\"INFO\",\"msg\":\"shutdown\"}")));
        // start the application
    }
}

When you need to poke around the network or a volume, use a throwaway container that shares the target's network namespace:

docker run --rm -it --network=container:api -v api-data:/data:ro alpine sh
# inside: wget -qO- http://localhost:8080/health

Checklist

1. Read the exit code, OOMKilled flag and recent logs before changing anything.
2. Use SIGQUIT for a thread dump instead of installing tools.
3. Configure heap dumps on out-of-memory errors, with a size cap and a clean-up plan.
4. Log one structured fatal line with no secrets.
5. Attach tools from a separate short-lived container and remove it afterwards.

Where this doesn't apply

If the failure is inside a native library or the container never starts, thread dumps will not help. You will need the platform's events and the host's logs.

One honest limitation

An outside container can see the network and volumes but not the process's memory. Heap dumps are sensitive too, since they can contain secrets, so store them carefully and delete them quickly.

Sources

• Docker Docs: docker container logs
• Docker Docs: docker container kill (signals)
• Oracle Java Documentation: Troubleshooting guide (thread dumps and heap dumps)

A Java application needs to talk to a database, and the connection should be encrypted, should check it is talking to the right server, and should not carry a password in the source. Those three requirements are simple to state and easy to get partly wrong.

I wanted one baseline that I can adjust upward to mutual TLS when a policy asks for it, without relying on settings that change from driver version to driver version.

Options and trade-offs

• Plain TCP with a username and password: simplest, with no transport protection and no server identity check.
• Server-authenticated TLS: protects traffic and verifies the server, with fewer files to handle. Many services allow it.
• Mutual TLS with a wallet: the client proves itself too. You need the wallet and a safe way to deliver it.

My recommendation

Default to TLS, and use mTLS with a wallet when policy requires client authentication. For Autonomous Database, use the TNS alias from the wallet's tnsnames.ora. Keep the password out of code by injecting it at start-up, and keep the pool conservative.

Worked example

With a wallet, point the driver at the directory that holds it and use the alias as the connection target. The ojdbc.properties file in the wallet supplies the wallet location.

// TNS_ADMIN is set outside the code (environment or -Doracle.net.tns_admin)
PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setURL("jdbc:oracle:thin:@dbname_high");   // alias from tnsnames.ora
pds.setUser(System.getenv("DB_USER"));
pds.setPassword(System.getenv("DB_PASSWORD"));  // injected, not hard-coded
pds.setMinPoolSize(2);
pds.setMaxPoolSize(10);
pds.setValidateConnectionOnBorrow(true);

Then prove that the connection really is encrypted, instead of assuming it:

try (var c = pds.getConnection();
     var s = c.createStatement();
     var rs = s.executeQuery(
        "SELECT sys_context('USERENV','NETWORK_PROTOCOL') FROM dual")) {
    rs.next();
    System.out.println(rs.getString(1));   // expect: tcps
}

The aliases in a downloaded wallet normally include a server certificate name check. Leave it on. If you use an auto-login wallet file with an older driver, you may also need the Oracle PKI jars on the classpath, so check the driver documentation for the version you run.

Checklist

1. Fetch the wallet securely, set TNS_ADMIN, and keep the files out of source control.
2. Use UCP with modest minimum and maximum sizes, and validate on borrow.
3. Choose TLS by default and mTLS only when required.
4. Inject database credentials from a secret store at start-up.
5. Keep the server certificate name check enabled.
6. Log connection failures and test a failover before go-live.

Where this doesn't apply

Old drivers that lack the TLS features you need should be upgraded first. Databases from other vendors use different drivers and different security models.

One honest limitation

Wallets add operational work, such as delivery, rotation and cleanup. Treat the wallet like any other secret and automate its life cycle before the first rotation, not after.

Sources

• Oracle JDBC Developer's Guide: Client-Side Security Features
• Oracle Universal Connection Pool Developer's Guide: Setting Connection Pool Properties
• Oracle Autonomous Database Documentation: Connect with JDBC Thin and a Wallet

• Formatting. Building a message string you never emit is wasted work.
• Allocation. Varargs arrays, boxed values, and message objects add garbage.
• I/O and locking. A synchronous appender can block request threads on disk or on a shared lock.
• Volume. Ten log lines per request at high traffic is a data pipeline you have to pay for.
• Cardinality. Metric and span attributes with unbounded values (user IDs, request IDs as labels) explode storage and backend cost.

Logging guardrails

// Bad: concatenation always runs
log.debug("Fetched profile " + id + " in " + elapsedMs + "ms");

// Good: SLF4J formats only if DEBUG is enabled
log.debug("Fetched profile {} in {}ms", id, elapsedMs);

if (log.isDebugEnabled()) {
    log.debug("State: {}", expensiveDump(state));
}

String snapshot;
synchronized (lock) {
    snapshot = state.summary();      // cheap copy
}
log.info("Rebalanced: {}", snapshot);  // logging outside the lock

Tracing guardrails

import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.samplers.Sampler;

SdkTracerProvider provider = SdkTracerProvider.builder()
        .setSampler(Sampler.parentBased(Sampler.traceIdRatioBased(0.05)))   // ~5% of new traces
        .build();

• Keep span attributes low-cardinality. Put high-cardinality identifiers in logs, not metric labels.
• Avoid creating spans in tight loops. Instrument the request and the major dependencies.
• Cap attribute value sizes.

Measure the overhead

import org.openjdk.jmh.annotations.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(java.util.concurrent.TimeUnit.NANOSECONDS)
public class LoggingBenchmark {

    private static final Logger log = LoggerFactory.getLogger(LoggingBenchmark.class);   // level = INFO
    private final String id = "user-12345";
    private final long elapsed = 42;

    @Benchmark
    public void concatenation() {
        log.debug("Fetched profile " + id + " in " + elapsed + "ms");
    }

    @Benchmark
    public void parameterized() {
        log.debug("Fetched profile {} in {}ms", id, elapsed);
    }
}

Failure modes

• High-cardinality labels that overload the metrics backend.
• Log amplification, where one failure triggers retries that each log at ERROR.
• Synchronous appenders blocking request threads.
• Tracing everything in staging, then discovering production cannot afford it.

A starter checklist

1. Parameterized logging everywhere, with guards for expensive arguments.
2. No logging inside locks or tight loops.
3. Async appender with a bounded queue.
4. Parent-based ratio sampling, with the ratio justified by numbers.
5. Attribute and label cardinality reviewed.
6. A benchmark or load test whenever you add instrumentation to a hot path.

Takeaways

• Observability has a runtime cost that shows up in the tail.
• Sample traces, bound log queues, and keep cardinality low.
• Benchmark instrumentation like any other hot-path code.

aceu-edge-infer is a modest inference demo, but I treat its packaging path as if another engineer will need to rebuild, inspect, and recover it. Its outputs remain illustrative, not evaluative of real people. This checklist is about making that kind of service repeatable.

Make model conversion reproducible

Keep conversion in a script or build step rather than a shell history. Record the source model hash, tool and runtime versions, input shapes, conversion options, and warnings. Use separate, reviewable configuration for FP32 and INT8 artifacts, and retain the conversion logs with the release build.

from pathlib import Path
import openvino as ov

source = Path("models/source.onnx")
output = Path("build/models/model_fp32.xml")
output.parent.mkdir(parents=True, exist_ok=True)

model = ov.convert_model(source)
ov.save_model(model, output)

This example shows the conversion step only. Production automation should also validate model inputs, run parity checks, record the tool version, and fail when expected artifacts are missing.

Version and identify the artifacts

Give model artifacts versions tied to the code and conversion configuration that produced them. Store IR XML and weights together, record hashes, and keep them immutable after release. If calibration or quantization changes, publish a new model version rather than overwriting a file called latest.

A release manifest can record the information operators need without embedding the model itself:

{
  "model_version": "<release-version>",
  "source_format": "ONNX",
  "runtime": "OpenVINO",
  "precision_artifacts": ["FP32", "INT8"],
  "device_policy": "CPU",
  "source_sha256": "<source-model-hash>",
  "conversion_config_sha256": "<config-hash>"
}

Replace the placeholders during the build. These are identifiers to calculate and record—not measured values to guess.

Disclose the execution path

Include the precision and device used in logs or response metadata. Where it makes sense for the product, provide a controlled option to route sensitive work through FP32. That small bit of transparency can make rollout analysis and incident response much faster. Keep the API contract stable and avoid exposing internal details that callers do not need.

Test fallbacks before a release

Simulate an unavailable optimized runtime and verify the intended behavior. A valid fallback should either return correct outputs on a supported slower path or fail clearly with a non-ready/unavailable state. It should not manufacture placeholder results or quietly claim the preferred precision.

If custom operators exist, test the fallback with representative inputs and confirm that the selected path is visible in logs. Document what operators or hardware are required for each execution mode.

Package the runtime deliberately

Use a multi-stage image where it keeps build tools out of the runtime. Run as a non-root user, keep the base image and dependencies maintained, copy only the application and model artifacts required at runtime, and provide a bounded health check.

# Illustrative Python service image; adapt paths and dependencies to your app.
FROM python:3.12-slim AS builder
WORKDIR /build
RUN python -m venv /venv
ENV PATH="/venv/bin:$PATH"
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.12-slim AS runtime
RUN useradd --uid 10001 --create-home appuser
WORKDIR /app
COPY --from=builder /venv /venv
COPY --chown=10001:10001 app/ ./app/
COPY --chown=10001:10001 models/ ./models/
ENV PATH="/venv/bin:$PATH" \
    PYTHONDONTWRITEBYTECODE=1
USER 10001
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD python -c 'import urllib.request; urllib.request.urlopen("http://localhost:8000/health", timeout=2)' || exit 1
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Confirm that the chosen base image contains everything the service actually needs. The Docker health check should stay lightweight; configure a separate platform readiness check if model loading or a sample inference must gate incoming traffic.

Build, validate, and scan in CI

Use the same conversion scripts in local development and CI. Check that expected files exist, compare model outputs against a golden set, build the service image, and scan it before release. Make missing artifacts, failed parity checks, and scan failures visible release blockers. Keep exceptions owned and time-bounded.

Keep operational notes close to the code

Give the README short, findable sections for conversion, artifact versions, precision modes, fallback behavior, and hardware assumptions. Add an incident note describing what to inspect when latency or output parity changes, and point to the known-good artifact for rollback.

Trade-offs and failure modes

• Process rigor vs. prototype speed: scripts and manifests take effort once and save time on the next rebuild.
• More API metadata vs. smaller contract: publish only what helps callers; keep additional operational detail in logs.
• Strict readiness vs. faster first traffic: waiting for model load and checks reduces false-green deploys.

“Mystery IR” files, silent FP32 fallbacks, and changed base images without review all make a service harder to reproduce. Version the inputs and outputs, disclose the selected path, and exercise rollback before you need it.

Try it before your next release

1. Script FP32 and INT8 conversion and retain logs with immutable model artifacts.
2. Record source and configuration hashes in a release manifest.
3. Add actual precision/device metadata and a tested force-FP32 option if the product needs it.
4. Test missing-runtime and unsupported-operator behavior before launch.
5. Build, validate, and scan the same container image in CI that you intend to deploy.

Closing thought

Packaging inference for production is about repeatability, transparency, and safe recovery. When another engineer can recreate the model, identify the execution path, and trigger a tested fallback, the service is ready for more than a successful demo.

The failure mode

You want contributions, so you design a rich, flexible plugin API. The first contributor struggles to understand how the parts fit together. Another contributor introduces side effects that break core behavior. You spend time triaging issues rather than merging useful additions. The fix is to narrow the contract and improve examples.

What I learned

Keep the interface tiny

Use a single method with a small, typed result. Avoid optional hooks and deep callback trees. If you need flexibility, let plugins implement it internally. Keep core expectations documented and enforced with tests.

Illustrative plugin contract

from dataclasses import dataclass
from typing import Literal, Protocol

@dataclass(frozen=True)
class EvidenceResult:
    status: Literal["present", "missing", "unverifiable"]
    note: str = ""

class EvidenceSource(Protocol):
    name: str
    def evaluate(self, profile: dict[str, object]) -> EvidenceResult: ...

Documentation-by-example beats long narratives

Write a short “Adding a plugin” doc that starts with a complete code example and a test. Follow with two variations that show common patterns (e.g., reading a field, combining small signals). Keep the words sparse and the code complete.

Sandboxing contributed code — practical guardrails

Run plugins within constraints:

Timeout per plugin evaluation.

Limit memory-intensive operations or external calls in the demo environment.

Validate types and restrict dangerous imports in plugin modules if possible.

A thread timeout is not a security sandbox: a host process cannot safely terminate arbitrary plugin code. For untrusted extensions, use a separate process or container with resource limits and restricted credentials, or load only trusted plugins.

Versioning and compatibility checks (keep them light)

Require plugins to declare supported interface versions and run contract tests. Keep the declaration simple, and make the CI failure message actionable (e.g., “Plugin supports 1.1.0; core is 1.2.0 — update SUPPORTED_VERSIONS”).

Golden paths and anti-patterns

Show five “golden path” examples contributors should follow:

Pure function: read inputs, return EvidenceResult, no side effects.

Lightweight computation: avoid heavy loops or large allocations.

Deterministic outputs: no random behavior or time-dependent outputs.

Typed result: always return one of the three states with a note.

No external calls: for demo simplicity, avoid network calls.

If you allow external calls, define strict limits and document them clearly.

Error messages designed for contributors

When a plugin fails a contract test, the message should name the plugin, the failing expectation, and the fix. Prefer helpful failures over terse stack traces. Provide a “common issues” page with targeted guidance.

“Hello world” PR built into the docs

Include a numbered guide that results in a merged PR:

Create plugin file.

Add minimal test.

Run pytest -q.

Open PR with the template.

Expected CI jobs and how to interpret them.

Merge expectations.

Trade-offs and failure modes

Trade-offs

Small interface vs expressiveness: Narrow contracts limit advanced features; they also reduce fragility and cognitive load. For small projects, this trade-off is worth it.

Sandboxing vs performance: Timeouts and thread isolation add overhead. Keep limits modest and configurable.

Documentation-by-example vs thorough reference: Examples speed onboarding; deep reference can come later if demand arises.

Failure modes

“Stealth side effects”: Plugins write to global state or files unexpectedly. Solution: sandbox and contract tests; make side-effect-free behavior a rule.

“Doc drift”: Examples fall behind reality. Solution: run example snippets in CI and fail when they don’t compile.

“Silent timeouts”: Sandboxing hides errors without surfacing enough context. Solution: attach structured error notes to EvidenceResult.

Try it yourself

Shrink your plugin interface to a single method that returns a small typed object.

Add a sandbox wrapper with timeouts and typed error handling. Run plugins through it in CI contract tests.

Write a “hello world plugin” guide with a copy-paste example and a minimal test, then merge a sample PR to validate the path.

The failure mode

You set versions in pom.xml but leave transitive ranges unconstrained. Locally, a cached artifact hides a minor release; in CI, it resolves a new version. Behavior changes. The fix is not “rerun until green.” It’s making builds deterministic and verifying them under clean conditions.

What I learned

Pin dependencies and plugins explicitly

Use dependencyManagement to pin versions of critical dependencies and plugins. Avoid wildcard ranges. If you depend on a BOM (bill of materials), understand what it pins and add explicit versions for anything critical that the BOM leaves flexible.

Example

<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.10.2</version>
</dependency>
<!-- pin more -->
</dependencies>
</dependencyManagement>

Use a lock mechanism to freeze transitive versions

A lock file captures resolved versions so CI doesn’t “discover” new transitive artifacts. Tools exist for Maven and Gradle that generate and enforce locks. The specifics depend on your chosen plugin; the principle is the same: generate once, commit, and enforce.

Generate lock file locally (or in CI).

Commit it.

Configure the build to fail if resolutions differ from the lock.

Clean-room rebuild in CI

Build in an isolated environment with a cleared local repository. Verify that resolving dependencies succeeds strictly against your lock. Optionally run the build twice—once with the lock (expected success), once with lock enforcement disabled to detect version drift. The second build should fail if drift occurred.

CI steps (illustrative)

- name: Clean local repo
run: rm -rf ~/.m2/repository
- name: Verify dependency locks
run: mvn -q -B clean verify -P enforce-lock
- name: “Clean room” rebuild
run: docker run --rm -v "$PWD:/w" -w /w eclipse-temurin:21-jdk bash -lc "mvn -q -B clean verify -P enforce-lock"

Reproducible artifacts (timestamps and ordering) Ensure JARs are reproducible: strip timestamps, stabilize entry ordering, and avoid embedding environment-specific paths. Many build tools support “reproducible build” flags. Enable them and verify artifact hashes are stable across clean rebuilds.

Digest references for external assets If your build fetches external tools or base images (for integration tests, for example), reference them by digest. Tags are mutable; digests fix identity. The trade-off is manual refresh when security updates appear—worth it for determinism.

Automate updates safely

Use a dependency-update bot that proposes version bumps. Gate those PRs with tests and scanning. Accepting updates becomes explicit, and drift is no longer silent.

Document the “how to fix drift” playbook

When lock enforcement fails, the fix should be documented: regenerate lock with your plugin, review changes, run tests, and commit. Avoid ad-hoc local resolutions—keep the workflow consistent.

Trade-offs and failure modes

Trade-offs

Pins vs agility: Strict pins and locks slow updates but eliminate surprises. Use bots to keep pace safely.

Clean rooms vs speed: Clean rebuilds take longer. For CI, prioritize correctness over a few minutes of build time.

Digests vs tags: Digests provide determinism; tags ease maintenance. For production-critical assets, default to digests.

Failure modes

“Partial pins”: You pin direct deps but miss transitive ones. Locks solve this; without locks, drift leaks in.

“Plugin drift”: Build plugins advance silently. Pin plugin versions in pluginManagement.

“Non-reproducible JARs”: Differences in timestamps or ordering break hash equality. Turn on reproducible flags and verify.

Try it yourself

Add dependency and plugin pins in dependencyManagement/pluginManagement.

Integrate a lock mechanism; commit the lock file; add an enforcement profile.

Run a clean-room rebuild in CI and compare the artifacts. Confirm artifact hashes match between runs.

The failure mode

A contributor tests locally with a .env file containing a signing key. A sloppy Dockerfile copies everything under the project directory, including .env. CI builds the image; the key ends up in a layer. The image is pushed to a registry. Even if you rotate later, the leak is permanent in history.

What I learned

Secrets never live in source or image layers

Keep signing keys out of your repository and artifact layers. Use a managed secret store pattern to provide keys at runtime via secure mounts or platform-managed environment variables that never touch the image.

.dockerignore is a security control

Treat .dockerignore as part of your security posture. Explicitly ignore .env, credential files, test fixtures containing secrets, and local caches. Keep the build context as small as possible.

Runtime retrieval, not build-time embedding

Load keys at runtime from the secret store client library or a platform-provided mount. Fail readiness until the key loads. If the secret is missing, return clear errors and refuse to sign. Do not “fall back” to dummy keys in production—reserve test keys for explicit non-prod builds.

Rotation: short lifetimes and multiple active keys

Signing keys should rotate regularly. The verifier should accept signatures from a small set of current keys, and the issuer should roll keys with overlap: publish new key, start signing with it, keep the old key valid for verification until the overlap window expires.

Audit and scanning: prevent accidental bake-ins

Add CI checks that scan images for secret patterns. Simple rules catch common mistakes (e.g., files named .env, PEM key headers). Block the build if found. Complement this with code scanning that flags dangerous COPY patterns.

Separate test and prod key handling

For demos, use explicitly marked test keys with sandboxed scopes. Never reuse test keys in environments accessible outside controlled demos. Label logs and status endpoints so it’s obvious which environment your verifier belongs to.

Minimize key surface: hardware-backed or managed services

Where possible, use hardware-backed key stores or provider-managed keys with restricted export. Keep application access tightly scoped to “sign” and “read public key.” The app should never be able to exfiltrate private key material.

Illustrative runtime secret loading

def load_signing_key(secret_store, key_id: str) -> bytes:
    key = secret_store.read(key_id)
    if not key:
        raise RuntimeError("signing key unavailable")
    return key

# Load keys from a managed store at runtime. Never copy key material
# into source, build arguments, container layers, or sample configuration.
# Keep readiness false until the signing dependency is ready.

Example .dockerignore

.env
*.pem
__pycache__/
*.pyc
.git/
node_modules/
tests/fixtures/secrets/

Example Dockerfile pattern

FROM python:3.12-slim AS runtime
WORKDIR /app
COPY pyproject.toml poetry.lock ./
RUN pip install --no-cache-dir poetry && poetry install --no-root
COPY --chown=10001:10001 app ./app
USER 10001
EXPOSE 8000

Trade-offs and failure modes

Trade-offs

Runtime dependency vs build independence: Secret store clients add runtime requirements. The gain is eliminating baked-in risk.

Rotation complexity vs safety: Overlapping keys complicate issuers and verifiers. The gain is smooth migration.

Strict readiness vs availability: Refusing to start without keys prevents undefined behavior. Availability dips are better than silent mis-signing.

Failure modes

“COPY .”: Blindly copying the entire source tree into the image. Fix with .dockerignore and explicit COPY.

“Fallback key”: Using a dummy key in production when secret load fails. Fix with hard failures and improved observability.

“Exportable private keys”: Allowing the app to export private keys. Fix with hardware-backed or provider-managed keys.

Try it yourself

Add .dockerignore entries for secrets and verify your build context (run docker build --no-cache and inspect logs).

Use strict readiness that fails until keys load from a managed store. Test rotation: accept signatures from two keys during an overlap window.

Add CI scans that fail on secret-like files in images. Track and eliminate false positives carefully.

Why is staging running something different from production? If the answer involves the word latest, you have found the bug. A tag is a movable pointer, so two environments can point at different content under the same name, and nobody can say which one is right. Release hygiene comes down to one idea: the artifact that passed your tests is exactly the artifact that runs later.

Options and trade-offs

• Mutable tags such as latest or a branch name: convenient, and impossible to trace.
• Commit-based tags: readable and traceable, but a tag can still be moved.
• Digests: the content address of the image, which cannot change. Deploying by digest guarantees what runs.

My recommendation

Tag every build with the commit it came from, record the digest, deploy by digest, and promote by copying the existing image rather than rebuilding. Keep a small catalog of what was released, so a rollback is just promoting an older entry.

Worked example

A release script that builds a multi-platform image, captures its digest, records it, and promotes the same digest to staging:

#!/usr/bin/env bash
set -euo pipefail

REGISTRY=registry.example.com/team/app
SHA=$(git rev-parse --short=12 HEAD)

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t "$REGISTRY:$SHA" \
  --metadata-file meta.json \
  --push .

DIGEST=$(jq -r '."containerimage.digest"' meta.json)
echo "$SHA $DIGEST $(date -u +%FT%TZ)" >> release-catalog.txt

# Promote: same content, new name. No rebuild.
docker buildx imagetools create --tag "$REGISTRY:staging" "$REGISTRY@$DIGEST"

# Deploy by digest, not tag:
echo "image: $REGISTRY@$DIGEST"

To make a running service tell you what it is, stamp the commit into the jar. In Gradle:

tasks.jar {
    manifest {
        attributes(
            "Implementation-Version" to providers.environmentVariable("GIT_SHA").orElse("dev")
        )
    }
}

public class Version {
    public static String current() {
        String v = Version.class.getPackage().getImplementationVersion();
        return v != null ? v : "dev";
    }
}

Expose Version.current() on a diagnostics route, and the answer to which build is this becomes one request.

Checklist

1. Tag every image with the short commit hash, and never reuse a tag.
2. Record commit, digest and time in a catalog.
3. Deploy by digest.
4. Promote by copying the image, not by rebuilding it.
5. Check that the promoted manifest still contains every platform you need.
6. Protect release images from registry clean-up so you can still debug and roll back.

Where this doesn't apply

For throwaway local images, a plain tag is fine. The discipline pays off from the first shared environment onward.

One honest limitation

Digest pinning needs tooling support, a catalog, and registry permissions to copy images. It is more work than a tag, and teams that skip the catalog lose most of the benefit.

Sources

• Docker Docs: Image digests
• Docker Docs: buildx imagetools create
• Docker Docs: Build metadata file

Auditing everything produces a trail nobody reads. Auditing nothing leaves you guessing after an incident. Traditional auditing was also split across several places, which made it hard to search.

Unified Auditing puts events in one trail. My question was which policies to switch on first, and how to keep that trail from filling a tablespace.

Options and trade-offs

• Predefined policies such as ORA_SECURECONFIG and ORA_LOGIN_LOGOUT: quick wins, and worth enabling selectively.
• Custom policies with BY, EXCEPT and WHEN: precise, and they need testing so they do not create blind spots.
• Mirroring to syslog: improves integrity checking, and adds infrastructure to run.

My recommendation

Enable a small number of predefined policies as a baseline, then add custom policies for the actions that matter, such as privileged schema changes and key management. Schedule purging, and use the audit management package instead of deleting rows by hand.

Worked example

Start by enabling the baseline policies and confirming they are active:

AUDIT POLICY ORA_SECURECONFIG;
AUDIT POLICY ORA_LOGIN_LOGOUT;
 
SELECT policy_name FROM audit_unified_enabled_policies;

Add custom policies for high-value actions. The EXCEPT clause keeps a batch account from flooding the trail.

CREATE AUDIT POLICY ddl_pol PRIVILEGES CREATE ANY TABLE, DROP ANY TABLE;
AUDIT POLICY ddl_pol EXCEPT app_batch;
 
CREATE AUDIT POLICY keymgmt_pol ACTIONS ADMINISTER KEY MANAGEMENT;
AUDIT POLICY keymgmt_pol;
 
CREATE AUDIT POLICY hr_fail_pol ACTIONS SELECT ON hr.employees;
AUDIT POLICY hr_fail_pol WHENEVER NOT SUCCESSFUL;

Keep the trail healthy by archiving and purging through DBMS_AUDIT_MGMT:

BEGIN
  DBMS_AUDIT_MGMT.SET_LAST_ARCHIVE_TIMESTAMP(
    audit_trail_type  => DBMS_AUDIT_MGMT.AUDIT_TRAIL_UNIFIED,
    last_archive_time => SYSTIMESTAMP - INTERVAL '90' DAY);
  DBMS_AUDIT_MGMT.CLEAN_AUDIT_TRAIL(
    audit_trail_type        => DBMS_AUDIT_MGMT.AUDIT_TRAIL_UNIFIED,
    use_last_arch_timestamp => TRUE);
END;
/

Only run the purge after you have exported what you need to keep. For stronger integrity, mirror selected events to syslog so a privileged database user cannot quietly erase the only copy.

Checklist

1. Turn on the secure configuration and login policies first.
2. Write named policies for privileged and high-value actions.
3. Use EXCEPT and WHEN to cut noise from known service accounts.
4. Mirror critical events to syslog or an external system.
5. Purge only with DBMS_AUDIT_MGMT, after archiving.
6. Document who owns each policy and how long events are kept.

Where this doesn't apply

If a central audit platform already collects database events, coordinate scopes with it so you do not store everything twice.

One honest limitation

Broad policies create a lot of data very quickly. Start narrow, look at what you actually collect for a week, and widen only where the gaps are real.

Sources

• Oracle Database Security Guide: Configuring Audit Policies
• Oracle Database SQL Language Reference: CREATE AUDIT POLICY (Unified Auditing)
• Oracle Database PL/SQL Packages Reference: DBMS_AUDIT_MGMT

I wanted data at rest encrypted without rewriting applications. Transparent Data Encryption (TDE) does that, and the application stays unaware. The real work is operational: where the keystore lives, who knows its password, and what happens when it is closed or lost.

A lost keystore means unreadable data, so the hygiene matters more than the encryption command.

Options and trade-offs

• Tablespace encryption: broad protection and a simple rollout with little application impact.
• Column encryption: targeted protection with more DDL planning, and special care around indexes and salt.
• Software keystore versus external key management such as OKV or OCI Vault: the software keystore is simpler, and external management centralizes keys and operations.

My recommendation

Encrypt tablespaces by default and reserve column encryption for specific fields. Configure the keystore location once, take a backup before every key change, and write the operator runbook before the first production cutover.

Worked example

Configure the keystore location. WALLET_ROOT is a static parameter, so it needs a restart. TDE_CONFIGURATION can then be set dynamically.

ALTER SYSTEM SET WALLET_ROOT = '/u01/app/oracle/admin/ORCL/wallet' SCOPE = SPFILE;
-- restart the instance, then:
ALTER SYSTEM SET TDE_CONFIGURATION = 'KEYSTORE_CONFIGURATION=FILE' SCOPE = BOTH;

Create and open the keystore, back it up, then create the master key and an auto-login keystore so the database can reopen after a restart:

ADMINISTER KEY MANAGEMENT CREATE KEYSTORE IDENTIFIED BY "<password>";
ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN IDENTIFIED BY "<password>";
ADMINISTER KEY MANAGEMENT BACKUP KEYSTORE USING 'before_first_key' IDENTIFIED BY "<password>";
ADMINISTER KEY MANAGEMENT SET KEY IDENTIFIED BY "<password>" WITH BACKUP;
ADMINISTER KEY MANAGEMENT CREATE AUTO_LOGIN KEYSTORE FROM KEYSTORE IDENTIFIED BY "<password>";

Create an encrypted tablespace, or convert an existing one online on Oracle Database 12.2 and later, then verify:

CREATE TABLESPACE secure_ts DATAFILE '<path>' SIZE 100M
  ENCRYPTION USING 'AES256' DEFAULT STORAGE (ENCRYPT);
 
ALTER TABLESPACE users ENCRYPTION ONLINE USING 'AES256' ENCRYPT;
 
SELECT wrl_type, status, wallet_type FROM v$encryption_wallet;
SELECT ts#, encryptedts FROM v$encrypted_tablespaces;

Defaults differ between releases, and multitenant databases add container-level options, so check the TDE documentation for your version before running any of this.

Checklist

1. Set the keystore location once, and write the path down.
2. Use AES256 for tablespace encryption unless policy says otherwise.
3. Back up the keystore before and after every key operation.
4. Store the keystore password and the backups separately from the database host.
5. Check keystore status after every restart.
6. Test export and import, and Data Guard behavior, before production.

Where this doesn't apply

Autonomous Database and many managed services already encrypt data at rest for you, so there is nothing to configure. Other database engines have their own native encryption features.

One honest limitation

The keystore is an operational dependency. If it is closed at the wrong moment, or a key rotation goes badly, the database can stop serving data. Rehearse the runbook on a non-production system first.

Sources

• Oracle Database Advanced Security Guide: Introduction to Transparent Data Encryption
• Oracle Database Advanced Security Guide: Configuring Transparent Data Encryption
• Oracle Database SQL Language Reference: ADMINISTER KEY MANAGEMENT

Identity workloads, however, bring constraints: some callers expect predictable latency, and some flows require warm caches or keys ready in memory. The trick is deciding where scale-to-zero helps and where “always on” wins.

The failure mode

You deploy a credential verifier that scales to zero. The first request after an idle period may be much slower than warm requests. A downstream system times out, retries, and amplifies load just as the platform is warming. Worse, your verifier loads keys and trust lists at startup, so the warming step is non-trivial. You saved on idle billing but created unpredictable tails.

What I learned

Cold start budget: measure and design around it

Measure cold starts: container pull, runtime initialization, key load, trust list load, and framework boot. If your end-to-end verification flow includes out-of-process checks (e.g., privacy-preserving revocation lookups), include those too. Decide whether your SLA tolerates the tails; if not, keep a minimum warm replica.

Warm critical paths with lazy loading and caches

Avoid monolithic startup. Load credential-signing keys via a managed secret store, but defer non-critical data until requested. Cache trust lists in memory and refresh asynchronously. Keep proof verification code hot by executing a tiny “self-check” during startup.

Set min replicas for latency-sensitive endpoints

If your platform supports minimum replicas, keep one verifier instance warm. Scale additional instances on demand. This hybrid approach preserves predictable latency while retaining cost controls.

Health, readiness, and the “thin ready” trap

Identity verifiers shouldn’t mark “ready” until critical dependencies (keys, trust lists) are present and a test proof passes. A “thin ready” (bare process up) can route traffic to a half-initialized instance, increasing failure rates.

Scale rules: event triggers vs queue length

If you use event-driven scale, ensure triggers reflect verifier load. Queue length or request rate are better signals than raw CPU in a verifier, since cryptographic checks are sporadic and short. Use conservative ramp-up to avoid stampedes during traffic bursts.

Idle billing: cost vs safety

Scale-to-zero saves money during periods of inactivity. In identity systems, weigh this against the cost of false timeouts and retries. For certain flows (e.g., login or checkout proofs), unpredictable tails are more expensive than idle replicas.

Pipeline hygiene: small image, quick pull

Use multi-stage builds, slim bases, and .dockerignore to keep images small. The smaller the image, the faster cold start. Avoid compiling dependencies at startup; freeze them in the image. Keep secrets out of layers entirely (next article).

Illustrative configuration

# Generic container-app.yaml structure
name: identity-service
ingress:
  external: true
  targetPort: 8000
template:
  containers:
    - image: your-registry/identity-service:tag
      resources:
        cpu: 0.5
        memory: 0.5Gi
  probes:
    liveness:
      httpGet: { path: /health, port: 8000 }
    readiness:
      httpGet: { path: /ready, port: 8000 }
  scale:
    minReplicas: 1 # Keep one warm replica
    maxReplicas: 10
    rules:
      - name: http-requests
        http: { metadata: { concurrentRequests: "50" } }

Trade-offs and failure modes

Trade-offs

Cost vs predictability: Scale-to-zero cuts idle spend but risks unpredictable tails. Hybrid min replicas often balance both.

Strict readiness vs fast green: Stricter readiness delays routing but avoids half-initialized failures. Bias toward strict in identity contexts.

Lazy load vs startup completeness: Deferring non-critical data speeds boot but adds first-request work. Choose wisely.

Failure modes

“Thin ready”: Traffic routed to instances without loaded keys or trust lists. Fix readiness to validate critical paths.

“Stampede”: Traffic surge triggers too many replicas and cache thrash. Fix scale rules and backpressure.

“Oversized image”: Cold starts dominated by image pulls. Fix Docker hygiene.

Try it yourself

Measure the cold start path and identify high-cost steps. Record cold-start and warm-request latency as separate measurements.

Add a minimum warm replica and track tails. Compare p95 across scale-to-zero vs hybrid.

Implement readiness that runs a tiny proof self-check and confirms secret load before returning ready.

Many teams use OpenVINO through another library, and ONNX Runtime's OpenVINO execution provider is one example. Each wrapper has its own configuration surface, and it is not identical to OpenVINO's own property names and behavior.

That difference causes a quiet class of problems. A setting that works when you call OpenVINO directly may be ignored or reshaped by a wrapper, and the default device may not be the one you expect.

The friction points are knowing the default device, knowing which OpenVINO version the wrapper bundles, and knowing which options your installed version really accepts.

Options and trade-offs

• Default device: the PyPI page for the OpenVINO execution provider package states that, by default, the Intel CPU is used for inference, and that you can change this to an integrated GPU, a discrete GPU, or an integrated NPU through the provider configuration's device type argument.
• Bundled runtime: the same page says the Linux wheels at that time came with pre-built libraries of OpenVINO 2025.4.1, so the OpenVINO inside the wrapper may differ from one you installed separately. Check the version your package carries.
• Provider options: ONNX Runtime GenAI's documentation for the OpenVINO provider shows device_type, cache_dir, and a load_config block with per-device properties, for example INFERENCE_PRECISION_HINT and PERFORMANCE_HINT under a CPU key. The page also advises choosing the device by goal: CPU for latency, GPU for throughput, and NPU for efficiency.
• Per-device properties: placing properties under the device they apply to, as in the load_config example, avoids applying a setting meant for one device to all of them.
• Cache directory: the same documentation shows cache_dir as a provider option to reduce model loading time on later runs. As with direct OpenVINO use, confirm with a restart test that it works.
• Direct OpenVINO use gives you every property OpenVINO exposes and clearer errors, at the cost of more integration code. If a wrapper does not expose something you need, consider using OpenVINO directly for that component.
• Recommendation: treat provider options as deployment configuration. Write them in one place, record the library versions beside them, and re-run a probe whenever you upgrade. Copy option names from the page for the version you installed, not from memory, and prefer explicit device names and per-device properties in production.
• Decision lens: convenience versus control. Example setup: a service that loads models through ONNX Runtime with one configuration file listing the device type, cache directory, and per-device properties, plus a note of the versions on which that file was tested.

Minimal working example

Prereqs

• Python 3 in a clean virtual environment with the onnxruntime-openvino package installed. Do not install it next to the plain onnxruntime package in the same environment.
• An ONNX model you already use.
• A writable cache folder.

provider_probe.py

import onnxruntime as ort

print("ONNX Runtime version:", ort.__version__)
print("Available providers:", ort.get_available_providers())

providers = [("OpenVINOExecutionProvider", {"device_type": "CPU", "cache_dir": "./ov_cache"})]
session = ort.InferenceSession("model.onnx", providers=providers)
print("Session providers:", session.get_providers())

If OpenVINOExecutionProvider is missing from the available providers, your installed package does not include it, and ONNX Runtime will fall back to another provider.

Adjust file names, model names, and paths to your own setup.

Test: Run provider_probe.py and check that OpenVINOExecutionProvider appears in the available providers and in the session's providers. Run it a second time to see whether load time drops with cache_dir. If the machine has a GPU or NPU, change device_type and record what changes.

Record: ONNX Runtime and provider package versions; the OpenVINO version bundled in the package; the providers available and the providers the session reports for each device type; load time with and without cache_dir, first versus second run; behavior when a requested device is absent; the configuration you chose for production.

Checklist

• Record every library version.
• Confirm the provider is actually available.
• Set the device type explicitly and place properties under the device they apply to.
• Test cache_dir with a restart.
• Keep the verified configuration in your repository and re-run the probe after every upgrade.
• Avoid: assuming the wrapper's OpenVINO version matches your standalone install.
• Avoid: copying option names from another version's documentation.

Where this doesn't apply

Applications that call OpenVINO directly, and systems using a different OpenVINO integration than ONNX Runtime.

Sources

• onnxruntime-openvino on PyPI
• ONNX Runtime GenAI: OpenVINO execution provider
• OpenVINO Model Caching Overview

---

• Which classes are allocated the most by bytes (not by count)?
• Which call paths create them?
• Does the allocation rate rise during the slow periods?

Step 2: corroborate with async-profiler

# Allocation profile for 30 seconds, written as an HTML flame graph
asprof -e alloc -d 30 -f alloc.html <pid>

Step 3: understand why boxing and collections allocate

• HashSet<Integer> and HashMap<Integer, ...> box every primitive into an Integer object (small values are cached; larger ones are not) and allocate a node per entry.
• Temporary lists and streams created per request.
• String concatenation and formatting in hot paths.

Step 4: prove an alternative with JMH

import it.unimi.dsi.fastutil.ints.IntOpenHashSet;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;

import java.util.HashSet;
import java.util.Random;
import java.util.Set;
import java.util.concurrent.TimeUnit;

@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(2)
public class SetBenchmark {

    @Param({"1000", "100000"})
    int size;

    int[] values;

    @Setup
    public void setup() {
        Random r = new Random(42);
        values = new int[size];
        for (int i = 0; i < size; i++) values[i] = r.nextInt();
    }

    @Benchmark
    public void boxedHashSet(Blackhole bh) {
        Set<Integer> set = new HashSet<>();
        for (int v : values) set.add(v);
        int hits = 0;
        for (int v : values) if (set.contains(v)) hits++;
        bh.consume(hits);
    }

    @Benchmark
    public void primitiveSet(Blackhole bh) {
        IntOpenHashSet set = new IntOpenHashSet();
        for (int v : values) set.add(v);
        int hits = 0;
        for (int v : values) if (set.contains(v)) hits++;
        bh.consume(hits);
    }
}

java -jar target/benchmarks.jar -prof gc

Step 5: validate in the real service

Failure modes

• Optimizing the wrong layer. A faster set does nothing if the allocation comes from JSON parsing upstream.
• Benchmarking unrealistic data. Use sizes and distributions that match production.
• Trading readability for nothing. Adopt a specialized library only where the profile justifies it.
• Forgetting dependencies. A new library adds maintenance and upgrade cost.

Takeaways

• Profile allocation with two tools before touching code.
• Boxing and per-entry nodes in collections are common sources of garbage.
• Prove a fix in JMH with -prof gc, then confirm with a production-like recording.

Copilot works well for many tasks, but its behavior depends on how it is licensed and configured. Individual plans and organization plans have different governance features. Pooled credits behave differently from per-user allowances. If seat types, credit budgets, and repository rules are not aligned with each other, Copilot can drift from helpful suggestions into unreviewed changes in critical code. Check the current plan page for seat prices and included credit amounts, since these change.

Options and trade-offs

• Individual plans: suited to solo work. They have no organization-level policy controls.
• Business plans: add organization policy management and pooled usage. Suitable for most teams that need governance.
• Enterprise plans: add codebase-aware features and stronger administrative controls. They require Enterprise Cloud, so confirm your account type first.
• Content exclusions: stop Copilot from reading or suggesting changes in designated paths. They reduce exposure but add admin upkeep.
• Audit logs: show who used which features. Useful for high-risk repositories, at the cost of more review work.

My take

Start with Business seats for most contributors and reserve higher tiers for maintainers who need deeper repository context. For directories like infrastructure or authentication code, use content exclusions and treat any agent-generated diff the same as a human-written one. Review it, test it, and request changes when needed.

Decision lens: security risk and maintenance cost.

Example setup: a monorepo with infrastructure and product code. Give most contributors Business seats, exclude infrastructure paths from Copilot, and limit agent edits to non-sensitive directories.

Template repository

README.md (excerpt): Describe which activities are allowed in the repository, such as summaries and code explanation, and which are restricted, such as agent edits in sensitive directories. Link to the current plan and credit documentation.

CONTRIBUTING.md (excerpt): Explain how to request the right seat type, how content exclusions are configured, and that agent-generated diffs go through normal review.

.github/policies/copilot.md: Document allowed agent scopes, default model choices, and the list of paths where agent edits are restricted.

Checklist

1. Map each role to a seat type.
2. Configure content exclusions for sensitive paths.
3. Enable audit logging for high-risk repositories.
4. Set a monthly credit budget and an owner for overages.
5. Write down which agent actions are allowed in each directory.
6. Require review for agent-generated changes.
7. Keep a discussion thread open for policy adjustments.

Common mistakes

• Mixing individual and organization plans, which leads to uneven governance.
• Skipping content exclusions, so agents can read secrets or policy documents.
• Leaving credit use unbounded, which causes unexpected charges.

Where this doesn't apply

• Organizations without access to the plan tiers you need.
• Repositories with legal restrictions on AI-assisted development.

How to get involved

• Good first issue: propose a minimal content-exclusion list for one repository.
• Discussion: share where Copilot helps and where it gets in the way.

Sources (verify current status)

• Copilot plans: https://docs.github.com/en/copilot/get-started/plans. Current plans and credits.
• Copilot billing for organizations and enterprises: https://docs.github.com/en/enterprise-cloud@latest/copilot/concepts/billing/organizations-and-enterprises. Seat and credit administration.

---

Secrets leak in boring ways. A token passed as a build argument ends up in image history. An exception handler prints the whole configuration. An environment variable set for a quick test is still there six months later. The fix is not one tool. It is a set of small defaults that make the safe path the easy path, covering the build, the runtime and the logs.

Options and trade-offs

• ARG and ENV for credentials: simple, but both are stored in image metadata and can be read back from the image.
• BuildKit secret mounts: the secret is available to one build step and is never written to a layer.
• Runtime secrets as environment variables: convenient, but they show up in diagnostics and process listings.
• Runtime secrets as mounted files: easier to restrict, audit and rotate without rebuilding.

My recommendation

Use BuildKit secret mounts for anything a build needs, mount credentials as files at runtime, and make the application hard to leak from by redacting secrets in its own types. Plan rotation as a restart, not a rebuild.

Worked example

A build step that reads a private repository token from a secret mount and exposes it only to that command:

# syntax=docker/dockerfile:1
FROM eclipse-temurin:21-jdk AS build
WORKDIR /src
COPY . .
RUN --mount=type=secret,id=repo_token \
    REPO_TOKEN="$(cat /run/secrets/repo_token)" ./gradlew --no-daemon build

docker build --secret id=repo_token,src=$HOME/.tokens/repo -t demo/app:dev .

# Verify nothing leaked into the layers
docker history --no-trunc demo/app:dev | grep -i token || echo "no token in history"

At runtime, mount the secret as a file and tell the app where it is:

services:
  api:
    image: demo/api:1.4.2
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt

In the application, read from the file when it is configured, and make the configuration type safe to print:

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

public final class Secrets {
    private Secrets() { }

    public static String read(String name) throws IOException {
        String path = System.getenv(name + "_FILE");
        if (path != null) {
            return Files.readString(Path.of(path)).strip();
        }
        String value = System.getenv(name);
        if (value == null) {
            throw new IllegalStateException("Missing secret: " + name);
        }
        return value;
    }
}

record DbConfig(String url, String password) {
    @Override
    public String toString() {
        return "DbConfig[url=" + url + ", password=****]";
    }
}

Checklist

1. Never pass credentials through ARG or ENV in a Dockerfile.
2. Use RUN --mount=type=secret, and check docker history afterwards.
3. Mount runtime secrets as files with read-only access.
4. Redact secrets in your own types so an accidental log line or stack trace cannot print them.
5. Rotate by replacing the file and restarting the container, not by rebuilding the image.
6. Run a secret scan on source and images and treat a hit as a release blocker.

Where this doesn't apply

A file-based secret in Compose is a bind mount for local use. In a real deployment, use your platform's secret store rather than a file in your repository, and keep that directory in .gitignore.

One honest limitation

Anyone who can run a shell in the container can read its mounted secrets. A secret mount removes the secret from the image, not from the running process.

Sources

• Docker Docs: Build secrets
• Docker Docs: Manage secrets in Compose
• OWASP Cheat Sheet: Secrets Management

The failure mode

A verifier expects a “proofOfX” field. The credential arrives without it. The code sets proofOfX=false. A downstream gate denies access. Later, it turns out “proofOfX” wasn’t required for this flow, or the holder didn’t have a way to request it yet. The deny was based on an absence, not a disproven claim.

What I learned

Model states, not booleans

Replace booleans with intent-rich types. For a given proof, consider states like Present, Missing, Expired, Revoked, Unverifiable (failed binding), and Invalid (malformed or bad signature). This simple taxonomy avoids flattening all “not present” into “false.”

Exhaustive decision logic at boundaries

At the verifier boundary, use exhaustive switches over the state. Force downstream callers to handle each branch. If your language supports pattern matching, lean on it to eliminate default fall-through behaviors.

Treat absence as an opportunity to request or fall back

If a required field is missing, decide whether to:

Prompt the holder to supply it (if applicable).

Proceed with reduced assurance and log the state.

Deny the request explicitly as “unverifiable,” not “invalid.” The right action depends on your policy and context; the key is keeping “unverifiable” distinct.

Make “invalid” a narrow, technical category

Invalid should mean “structured wrong,” “bad signature,” or “contradiction.” It shouldn’t include “not provided,” “not bound,” or “not present yet.” Narrow “invalid” ensures you don’t over-penalize users for the system’s constraints.

Propagate state through APIs and UX

Expose the state in API responses: include status: "unverifiable" with a short note. Avoid returning false or 0 as a sentinel for “missing.” In UX, explain that more evidence is needed or that a different path is available.

Beware implicit type coercions

JSON, databases, and ORMs often coerce nulls to zero or false. Audit these boundaries. Ensure your serializers and persistence layers preserve state labels rather than defaulting values.

Test for fairness: counterfactuals

Put fairness into tests. For example: if “proofOfX” is missing, does the system deny or ask for more evidence? Are there alternative flows? Keep tests that assert the difference between “unverifiable” and “invalid.”

Explicit verification outcomes

from dataclasses import dataclass
from typing import Literal

Status = Literal["accepted", "unverifiable", "invalid"]

@dataclass(frozen=True)
class FieldCheck:
    name: str
    status: Status
    note: str | None = None

Trade-offs and failure modes

Trade-offs

More state complexity vs simplicity: Explicit states add a little code. The clarity and fairness gains are worth it.

UX friction vs silent denial: Asking for more evidence can add steps. Silent denials are worse. Balance friction with guidance.

Logging granularity vs privacy: Notes improve debugging; ensure they contain no sensitive data.

Failure modes

“Hidden coercion”: Tooling collapses null to false. Solution: audit boundaries and enforce state labels.

“Catch-all invalid”: Development shortcuts treat everything negative as invalid. Solution: write exhaustive switches and linters.

“Unverifiable loop”: Repeated prompts for evidence without alternative paths. Solution: define fallback policies clearly.

Try it yourself

Replace boolean checks for critical credential fields with stateful outcomes (“accepted,” “unverifiable,” “invalid”).

Expose the state in your verifier’s API. Track how often “unverifiable” occurs and address UX with prompts or fallbacks.

Add tests that assert denials only occur for “invalid,” not “unverifiable.”

Teams still choose these patterns because they support familiar workflows. The important thing is to make the choice explicit, restrict which code can use it, and avoid a “green” result when container tests never ran.

First, know which daemon the job is using

“Docker in CI” does not always mean DinD. Some runners expose a Docker daemon to the job; DinD starts a separate Docker daemon inside a service container. A mounted /var/run/docker.sock is a third pattern: the job’s Docker CLI talks to the host daemon. These setups have different isolation properties. Document the runner architecture before relying on it.

Verify the daemon before running tests

Make readiness an explicit step. If the daemon is missing or unresponsive, stop the job before it reports container-dependent tests as successful:

# Run only on a runner configured to provide Docker access.
set -eu
docker version
docker info

IMAGE="aceu-trust-demo:${GITHUB_SHA:-local}"
docker build --tag "$IMAGE" .
docker run --rm "$IMAGE" pytest -q

For an actual DinD service, the job also needs to wait for that service’s daemon to become available. A fixed sleep is brittle; use a bounded readiness loop and fail with a clear message if the daemon never responds.

Do not silently skip container-dependent tests

If the daemon is a prerequisite, the test should fail when Docker is unavailable. Avoid code that catches every exception and quietly returns a passing or skipped result in a required CI job:

import subprocess


def docker_available() -> bool:
    try:
        subprocess.run(
            ["docker", "info"],
            check=True,
            capture_output=True,
            text=True,
            timeout=10,
        )
        return True
    except (FileNotFoundError, subprocess.CalledProcessError, subprocess.TimeoutExpired):
        return False


def test_containerized_health_check_requires_docker():
    assert docker_available(), "Docker daemon is required for this test"
    # Continue with the containerized test flow.

Tag Docker-dependent tests so developers can distinguish them from pure unit tests, but keep them required in the CI path that claims to validate the container.

Choose the access pattern with eyes open

• Privileged DinD: familiar and flexible, but grants elevated capabilities to the service container. Restrict it to trusted workflows and isolate the runner.
• Host socket mount: avoids starting another daemon, but the mounted socket can let job code control the host Docker daemon. Treat access as highly privileged.
• Daemonless or remote builders: can reduce direct daemon exposure in some environments, but require setup and may change the build workflow.

In every case, pin build images, keep credentials scoped, avoid running untrusted pull-request code on privileged self-hosted runners, and clean up jobs so containers and volumes do not leak into later runs.

Watch for isolation leaks and flaky runners

Look for unexpected host mounts, files written outside the workspace, orphaned containers, and daemon failures. Runner fleets can have transient problems. A documented single retry can filter a brief infrastructure hiccup, but repeated failure should remain visible and investigated—not retried until a broken test turns green.

Trade-offs to discuss with the team

• Speed vs. isolation: DinD is familiar; stricter alternatives can require more setup.
• Fail closed vs. ergonomics: blocking on daemon failure catches false greens but can be frustrating when runner health is poor. Give contributors a clear error and retry path.
• Socket mount vs. privileged mode: neither is a free isolation boundary. Limit trusted code and runner scope for either design.

Try it in your CI

1. Write down whether jobs use host Docker, a mounted socket, or a separate DinD service.
2. Add a daemon readiness check before any container-dependent test.
3. Make those tests fail if Docker is unavailable; do not silently skip them.
4. Evaluate a daemonless or remote builder against your runner threat model.
5. Document when one retry is appropriate and when repeated failure needs investigation.

Closing thought

DinD can be pragmatic, but it should never be invisible. Verify the daemon, make tests fail closed, and choose an isolation pattern that matches what the CI job is trusted to do.

PL/SQL runs with one of two sets of privileges: the owner's (definer's rights) or the caller's (invoker's rights). Choosing the wrong one is how a harmless helper procedure ends up with far more reach than anyone intended.

I wanted a decision path I can apply in seconds and defend in a code review.

Options and trade-offs

• Definer's rights (AUTHID DEFINER, the default): privileges come from the owner. Callers need only EXECUTE, and the risk is an owner with broad privileges.
• Invoker's rights (AUTHID CURRENT_USER): the code works within the caller's own privileges, which suits shared code across schemas or tenants. Callers need access to every object it touches.

My recommendation

Make shared, reusable code invoker's rights, so it can never do more than the person calling it. Use definer's rights for a small number of deliberate entry points that expose one action without handing out table privileges, and restrict who may call them.

Worked example

An invoker's rights procedure resolves unqualified names at run time in the caller's schema, so it reads the caller's own orders table.

CREATE OR REPLACE PROCEDURE app_util.list_my_orders
  AUTHID CURRENT_USER
AS
BEGIN
  FOR r IN (SELECT order_id, status FROM orders) LOOP
    DBMS_OUTPUT.PUT_LINE(r.order_id || ' ' || r.status);
  END LOOP;
END;
/

A definer's rights entry point is the opposite. It performs one privileged action on the owner's data, and ACCESSIBLE BY restricts which program units may call it.

CREATE OR REPLACE PROCEDURE app_admin.close_period (p_period IN NUMBER)
  AUTHID DEFINER
  ACCESSIBLE BY (PACKAGE app_api.billing)
AS
BEGIN
  UPDATE fiscal_periods SET status = 'CLOSED' WHERE period_id = p_period;
END;
/
GRANT EXECUTE ON app_admin.close_period TO billing_role;

A few facts that help in practice. Roles are disabled inside definer's rights code, so the owner needs direct grants. Since Oracle Database 12c, an invoker's rights unit can use the caller's privileges only if its owner has the INHERIT PRIVILEGES privilege on the caller, or INHERIT ANY PRIVILEGES. Anonymous blocks run with the privileges of whoever executes them.

Checklist

1. Decide who should own the effective privileges before writing the code.
2. For invoker's rights, confirm callers have the object privileges the code needs.
3. Keep definer's rights units small, and keep the owner's privileges minimal.
4. Use ACCESSIBLE BY to limit callers of sensitive procedures.
5. Document entry points and who calls them, and audit executions.
6. Review dynamic SQL, because it resolves names at run time.

Where this doesn't apply

Anonymous blocks already behave like invoker's rights, so the choice does not apply to them.

One honest limitation

Invoker's rights with static SQL expects matching objects to exist for the owner at compile time and for the caller at run time. Plan schema design so those differences do not produce subtle errors.

Sources

• Oracle Database PL/SQL Language Reference: Invoker's Rights and Definer's Rights (AUTHID Property)
• Oracle Database Security Guide: Managing Security for Definer's Rights and Invoker's Rights
• Oracle Database PL/SQL Language Reference: ACCESSIBLE BY Clause

aceu-edge-infer is a small CPU-first inference demo with a documented OpenVINO runtime path. Its ACE-U outputs are illustrative—not a validated assessment of people. The design lesson is broader: state the hardware contract and make the service’s behavior when that contract is not met explicit.

Document the hardware contract

Say which device is required and what is optional. If CPU-only execution is supported, describe the expected model format, available runtime, resource needs, and what performance behavior users should plan for. If an accelerator is optional, list supported devices and the path selected when one is missing.

Make device selection explicit

For an OpenVINO model intended for CPU execution, specify the device in the compile call rather than relying on a default selection:

import openvino as ov

core = ov.Core()
model = core.read_model("models/aceu_ir.xml")
compiled_model = core.compile_model(model, "CPU")

Test the exact runtime package and operating system you plan to ship. If startup cannot load the model or compile it for CPU, do not report the service as ready.

Distinguish a fallback from a failure

A fallback is a real, tested alternative execution path. It may be a different model artifact, a portable runtime, or a slower implementation that produces valid outputs. If none exists, the honest response to a missing runtime is a clear startup failure or an unavailable response—not placeholder scores or a “fallback” label that hides the problem.

Expose execution metadata that helps operators understand the path without leaking environment details:

execution_info = {
    "device": "CPU",
    "precision": "INT8",
    "fallback_used": False,
}

In a real service, derive these values from the compiled model and request path; do not hard-code them when the runtime can choose a different path.

Degrade predictably on constrained hardware

Assume memory and CPU shares may be limited. Measure the model’s memory footprint and end-to-end latency on the target class of device. Start with conservative concurrency; more threads do not automatically mean better p95 latency. If the service supports it, adjust batch size, request timeout, or concurrency by deployment profile, and document the effect.

Do not silently lower quality when the machine is under pressure. If a request is rejected, deferred, or routed to a different precision, make the behavior observable and keep the response contract clear.

Separate startup, readiness, and liveness

• Startup: load the model and confirm the selected CPU runtime can compile it.
• Readiness: confirm the service can complete the critical inference path before it receives traffic.
• Liveness: use a cheap check for a process that is stuck, not an expensive inference on every probe.

A stricter readiness check may delay the first “ready” signal. That is preferable to routing requests to a service that has not loaded its model. Keep checks bounded so probes do not become a source of load.

Keep operational assumptions beside the code

Add a “Hardware assumptions” section to the README and link it from deployment instructions. Include supported devices, runtime versions, precision choices, resource settings, fallback behavior, health endpoints, and a short way to verify the selected execution mode. The next engineer should not have to infer these from source code.

Trade-offs and failure modes

• CPU portability vs. latency: a CPU path broadens where the service can run, but may not meet a workload’s latency target. Measure before promising an SLA.
• Optional accelerators vs. maintenance: each extra path adds test and support cost; add only devices the product needs.
• Strict readiness vs. quick deployment: waiting for a successful model load delays traffic, but avoids a false-green service.

Common failures include assuming hardware that is not present, retrying device selection in a tight loop, and leaving hardware requirements undocumented. Provide a clear failure state and a controlled recovery path instead.

Try it yourself

1. Write the CPU and optional accelerator requirements in your README.
2. Expose the actual device and precision selected for each execution path.
3. Simulate a missing optimized runtime and verify that the service fails clearly or uses a genuine tested fallback.
4. Measure memory and latency under representative CPU limits before selecting concurrency defaults.

Closing thought

Designing for CPU-only execution is not a claim that CPU is always enough. It is a way to make assumptions visible, provide a credible fallback where one exists, and keep the service predictable on the hardware it supports.

The failure mode

You enable parallel test execution and see intermittent failures: “record not found” or “unexpected duplicate.” You re-run and it passes. Blaming the test runner is easy. The problem is shared state: two tests reading/writing the same fixture or relying on an implicit order.

What I learned

Default to per-test fixtures

Use per-test initialization to avoid shared mutable state. In JUnit 5, keep your test instance lifecycle at PER_METHOD and recreate fixtures for each test. If the fixture is expensive, consider a template with parameterization that still isolates state.

Example setup

import org.junit.jupiter.api.*;
@TestInstance(TestInstance.Lifecycle.PER_METHOD)
class EvidenceServiceTest {
private InMemoryStore store;
private EvidenceService service;
@BeforeEach
void init() {
store = new InMemoryStore(); // new store per test
service = new EvidenceService(store);
store.seedProfile("alice", syntheticProfileAlice());
}
@Test
void returnsMissingWhenTimeout() {
// ...
}
@Test
void returnsUnverifiableOnExceptions() {
// ...
}
}

Avoid static mutable fields—especially in helpers

Static caches and helpers make tests faster locally but brittle in parallel. If you truly need a cache, scope it per test, or guard it with thread-safe constructs and clear it explicitly. Better: don’t cache at all in tests unless you benchmarked the need.

Make isolation visible—name-bound fixtures

If a test seeds data, bind it to a unique key per test. Use random UUIDs or test names in keys to prevent accidental cross-test reuse. With Testcontainers, create per-test containers when state isolation is critical; share containers only for read-only tests.

Parallel execution deserves explicit design

JUnit 5 lets you run tests in parallel. That doesn’t mean every test should be parallel. Annotate classes that require single-thread execution, but treat that as a last resort. Prefer making tests parallel-safe by design.

JUnit configuration (illustrative):

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.mode.classes.default = concurrent

Deterministic time and waiting

Tests that sleep and “hope the system is ready” fail under parallel runs. Replace sleeps with explicit synchronization: await a condition, poll with a small backoff, or use latches. Keep timeouts small and deterministic.

Embrace “given-when-then” with isolated builders

Build input data per test with dedicated builders. Don’t reuse a mutable “default profile” object across tests; copy or reconstruct it. Records help: immutable inputs prevent accidental mutation.

Diagnose with logging and seeds

When a test fails intermittently, print seeds and keys that identify data. If you see the same key in two tests, you found shared state. If you see non-deterministic ordering in a collection, sort inputs or assert set equivalence rather than list order.

Per-test container with unique data

class StoreTests {
    private PostgreSQLContainer<?> database;

    @BeforeEach
    void setUp() {
        database = new PostgreSQLContainer<>("postgres:alpine")
            .withDatabaseName("test_" + UUID.randomUUID())
            .withUsername("user")
            .withPassword("test-only");
        database.start();
    }

    @AfterEach
    void tearDown() {
        database.stop();
    }
}

Trade-offs and failure modes

Trade-offs

Isolation vs speed: Per-test fixtures are slower than shared ones. Optimize later—start with correct behavior and only share read-only fixtures.

Parallelism vs determinism: Parallel runs surface concurrency bugs but may require more careful test design. The benefits outweigh the cost.

Container lifecycle vs developer ergonomics: Per-test containers are robust but slower. Consider class-level containers for read-only tests with clear resets.

Failure modes

“Hidden globals”

A helper modifies a static map. Under parallel, tests interleave and fail intermittently.

“Order assumptions”: Tests rely on iteration order of sets or maps; parallel runs expose different orders.

“Implicit sleeps”: Tests use fixed sleeps; faster/slower environments cause out-of-order expectations.

Try it yourself

Switch to PER_METHOD lifecycle and remove static mutable fields in tests. Add unique keys per test to seed data. Enable parallel execution and watch for failures that indicate hidden shared state.

Replace sleep-based readiness with explicit condition checks. Add timeouts and backoff policies.

Document which tests are parallel-safe and why; make non-parallel tests the exception and note the reason.

// Works from an IDE or the repo root; breaks in CI or when packaged
String json = Files.readString(Path.of("config/app-manifest.json"));

try (InputStream in = AppManifest.class.getResourceAsStream("/manifests/app-manifest.json")) {
    if (in == null) {
        throw new IllegalStateException("Missing classpath resource: /manifests/app-manifest.json");
    }
    return new String(in.readAllBytes(), StandardCharsets.UTF_8);
}

Fail the build when packaging is wrong

// build.gradle.kts
val verifyManifest by tasks.registering {
    dependsOn(tasks.jar)
    doLast {
        val jarFile = tasks.jar.get().archiveFile.get().asFile
        java.util.zip.ZipFile(jarFile).use { zip ->
            check(zip.getEntry("manifests/app-manifest.json") != null) {
                "manifests/app-manifest.json missing from ${jarFile.name}"
            }
        }
    }
}
tasks.check { dependsOn(verifyManifest) }

Avoid classpath hacks in tests

Pin dependencies

# gradle/libs.versions.toml
[versions]
junit = "5.10.2"

[libraries]
junit-jupiter = { module = "org.junit.jupiter:junit-jupiter", version.ref = "junit" }

dependencyLocking { lockAllConfigurations() }

./gradlew dependencies --write-locks      # create or update lock files

Make the output reproducible

tasks.withType<AbstractArchiveTask>().configureEach {
    isPreserveFileTimestamps = false
    isReproducibleFileOrder = true
}

./gradlew clean jar && sha256sum build/libs/*.jar > first.sum
./gradlew clean jar && sha256sum build/libs/*.jar > second.sum
diff first.sum second.sum && echo "reproducible"

A repeatable CI parity checklist

1. Use the Gradle wrapper, and commit the wrapper files.
2. Pin the JDK with a toolchain.
3. Commit dependency lock files.
4. Run ./gradlew clean build locally from a clean checkout before pushing.
5. Fail the build on missing resources, as shown above.
6. Compare artifact checksums between two builds periodically.

Demo to build

Failure modes

• Resources loaded by file path instead of classpath.
• Floating dependency versions such as 1.+ or latest.
• Tests that pass only because of leftover files in a local build directory (always test after clean).
• Build plugins and tools that differ between developer machines and CI.

Takeaways

• Load files from the classpath, and fail loudly when they are missing.
• Pin dependencies and the JDK so two builds mean the same thing.
• Make archives reproducible and prove it with a checksum comparison.

I wanted the smallest useful baseline: a Java service in a container, on a cheap OCI instance, connecting to an Oracle database over TLS, with no credentials baked into the image. It should be small enough to read in one sitting and safe enough to build on.

Options and trade-offs

• Ampere A1 with Podman: efficient and economical, and you need arm64 images.
• Docker on an x86 shape: the widest image support, at a higher cost as you scale.
• TLS only versus mTLS with a wallet: TLS means less to handle, and mTLS gives stronger assurance.

My recommendation

For a small service I use an Ampere A1 instance with Podman, TLS first, and mTLS when policy requires it. The image holds only the application. Credentials and any wallet are injected when the container starts. UCP stays at conservative sizes.

Worked example

Install the container tooling on Oracle Linux 8 and build on the same architecture you will run on. Building on the A1 instance avoids cross-architecture surprises.

sudo dnf module install -y container-tools:ol8
sudo dnf install -y podman-docker git

A Dockerfile that runs as a non-root user. The Temurin JRE image is published for arm64 as well as x86.

FROM eclipse-temurin:17-jre
RUN useradd --system --no-create-home appuser
WORKDIR /app
COPY target/app.jar /app/app.jar
USER appuser
CMD ["java", "-jar", "/app/app.jar"]

Run it with the password supplied as a Podman secret, and with a read-only filesystem and no extra capabilities:

printf '%s' "$DB_PASSWORD" | podman secret create db_password -
 
podman run -d --name app -p 8080:8080 \
  --read-only --tmpfs /tmp --cap-drop=ALL \
  --env JDBC_URL='<tcps_jdbc_url>' --env DB_USER='<db_user>' \
  --secret db_password,type=env,target=DB_PASSWORD \
  <region-key>.ocir.io/<tenancy-namespace>/myteam/myapp:1.4.2

In the application, build the pool from environment values and keep the sizes modest:

PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setURL(System.getenv("JDBC_URL"));          // tcps:// URL
pds.setUser(System.getenv("DB_USER"));
pds.setPassword(System.getenv("DB_PASSWORD"));
pds.setMinPoolSize(2);
pds.setMaxPoolSize(10);
pds.setValidateConnectionOnBorrow(true);

For mTLS, mount the wallet directory read-only at run time, set TNS_ADMIN to it, and let ojdbc.properties supply the wallet location. Do not copy it into the image. Remember to open the application port in both the instance firewall and the network security group.

Checklist

1. Confirm every dependency and base image supports arm64.
2. Run as a non-root user with a read-only filesystem.
3. Inject database credentials at start-up from a secret store.
4. Start with TLS, and move to mTLS when required.
5. Size the pool for the database, not for the instance.
6. Test a bad certificate and a database restart before cutover.

Where this doesn't apply

Large frameworks with big memory footprints may be a poor fit for a small instance. Pick a shape for the workload instead of forcing it onto the smallest one.

One honest limitation

TLS and wallet problems tend to show up as vague connection errors. I keep a tiny test program that only opens a connection, so I can separate driver and certificate problems from application bugs.

Sources

• OCI Documentation: Deploying Java Applications on Ampere A1
• Podman Documentation: podman secret and podman run
• Oracle JDBC Developer's Guide: Client-Side Security Features

The failure mode

A caller triggers N downstream calls per request (fan-out). With a fixed pool, saturation appears quickly under spikes; with virtual threads, spikes are absorbed but not eliminated. If you translate “we can spawn thousands of threads” into “we don’t need back pressure,” you’ll rediscover failures—just in different places.

What I learned

Think “task-per-thread” for blocking I/O, not “pool-per-service”

Virtual threads make “one virtual thread per blocking task” feasible. You can write linear code for a downstream call and not contort it into callbacks just to avoid blocking. That simplifies failure handling and resource cleanup.

Example fan-out

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    List<Callable<EvidenceResult>> tasks = sources.stream()
        .map(source -> (Callable<EvidenceResult>) source::fetchEvidence)
        .toList();
    List<Future<EvidenceResult>> futures = executor.invokeAll(tasks);
    List<EvidenceResult> results = new ArrayList<>(futures.size());
    for (Future<EvidenceResult> future : futures) {
        results.add(future.get());
    }
    return aggregate(results);
}

This reads like blocking I/O because it is—but with virtual threads, the scheduler multiplexes them efficiently.

Back pressure still matters

If downstream services or data sources choke under spikes, virtual threads won’t magically fix it. The caller can spin up many virtual threads, but the destination still has limits. You need timeouts, rate limits, and circuit breakers. You also need to avoid accidental fan-in (e.g., mass aggregation that blocks on slowest responders indefinitely).

CPU-bound work belongs elsewhere

Virtual threads help with blocking I/O, not CPU-heavy tasks. If a task saturates CPU, the scheduler can’t help—you need to bound parallelism with a platform-appropriate mechanism (e.g., a limited executor or structured concurrency approach) to avoid oversubscription.

Failure semantics are simpler when code is linear

Exception handling in “straight-line” code is clearer than in callback-based designs. You still need to decide whether partial evidence returns a partial result (with Missing/Unverifiable states) or you fail the whole request. Virtual threads don’t make that decision; they just make the code easier to read.

“Thousands of threads” doesn’t mean “no limits”

Virtual threads are cheap, but not free. Sockets, file descriptors, and memory buffers are finite. The OS still enforces resource ceilings. Guard your fan-out with sensible limits: per-request caps, per-destination timeouts, and global safety checks.

Observability shifts upstream

With fixed pools, saturation appears as queue wait time. With virtual threads, saturation shifts to downstream latency and timeouts. Make sure your metrics capture per-destination latency percentiles and error rates. A spike in virtual-thread creation can be fine if destinations are healthy; it’s not fine if percentiles explode.

Test under synthetic spike and tail latency

I haven’t yet measured performance between virtual threads and a fixed pool for the demo. What I do measure is behavior under artificial spikes: do timeouts fire? Does the aggregation return timely partial results with Missing/Unverifiable? Tests simulate slow destinations and verify we don’t hang.

Timeout per downstream call

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    Future<EvidenceResult> future = executor.submit(source::fetch);
    try {
        return future.get(timeout.toMillis(), TimeUnit.MILLISECONDS);
    } catch (TimeoutException timeoutError) {
        future.cancel(true);
        return EvidenceResult.unavailable("timeout");
    }
}

Trade-offs and failure modes

Trade-offs

Simplicity vs explicit limits: Virtual threads reduce boilerplate, but you still must implement back pressure. Silent defaults can mask saturation until it’s too late.

“Everything virtual” vs CPU realities: Virtual threads help with I/O. For CPU-bound work, a bounded executor or workload partitioning is safer.

Observability differences: You lose fixed-pool queue metrics and gain per-destination latency metrics. Update dashboards and alerts accordingly.

Failure modes

“Soft hangs”: Aggregation waits on the slowest responder forever. Add timeouts and partial-return logic.

“Resource leaks”: Thousands of virtual threads keep sockets open longer than intended. Validate resource cleanup paths.

“Oversubscription”: CPU-heavy tasks in virtual threads degrade the whole process. Bound CPU parallelism explicitly.

Try it yourself

Replace a fixed thread pool fan-out with newVirtualThreadPerTaskExecutor() and add per-destination timeouts. Observe behavior under synthetic spikes.

Add aggregation logic that treats timeouts as Missing and downstream errors as Unverifiable. Verify exhaustive handling with pattern matching.

Measure throughput and p95 latency in a controlled environment to inform whether virtual threads help your workload.

A generative model server keeps a key-value (KV) cache for every sequence it is working on. That cache grows with context length and with the number of sequences processed together. If both grow without limits, throughput looks good in a short test and latency becomes unpredictable under real traffic.

OpenVINO Model Server serves language models with continuous batching and paged attention. Its documentation shows the main task-specific options, including whether prompt tokens are cached across requests and how much memory the KV cache may use.

The friction points are choosing a cache size for your hardware, deciding whether prefix caching is worth it for your traffic, and measuring both honestly instead of trusting a short single-user test.

Options and trade-offs

• --enable_prefix_caching: described in the documentation as caching prompt tokens (KV blocks) so repeated prefixes are not evaluated again. The performance tuning guide lists --enable_prefix_caching true as the default. It helps when clients resend the same conversation history and does little when every prompt is unique.
• --cache_size: the memory for the KV cache, in gigabytes. The documentation's examples set it explicitly, for example 20 for an 8-billion-parameter 4-bit model on a 16-core CPU system and 2 for a GPU example. Treat those as illustrations of the option, not recommendations for your hardware.
• --target_device: the continuous batching demo explains that, if you do not set it, the device is chosen from detected hardware with priority discrete GPU, then integrated GPU, then CPU. Set it explicitly when you need a particular device.
• --task text_generation with --source_model and --model_repository_path is the documented way to start serving a language model from a model repository. The pull guide describes converting and quantizing a Hugging Face model, with options such as --weight-format int8.
• Example from the documentation (CPU, Docker): docker run -d --name ovms --cpuset-cpus 0-15 --user $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models openvino/model_server:weekly --rest_port 8000 --model_repository_path /models --source_model OpenVINO/Qwen3-8B-int4-ov --enable_prefix_caching true --cache_size 20 --task text_generation.
• Other batching limits exist in the server's task-specific options. List them with the help output for the version you run, and record any you change.
• Recommendation: start from the documented command for your device, change one option at a time, and measure. Set --cache_size explicitly in production so memory use is a decision you made and wrote down. Keep prefix caching enabled for chat-style traffic and test it with a shared-prefix workload and a unique-prompt workload. Limit concurrency at the edge of your system as well.
• Decision lens: throughput against tail latency and memory risk. Example setup: a single host serving a small model, with three configurations compared under the same request mix: a small cache, a larger cache, and a larger cache with a front-end concurrency limit.

Minimal working example

Prereqs

• Docker, and a machine with enough memory for the model you choose. The documentation's example uses an 8-billion-parameter 4-bit model on a large CPU system; pick a smaller model if your hardware is smaller.
• The Model Server image and a model from a repository, started with a command in the style shown above.
• Python 3 with requests for the client.

client.py (request generator for the server's OpenAI-compatible endpoint)

import time, statistics, requests

URL = "http://localhost:8000/v3/chat/completions"
MODEL = "OpenVINO/Qwen3-8B-int4-ov"  # use the model name you deployed

def one_request(prompt):
    start = time.perf_counter()
    r = requests.post(URL, json={"model": MODEL, "messages": [{"role": "user", "content": prompt}], "max_tokens": 32, "stream": False})
    r.raise_for_status()
    return time.perf_counter() - start

shared_prefix = "Summarize the following notes. " * 200
latencies = [one_request(shared_prefix + f" Note {i}.") for i in range(30)]
print("median:", statistics.median(latencies))
print("p95:", sorted(latencies)[int(0.95 * len(latencies)) - 1])

This measures total request time with a small output, a rough proxy for time to first token. For a precise first-token time, use a streaming request and record when the first chunk arrives. Check the endpoint path and model name against the documentation for your version.

Adjust file names, model names, and paths to your own setup.

Test: Start the server with prefix caching on and an explicit --cache_size. Run client.py with a shared long prefix and again with unique prompts, and watch memory with docker stats. Then change one option, such as the cache size or prefix caching, and repeat. Compare the median and 95th percentile of request time between configurations.

Record: model, Model Server version, and host hardware; the full command line; the request mix (prompt lengths, shared-prefix share, concurrency); median and 95th percentile with prefix caching on and off; peak memory per configuration; your chosen --cache_size and the reason.

Checklist

• Record the version and every option you set.
• Set --cache_size explicitly.
• Test shared-prefix and unique-prompt workloads separately.
• Change one option per run and watch memory during the run.
• Report percentiles, not averages, and write the chosen values and evidence in your runbook.
• Avoid: trusting a short, single-user test.
• Avoid: copying the documentation's example --cache_size to different hardware.

Where this doesn't apply

Single-user local tools where concurrency is one, workloads whose prompts never repeat a prefix (prefix caching adds little), and deployments where an upstream gateway already limits concurrency tightly.

Sources

• Agentic AI with OpenVINO Model Server (example commands)
• Model Server continuous batching demo
• Model Server pull and conversion guide
• Model Server performance tuning guide
• OpenVINO Model Server parameters
• OpenVINO Model Server LLM reference

---

Logs, metrics and traces are three different tools, and they only help in an incident when they can be tied together. The common failure is not the lack of data. It is that the data is unstructured, expensive, or impossible to connect to a single request. Containers add a convention on top: write to standard output, and let the platform collect it. Everything else is about keeping that stream useful and cheap.

Options and trade-offs

• Plain-text logs written to files: easy at first, but hard to parse, and the files fill the container's disk.
• Structured JSON logs on standard output: parseable, collected by the platform, and the right default.
• Tracing everything: great detail, but the CPU and storage bill grows with traffic. Sampling keeps the cost predictable.

My recommendation

Log structured JSON to standard output with a small, fixed set of fields. Put a correlation ID on every request at the edge and carry it through the logs. Keep metric labels low cardinality, and sample traces. Alert on user-facing symptoms, not on every raw number.

Worked example

A servlet filter that makes sure every request has a correlation ID, puts it in the logging context, and returns it to the caller.

import jakarta.servlet.*;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.slf4j.MDC;

import java.io.IOException;
import java.util.UUID;

public class CorrelationFilter implements Filter {
    private static final String HEADER = "X-Correlation-Id";

    @Override
    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        String id = ((HttpServletRequest) req).getHeader(HEADER);
        if (id == null || id.isBlank() || id.length() > 64) {
            id = UUID.randomUUID().toString();
        }
        MDC.put("correlation_id", id);
        ((HttpServletResponse) res).setHeader(HEADER, id);
        try {
            chain.doFilter(req, res);
        } finally {
            MDC.remove("correlation_id");
        }
    }
}

The Logback configuration that writes each event as one JSON line to standard output, including the correlation ID from the MDC. It uses the open-source Logstash encoder.

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
  </appender>
  <root level="INFO">
    <appender-ref ref="STDOUT"/>
  </root>
</configuration>

Then log with parameters, so nothing is formatted when the level is off:

log.info("request_finished route={} status={} durationMs={}", route, status, elapsedMs);
log.warn("upstream_failed upstream={} reason={}", "inventory", "timeout");

Checklist

1. Log JSON to standard output, with level, timestamp, component, message and correlation ID.
2. Create the correlation ID at the edge, accept a sane inbound one, and cap its length.
3. Keep metric labels to a small set such as route class and outcome, never per user.
4. Sample traces, and raise the rate temporarily during an incident.
5. Alert on error rate and latency, and link the alert to a dashboard or runbook.

Where this doesn't apply

A single small service may not need distributed tracing at all. Structured logs with a correlation ID will already answer most questions.

One honest limitation

Observability has a cost. Every extra log line and span uses CPU and memory, and during an incident, noisy logging can make the incident worse.

Sources

• SLF4J Manual: Mapped Diagnostic Context (MDC)
• Logback Documentation: Appenders
• OpenTelemetry Documentation: Sampling
• Docker Docs: View container logs

The failure mode

You wake up to ten new issues and three PRs. Two are duplicates, three ask for features outside the project scope, and several lack context. You try to answer quickly and get pulled into long threads that don’t move the project forward. A week later, you’re behind, less responsive, and a contributor feels ignored.

What I learned

Label early, label consistently

Set up labels that reflect workflow, not internal discussions: “good first issue,” “help wanted,” “needs reproduction,” “needs test,” “scope discussion,” “declined” (with a short reason). Use them as soon as you read an issue. Labels create a shared vocabulary and make your backlog scannable.

Response-time expectations reduce anxiety

State response-time expectations publicly: for example, “New issues are triaged within two business days; PR reviews are typically completed within one week.” Adjust those targets to fit your team. If you’re out-of-office, post a brief note in Discussions or the README. Contributors appreciate clarity.

Friendly declines are a kindness — write them well

Declining an idea is not rejecting a person. Use a short, warm template that explains scope, suggests alternatives, and invites contribution along defined paths (e.g., plugins rather than core changes). Label the issue “declined” and close it with the message.

Decline template

Thanks for the idea! For now, this is out of scope for the project.

We’re keeping the core lean and recommending plugins for extensions.

If you’d like to explore it as a plugin, happy to provide a template link.

Require minimal tests, not maximal rigor

For small projects, a minimal test required by the PR template keeps quality up without deterring contributors. Tests should be fast, easy to write, and close to the code change. If a PR lacks tests, add a gentle reminder and label “needs test.”

Batch reviews and avoid context switching

Schedule triage windows — twice a week, for example. Batch issue labeling and PR reviews in those windows. It reduces context switching and lets contributors anticipate when they’ll hear back. If needed, add a triage bot that labels based on keywords to reduce manual work.

Document “how to get a fast review”

Spell out what gets fast attention: small PRs with tests, clean diffs, and narrow scope; issues with reproductions and logs; ideas aligned with plugin paths. Encourage contributors to write “before/after” summaries in PR descriptions and to link to the relevant part of the code.

Track your backlog and set a capacity

If too many issues pile up, prune. Close old issues that lack detail after a friendly ping. Move discussions into one “scope discussion” thread for big feature ideas. Keep your backlog reasonable and honest about capacity.

Trade-offs and failure modes

Trade-offs

Strict labels vs flexibility: Rules reduce effort but can feel rigid. Counterbalance with friendly messages and occasional exceptions.

Minimal tests vs depth: Minimal tests ensure basic correctness; complex changes may need deeper coverage. Ask for more when a change warrants it.

Batch triage vs real-time responsiveness: Batching reduces burnout; some contributors prefer faster feedback. Make expectations explicit.

Failure modes

“Unlabeled backlog”: Issues pile up without labels; you forget context. Solution: label on first read.

“Harsh declines”

A terse “no” closes doors. Solution: a kind template that suggests plugin paths or future reconsideration.

“Hidden OOO”: You vanish for a week without notice; contributors lose trust. Solution: brief status notes when you’re unavailable.

Try it yourself

Create and apply a lightweight label set. Add “good first issue” and curate five tasks.

Publish response-time expectations

and measure response times over the next month.

Adopt a friendly decline template and a short “how to get a fast review” section in CONTRIBUTING.md.

The failure mode

A team builds a credential verifier that checks JSON shape and a signature. The code reports “valid.” A downstream feature treats the claim as true. Weeks later, someone discovers the issuer did, in fact, sign the credential, but the subject was not the person currently presenting it, or the credential expired months ago. The failure wasn’t cryptography; it was process: treating “verifiable” as “verified.”

What I learned

Roles and responsibilities: issuer, holder, verifier

Issuer: Asserts something about a subject (the holder) and signs the assertion. Their trustworthiness is a human, contractual, and policy question, not a purely technical one.
Holder: Receives and stores the credential; presents it to a verifier. Binding the holder to the subject of the credential (is this person who the claim describes?) is a separate step.
Verifier: Evaluates signature validity, checks schema, ensures the issuer is trusted for that claim type, confirms the subject binding, interprets policy (expiry, audience, revocation), and decides whether to accept the claim.

Well-formed is necessary but not sufficient

A credential can be:

Well-formed: It conforms to expected JSON structure.
Signed: The cryptographic proof validates against the issuer’s public key.
Trusted: The issuer is recognized and allowed to assert this particular claim type.
Bound: The subject is provably the presenter.
Current: It hasn’t expired and isn’t revoked. Only when all are satisfied does “valid” approach “usable.”

Separate signature validation from trust decision

Cryptography answers “did the claimed issuer sign this payload?” Trust answers “do we trust this issuer for this claim type right now?” You need both. If the issuer isn’t in your trust registry for the claim being evaluated, the result shouldn’t be “invalid claim,” it should be “untrusted issuer for this policy.”

Subject binding is a first-class step

A classic failure mode: someone presents a signed credential about a subject, but the presenter isn’t the subject. In practical flows, the verifier should confirm that the presenter’s wallet or identity key pair matches the credential subject, or use an out-of-band binding method aligned with your policy. If binding is missing, treat the result as “unverifiable” rather than “false.”

Expiry and audience are policy gates, not parsing details

Expiry and audience are part of the claim semantics. A credential can be signed and well-formed yet expired (or intended for a different audience). These conditions should result in “policy failure” states distinct from “cryptographic failure.”

Revocation closes the loop

Issuance is easy; ensuring the claim is still valid is the hard part. Revocation checks must avoid “phone home” privacy leaks (more in the revocation article below). Even a toy system benefits from an offline revocation list or privacy-preserving status check.

Distinguish “invalid” from “unverifiable”

Illustrative verification flow

from dataclasses import dataclass
from typing import Literal

Status = Literal["accepted", "unverifiable", "invalid"]

@dataclass(frozen=True)
class ValidationResult:
    signature_valid: bool
    issuer_trusted: bool
    subject_bound: bool
    policy_ok: bool
    status: Status

# Keep proof validation, issuer trust, subject binding, and policy checks separate.
# The exact proof checks depend on the credential format and verifier library supported by the deployment.

Trade-offs and failure modes

Trade-offs

Strict policy gates vs user friction: Strong audience/expiry checks reduce misuse but can frustrate users who present slightly out-of-date credentials. The fix is short-lived credentials and good UX.

Offline trust lists vs dynamic fetch: Offline lists avoid privacy leaks but risk staleness. Consider periodic sync with no per-verification lookups.

Subject binding rigor vs simplicity: Strong binding reduces fraud but adds ceremony. Pick the approach that fits your risk.

Failure modes

“Signed, therefore true”: Conflating cryptographic validity with trust. Fix by separating proof validation from issuer trust policy.

“Presenter ≠ subject”: Accepting credentials without subject binding. Fix by explicit binding checks.

“Policy drift”: Undocumented audience/expiry rules. Fix with centralized policy and tests.

Try it yourself

Implement a verifier that separately evaluates cryptographic proof, issuer trust, subject binding, and policy gates.

Return “unverifiable” for missing binding and “invalid” for failed policy (e.g., expiry).

Measure performance across well-formed accepts vs policy failures to understand user impact.

public interface ProfileApi {
    Optional<Profile> get(String id);
    Map<String, Profile> batchGet(Collection<String> ids);   // missing ids are omitted
}

public record Profile(String id, String headline) { }

import org.junit.jupiter.api.Test;
import java.util.*;
import static org.junit.jupiter.api.Assertions.*;

public abstract class ProfileApiContract {

    protected abstract ProfileApi api();

    @Test
    void getReturnsEmptyForUnknownId() {
        assertTrue(api().get("does-not-exist").isEmpty());
    }

    @Test
    void batchGetOmitsMissingIdsAndKeepsTheRest() {
        Map<String, Profile> result = api().batchGet(List.of("p1", "missing", "p2"));
        assertEquals(Set.of("p1", "p2"), result.keySet());
    }

    @Test
    void batchGetWithNoIdsReturnsEmptyMap() {
        assertTrue(api().batchGet(List.of()).isEmpty());
    }

    @Test
    void batchGetWithDuplicateIdsReturnsEachOnce() {
        assertEquals(1, api().batchGet(List.of("p1", "p1")).size());
    }
}

class LegacyProfileApiTest extends ProfileApiContract {
    @Override protected ProfileApi api() { return TestFixtures.legacyApi(); }
}

class BridgedProfileApiTest extends ProfileApiContract {
    @Override protected ProfileApi api() { return TestFixtures.grpcBackedApi(); }
}

Make gRPC tests hermetic

import io.grpc.ManagedChannel;
import io.grpc.Server;
import io.grpc.inprocess.InProcessChannelBuilder;
import io.grpc.inprocess.InProcessServerBuilder;

String name = InProcessServerBuilder.generateName();
Server server = InProcessServerBuilder.forName(name)
        .directExecutor()
        .addService(new ProfileGrpcService(fakeBackend))
        .build().start();
ManagedChannel channel = InProcessChannelBuilder.forName(name).directExecutor().build();

Record real interactions, replay them in tests

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;

class ReplayTest {

    record Fixture(String name, List<String> ids, Set<String> expectedKeys) { }

    static List<Fixture> fixtures() {
        return FixtureLoader.loadAll("fixtures/");     // JSON files kept in src/test/resources
    }

    @ParameterizedTest(name = "{0}")
    @MethodSource("fixtures")
    void bridgeMatchesRecordedBehavior(Fixture f) {
        Map<String, Profile> result = TestFixtures.grpcBackedApi().batchGet(f.ids());
        assertEquals(f.expectedKeys(), result.keySet());
    }
}

Reducing flakiness

• No real clocks or randomness. Inject a Clock and seed random generators.
• No shared state across tests. Build fresh fixtures per test.
• Wait on conditions, not sleeps. Use Awaitility or latches.
• Quarantine, then fix. Tag known-flaky tests so they do not block merges, and track them.

Coverage that matters

Failure modes

• Test wiring that works locally but misses configuration files in CI. Keep test resources under standard source-set paths.
• Contract tests that only cover happy paths.
• Fixtures that go stale. Re-record on a schedule.

Takeaways

• Test the bridge with one shared contract run against both implementations.
• Use in-process gRPC for hermetic tests, and recorded, scrubbed fixtures for realism.
• Treat batch behavior and error mapping as first-class contract items.

This is not a claim that scanners make an image safe. A scan is one control in a larger process. The goal is to replace last-minute surprises with a repeatable decision before an image reaches production.

Start with an inventory

A Software Bill of Materials (SBOM) lists software components included in an image. Generating one during the build gives scanners and reviewers a concrete inventory to inspect. Docker Buildx can attach an SBOM attestation to an image:

# Build and push to a registry so the SBOM attestation travels with the image.
docker buildx build \
  --tag "$IMAGE" \
  --sbom=true \
  --push .

Build attestations depend on the BuildKit builder and where the image is stored. Verify that the registry contains the SBOM and that your scanner can read it; do not assume a successful command means the inventory is being retained in the place you expect.

Gate on findings the team can act on

Pick a severity policy and make it a failing CI step. For example, Trivy can return a non-zero exit status when High or Critical findings are present:

# IMAGE is the exact tag or digest built by the preceding step.
trivy image \
  --exit-code 1 \
  --severity HIGH,CRITICAL \
  "$IMAGE"

Choose policy with the people who own the service. Severity labels are useful triage signals, but they do not tell you whether a vulnerable code path is reachable in your application. Prioritize fixable findings; keep visibility into the rest and record remediation plans.

Make exceptions narrow and temporary

Sometimes a scanner reports a finding with no available fix, or a team can show that a vulnerable path is not reachable in its deployment. If your policy allows an exception, require at least:

• a named owner responsible for revisiting it;
• a short technical justification and the affected image digest;
• a remediation plan or a reason no fix is available;
• an expiry date that triggers re-review.

An exception should have an end. An expired exception returns to the gate for a fresh decision.

Scan third-party images and every release candidate

Review a base image or build tool before first use, prefer trustworthy publishers, and record the digest of the image you evaluated. Scan the image that will actually be released, not only the Dockerfile or an earlier local build. Repeat the scan when rebuilding because packages and advisories change.

Treat scanner failures as operational failures

Scanning can fail because a registry is unavailable, an advisory feed cannot be fetched, a token expired, or the runner ran out of disk. These states are different from “the scan completed and found a vulnerability.” Preserve enough CI status and logs to tell which happened, and make sure infrastructure errors cannot turn into a silent pass.

Keep the gate fast enough to keep

If every scan takes too long, teams will look for ways around it. Cache what is safe to cache, tune the scanner, and make results easy to understand—but do not quietly skip scanning for small code changes. Also record who approved exceptions and when; keep logs free of credentials and other secrets.

Failure modes to test

• The gate reports but does not block: test the pipeline with a known finding and confirm the build fails.
• Advisory access expires: make registry or scanner authentication failures visible and blocking.
• The SBOM is missing: treat a missing expected artifact as a failed build, not an empty inventory.
• A clean scan becomes stale: rebuild and rescan as dependencies or vulnerability data change.

Try it in your pipeline

1. Generate an SBOM for each release image and verify that it is retained.
2. Run a vulnerability scan on the exact image tag or digest that will ship.
3. Set a failing threshold, such as High and Critical, with a written exception policy.
4. Keep scanner outages separate from vulnerability findings in the CI result.

Closing thought

A scan is most useful when it changes a release decision. Make image contents visible, agree on what blocks a release, and give every exception an owner and a clock.

The failure mode

You change an endpoint’s response shape from {type:"present"} to {status:"present"} and add a field. Tests in your project pass because you updated them. A user’s integration breaks silently because they were parsing the old field. You release as a minor bump. Issues pile up. SemVer wasn’t just a label; it was a promise you broke.

What I learned

Define “breaking” for your project — then honor it

For a public API, breaking means any change that requires user action. That includes renames, removals, default behavior changes, stricter validation, and shifts in documented semantics. For plugin contracts, it includes method signatures and return-object fields. Write this down in CONTRIBUTING.md or RELEASE.md.

Use PATCH for fixes, MINOR for additive changes, MAJOR for breaking

PATCH: bug fixes that don’t change behavior or contracts. MINOR: new additive features, fields, or optional paths that don’t require code changes from users. MAJOR: anything else. If in doubt, lean toward stricter classification — small projects benefit from conservative signaling.

Keep a changelog that’s useful, not performative

A good CHANGELOG.md lists new features, fixes, and breaking changes with clear “migration” steps. Link to PRs and discussions when helpful, but don’t drown readers in noise. Make breaking changes a dedicated section with bullet points and example code.

Changelog entry (example)

## [2.0.0] - 2026-02-15
### Breaking
- Plugin interface: `evaluate()` return now requires `score_delta`.
- API response: field `note` renamed to `message`.
- Migration:
  - Plugins: provide a score delta when constructing the result.
  - Clients: read the renamed message field.
### Added
- New synthetic evidence source: "activity_count".
### Fixed
- Corrected null-handling in JSON serialization for missing evidence.

Automate release notes and tags, but write the words yourself

Automated release workflows help: tag versions, publish notes, and attach artifacts. The content of the notes should be authored by you or a maintainer who understands the changes. Meaningful context beats auto-generated lists.

Communicate before you break

— deprecation notes and timelines If you plan a breaking change, announce it with a deprecation note. Add warnings to logs, include a deprecation section in the changelog, and post a short explanation in discussions. Offer a migration window (e.g., two minor releases) and stick to it.

Validate SemVer behavior with contract tests

Combine SemVer discipline with tests: run a suite that loads the previous minor version’s contracts against the new release where feasible. If tests fail because you changed behavior without bumping MAJOR, you’ll catch it before publishing.

Keep scope under control

— fewer breaking changes, batched when necessary Batch breaking changes into a single major release rather than sprinkling them across minors. This helps users plan upgrades and reduces your triage load. Once you ship a major, leave a quiet window before the next one.

Trade-offs and failure modes

Trade-offs

Strict SemVer vs development speed: Bumping MAJOR and writing migration notes take time. They also save user time and reduce issue volume.

Deprecation windows vs maintenance overhead: Supporting old paths gives contributors time to adapt. For small projects, choose short, firm windows.

Automated release processes vs human judgment: Automation speeds delivery; human-curated notes ensure clarity. Pair them.

Failure modes

“Silent breaking minor”: Change that requires user action shipped as MINOR. It erodes trust and increases support requirements.

“Vague changelog”: Readers can’t tell what changed. Solution: explicit “Breaking,” “Added,” “Fixed” sections with migration steps.

“SemVer theater”: You declare SemVer, then ignore it in practice. Users stop trusting your version signals. Solution: define rules and follow them.

Try it yourself

Write a short “What counts as breaking” policy tailored to your project and add it to CONTRIBUTING.md.

Adopt Keep a Changelog structure and commit to explicit migration notes for breaking changes.

Add a contract test suite that runs compatibility checks against the previous minor version. If it fails, reconsider your classification.

Oracle Cloud Infrastructure Registry (OCIR) is simple once you know the rules, and confusing until then. Most failures come from the same few places: a wrong region endpoint, a wrong username format, or an expired token.

I wanted a short reference that gets a private repository working and keeps it private.

Options and trade-offs

• Public repositories: easy to share, and only suitable for images that are safe for anyone to pull.
• Private repositories with auth tokens: the standard approach, and it needs good token hygiene.
• Short-lived or bearer-style credentials: possible for pipelines, with more setup to maintain.

My recommendation

Use private repositories and user auth tokens. Create the repository explicitly in the compartment that owns it, instead of letting a first push create it. Keep the regional endpoint and the username format written down where the team can find them.

Worked example

Log in with the registry endpoint for your region. The username begins with your tenancy namespace. If your user belongs to a non-default identity domain or is federated, the username format changes, so check the documentation for your case.

docker login <region-key>.ocir.io
# Username: <tenancy-namespace>/<username>
# Password: <auth-token>
 
oci artifacts container repository create \
  --compartment-id <compartment_ocid> \
  --display-name myteam/myapp \
  --is-public false
 
docker tag myapp:1.4.2 <region-key>.ocir.io/<tenancy-namespace>/myteam/myapp:1.4.2
docker push <region-key>.ocir.io/<tenancy-namespace>/myteam/myapp:1.4.2

And the policies, kept separate for people who browse and people who manage:

Allow group image-viewers to inspect repos in tenancy
Allow group image-publishers to manage repos in compartment registry

I tag with a version, not latest, so a rollback is a one-line change. A user can hold only a small number of auth tokens, so plan token replacement for your pipelines before it is urgent.

Checklist

1. Know the endpoint for your region and use it consistently.
2. Use the tenancy namespace in the username, and the auth token as the password.
3. Create repositories explicitly and confirm they are private.
4. Set separate policies for viewing and managing repositories.
5. Tag images with versions, and avoid latest in production.
6. Set a retention rule, and watch pulls for surprises.

Where this doesn't apply

Air-gapped clusters cannot reach OCIR directly. They need a mirrored or internal registry instead.

One honest limitation

Auth tokens expire or get replaced, and a stale token in a pipeline fails at the worst time. Treat it like any other credential and give it an owner and a rotation date.

Sources

• OCI Documentation: Pushing Images Using the Docker CLI
• OCI Documentation: Logging in to Oracle Cloud Infrastructure Registry
• OCI Documentation: Required Policies for Container Registry

AKS Workload Identity replaces the deprecated AAD Pod Identity and removes static secrets from pods. A rollout touches the control plane, depends on SDK versions in every workload, and can fail in ways that only show up under load or after a token refresh. Without a plan, the first sign of a problem is an application that cannot reach its database.

Options and trade-offs

• Enable the OIDC issuer and Workload Identity on the cluster: a one-time change with a short control-plane impact. Announce a window for it.
• Parallel migration: run Workload Identity next to Pod Identity, move one workload at a time, verify each, then remove Pod Identity. It is slower, and it is the only approach with a clean rollback.
• Sidecar proxy for older SDKs: a temporary bridge that intercepts calls to the instance metadata endpoint for workloads that cannot upgrade yet. Treat it as a deadline, not a destination.

My recommendation

Migrate per namespace, starting with a low-risk internal workload. Define rollback criteria before the first change: error rate on token acquisition, authentication failures to the target service, and pod restarts. Do not remove Pod Identity until every workload has run on Workload Identity through at least one full token-renewal cycle.

Minimal working example

Enable the cluster features and look up the issuer:

az aks update --resource-group <rg> --name <aks> \
  --enable-oidc-issuer --enable-workload-identity

OIDC_ISSUER=$(az aks show -g <rg> -n <aks> --query "oidcIssuerProfile.issuerUrl" -o tsv)

Create a managed identity and federate it with one service account:

az identity create --name app-identity --resource-group <rg>
CLIENT_ID=$(az identity show -n app-identity -g <rg> --query clientId -o tsv)

az identity federated-credential create \
  --name app-fic \
  --identity-name app-identity \
  --resource-group <rg> \
  --issuer "$OIDC_ISSUER" \
  --subject system:serviceaccount:<namespace>:app-sa \
  --audience api://AzureADTokenExchange

Annotate the service account and label the pod:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa
  namespace: <namespace>
  annotations:
    azure.workload.identity/client-id: "<client-id>"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
  namespace: <namespace>
spec:
  replicas: 2
  selector:
    matchLabels: { app: app }
  template:
    metadata:
      labels:
        app: app
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: app-sa
      containers:
        - name: app
          image: <registry>/app:<tag>

Test it: deploy one workload, confirm it acquires a token and reaches its Azure dependency, then restart it and confirm it still does after the token refreshes.

Checklist

• Inventory workloads that still call the instance metadata endpoint or use older SDKs, and decide per workload whether to upgrade or bridge.
• Use one service account and one federated credential per workload identity, and keep least-privilege Azure RBAC on the identity.
• Write down rollback criteria and who decides, before the change window.
• Migrate one namespace at a time, and watch token errors and pod restarts after each.
• Remove Pod Identity only after every workload has been stable on Workload Identity, and confirm the add-on's end-of-support dates.
• Keep a record of every federated credential so stale ones can be cleaned up.

Where this doesn't apply

Kubernetes clusters outside Azure. They need the equivalent OIDC issuer configuration and federation, if the platform offers it.

Sources

• Workload Identity overview
• Migrate from Pod Identity
• Use AAD Pod Identity (deprecation notice)

---

Modern Java can express these outcomes as records and sealed types: Present, Missing, and Unverifiable. The same pattern works in ordinary service domains; it does not depend on a scoring model.

The failure mode

You return a number for a score and a boolean “hasEvidence.” A caller subtracts two scores and treats “0” as “no evidence,” then logs an anomaly when “0” appears alongside “hasEvidence=true.” The model caused this. Nulls and booleans encourage shortcuts and misinterpretation.

What I learned

Use records for immutable, transparent domain data

Records let me define concise, immutable carriers without boilerplate. They encourage a bias toward immutability and make the intent clear.

Example response

public record AceuResponse(
ScoreOrUnknown authenticity,
ScoreOrUnknown credibility,
ScoreOrUnknown empathy,
ScoreOrUnknown uniqueness
) {}
public sealed interface ScoreOrUnknown permits Score, Missing, Unverifiable {}
public record Score(double value) implements ScoreOrUnknown {
public Score {
if (Double.isNaN(value) || value < 0.0 || value > 1.0) {
throw new IllegalArgumentException("Score must be [0,1]");
}
}
}
public record Missing(String reason) implements ScoreOrUnknown {}
public record Unverifiable(String note) implements ScoreOrUnknown {}

Records keep the data small and immutable. The sealed hierarchy enforces the set of states at compile time.

Replace nulls and boolean flags with sealed states

The temptation to use Double/Boolean and encode special states with null or false is strong. Resist it. Sealed interfaces make the valid state space explicit and force exhaustive handling. A boolean can’t distinguish “not applicable,” “not yet computed,” and “calculation failed.”

Use pattern matching for exhaustive case handling

Pattern matching in switch makes code read like the domain. It also turns omissions into compile errors when you use exhaustive switches.

Example handler:

public String render(ScoreOrUnknown s) {
return switch (s) {
case Score sc          -> "score=" + sc.value();
case Missing miss      -> "missing=" + miss.reason();
case Unverifiable unv  -> "unverifiable=" + unv.note();

};

}

By handling each case explicitly, I make it impossible to “forget” one or conflate states.

Prefer intent-rich types over ad-hoc maps and strings

When a state needs more context (e.g., why data is missing), capture it in the type. Missing(String reason) and Unverifiable(String note) beat opaque strings and comments. If callers need to act differently based on the reason, add a reason code enum to the record.

Keep aggregates typed—don’t flatten too early

A response with four dimensions should preserve each dimension’s state. Avoid flattening to a single “overall score.” The minute you collapse, you discard information the caller needs for good decisions. In the demo, each dimension returns a ScoreOrUnknown. Only the caller decides how to combine them, and the compiler forces handling of unknowns.

Embed validation where it belongs

Put invariants in the type’s constructor. Scores are [0,1]; reasons should be concise, not multi-paragraph blobs. Records make these guardrails trivial to implement consistently.

Use test fixtures that exercise every branch

Define parameterized tests for your sealed states. For each dimension, test Score, Missing, and Unverifiable. Assert that pattern matching hits each branch, and that any new state added later causes compilation failures in exhaustive switches until it’s handled.

Trade-offs and failure modes

Trade-offs

More types vs fewer booleans: Sealed types add structure and a few extra classes. The upside is clarity and compile-time enforcement; the downside is minor boilerplate, which records minimize.

Exhaustiveness vs flexibility: Exhaustive switches break on new states until handled. That’s a feature (safety) but can feel strict when you prototype quickly.

Text reasons vs structured codes: Free-form strings are easier; structured enums enable programmatic handling. Choose based on how callers use the data.

Failure modes

“Shadow nulls”

A downstream library returns null and you forget to wrap it in a sealed state. Guard external boundaries with adapters.

“State leakage”: Someone serializes ScoreOrUnknown to a shape that drops the state label (e.g., value=0.0 only). Add JSON configuration that includes a type field and test the wire format.

“Catch-all case”: Using a default branch in pattern matching reintroduces ambiguity. Avoid default; keep exhaustive cases explicit.

Try it yourself

Replace nullable fields for ambiguous states with a sealed interface and record implementations. Add exhaustive pattern matching where you consume them.

Write tests that construct each state per dimension and assert wire-format correctness (e.g., JSON includes the state “type”).

Audit one path where a null sneaks through and add an adapter to convert it into Missing or Unverifiable with a reason code.

OpenVINO lets you name a device directly, ask for AUTO selection, or split a model across devices with HETERO. They look like three spellings of one decision, but they behave differently when a device is missing, when you restart, and when you cache compiled models.

The wrong choice rarely fails loudly. It shows up as a first request served by a different device than you expected, a cache that never hits, or a fallback that quietly runs on the CPU and misses a latency target.

The friction points are knowing what AUTO does at startup, knowing where work actually ran, and testing restarts instead of only first starts.

Options and trade-offs

• Explicit device: compile for CPU, GPU, or NPU by name. The behavior is easiest to reason about, and the compiled-model cache belongs to that concrete device. Routing and failover are your application's job.
• AUTO: the documentation says AUTO detects available devices and selects the best one automatically. By default it starts inference on the CPU unless there is a cached model for the best-suited device, because the CPU can start with no additional delay. While the CPU runs inference, AUTO keeps loading the model onto the better device and hands the work over when ready.
• AUTO startup behavior is configurable: the enable_startup_fallback property controls whether the CPU is used as the initial helper device (default true), and enable_runtime_fallback controls whether a failed inference request is retried on another device (default true). Excluding the CPU from the priority list disables the initial CPU acceleration.
• AUTO reports where work actually ran: the execution_devices property lists the devices currently executing inference. Logging it is the cheapest way to avoid surprises.
• A published OpenVINO blog describes AUTO trying the next best device if the model does not compile on the first choice, with the CPU as the final fallback because it supports all operations and precisions.
• HETERO splits one model across several devices. It is meant for cases where a model does not fit on one device or part of it needs different hardware. It adds configuration and testing work, so treat it as a special-purpose tool and test the exact model and device pair you intend to ship.
• Caching interaction: compiling explicitly for each device you use makes cache behavior easy to reason about. With AUTO, early requests may be served by the CPU while the better device loads, so first-request latency depends on whether a cached model exists for that better device.
• Recommendation: use an explicit device when you need predictable startup and cache behavior, and put routing and failover in your application where you can log and test them. Use AUTO when portability across machines matters more than deterministic timing, and always log execution_devices. Treat HETERO as an advanced option adopted only when a specific model needs it.
• Decision lens: predictability versus portability. Example setup: an application that runs on laptops with varied hardware uses AUTO and logs its execution device; a service on a fixed server fleet compiles explicitly for the device it has.

Minimal working example

Prereqs

• Python 3 with the openvino package installed.
• A machine with at least two device types if possible, such as CPU and an integrated GPU or NPU.
• A model you already run.

compare_devices.py

import sys, time
import openvino as ov

model_path = sys.argv[1]
core = ov.Core()
core.set_property({"CACHE_DIR": "./model_cache"})
print("Available devices:", core.available_devices)

for target in ["CPU", "AUTO"]:
    start = time.perf_counter()
    compiled = core.compile_model(model_path, target)
    elapsed = time.perf_counter() - start
    print(target, f"compile {elapsed:.3f}s", "executing on:", compiled.get_property("EXECUTION_DEVICES"))

Add "GPU" or "NPU" to the list if your machine has them.

Adjust file names, model names, and paths to your own setup.

Test: Run compare_devices.py twice in separate processes, once with an empty cache folder and once with a populated one. Note the device reported by EXECUTION_DEVICES right after compilation and again after a few inferences, because AUTO can switch devices. If you can do it safely, disable a device and rerun to see how each strategy reacts.

Record: devices present and OpenVINO version; device reported by AUTO right after compile and after warm-up; time to first inference for each strategy with an empty and a populated cache, as median and 95th percentile; behavior when the preferred device is unavailable; your decision and the reason.

Checklist

• Log the executing device in every run.
• Compile explicitly per device when cache behavior must be predictable.
• Review the startup and runtime fallback settings if you use AUTO.
• Test restarts, not only first starts, and pin the versions you tested.
• Document the failover behavior you observed.
• Avoid: assuming AUTO serves the first request on the best device.
• Avoid: adopting HETERO without testing the exact model.

Where this doesn't apply

Single-device deployments where the choice is trivial, systems whose hardware is fixed and known (explicit selection is simpler), and models too large for any single device, where HETERO may be unavoidable.

Sources

• Automatic Device Selection documentation
• Automatic Device Selection and Configuration (OpenVINO blog)
• OpenVINO Model Caching Overview

---

Caching exists to save time, so it is tempting to treat it as invisible. But a cached artifact is a compiled program saved to disk. Reusing it assumes that the conditions that produced it still hold: the same device, a compatible software version, and the same model and input shapes.

Two things make this risky in practice. OpenVINO's documentation says devices without import and export support run normally but do not get the speed-up, so a missing cache does not announce itself. And an artifact left over from an earlier version or a different model can be picked up by accident if folders are not separated.

The friction points are confirming that a cache was created, confirming that a warm start produces the same outputs as a cold start, and refusing to serve when it does not.

Options and trade-offs

• Check that something was cached: after the first run, the cache folder should contain new files. An empty folder after a successful compile suggests the device does not support caching or the folder is not writable.
• Compare outputs: run one fixed input through a freshly compiled model and through a model loaded from the cache, then compare the results with a tolerance you choose in advance. For deterministic models on the same device, exact equality is a reasonable expectation; choose a tolerance only if your pipeline needs one.
• Separate cache folders by software version, model version, device, and input shape. This costs nothing and removes a whole class of confusing failures.
• Fail closed: if the output comparison fails or loading from the cache raises an error, delete that cache entry, recompile, and log the event. Do not serve from a cache you could not verify.
• Keep the check cheap: a single fixed input at startup usually catches a gross failure, and it can run alongside your warm-up request.
• Be cautious with outside reports: issue trackers contain reports about specific edge cases. Treat them as leads for your own tests, and do not build policy around a problem you cannot reproduce on your version.
• Recommendation: add a verification step to startup. After the model loads, run one fixed input and compare it with a stored reference. If the comparison fails, discard the cache entry, recompile, and record that the warm start was rejected. Log whether each start was cold or cached, how long compilation took, and which device and versions were involved.
• Decision lens: speed against silent error risk. Example setup: a service checks one reference input after every restart, treats a mismatch as a failed start, and writes each version's cache to its own directory so an upgrade never reads the previous version's files.

Minimal working example

Prereqs

• Python 3 with openvino and numpy installed.
• A model with a floating-point input; adjust the dtype in the script if yours differs.
• A writable cache folder.

verify_cache.py (run with fresh first, then cached twice)

import sys
import numpy as np
import openvino as ov

model_path, mode = sys.argv[1], sys.argv[2]  # mode: fresh or cached
core = ov.Core()
if mode == "cached":
    core.set_property({"CACHE_DIR": "./model_cache"})
compiled = core.compile_model(model_path, "CPU")

rng = np.random.default_rng(0)
shape = [d.get_length() for d in compiled.input(0).partial_shape]
x = rng.random(shape, dtype=np.float32)  # adjust dtype to match your model
result = compiled(x)[compiled.output(0)]

if mode == "fresh":
    np.save("reference.npy", result)
    print("saved reference")
else:
    reference = np.load("reference.npy")
    print("max abs difference:", float(np.max(np.abs(reference - result))))

Adjust file names, model names, and paths to your own setup.

Test: Run verify_cache.py with fresh to store a reference, then with cached to populate the cache, then with cached again to compare an imported model against the reference. Next, change the model file (or its version folder) and confirm that your folder naming prevents the old cache from being used. Repeat after upgrading OpenVINO. A difference larger than the tolerance you chose is a failed start.

Record: device, operating system, OpenVINO version, and model; files present in the cache folder after the first run; maximum absolute output difference between cold and cached; the tolerance you chose and why; behavior after the model changes; behavior after an upgrade; time added by the startup check.

Checklist

• Confirm the cache folder is populated after the first run.
• Compare a fixed output between a cold run and a cached run, and record your tolerance.
• Separate cache folders by version, model, device, and shape.
• Fail closed and recompile on a mismatch.
• Log cold or cached on every start, and re-run the check after every upgrade.
• Avoid: assuming a cache exists because no error appeared.
• Avoid: reusing one cache folder across versions or models.

Where this doesn't apply

Services that never use compiled-model caching, and systems that already compare outputs against a reference on every start.

Sources

• OpenVINO Model Caching Overview
• OpenVINO Model Server parameters
• OpenVINO NPU device documentation

---

The failure mode

A contributor refactors a plugin interface, renaming a method and changing its return shape. Their tests pass. Your CI passes. You merge. One week later, a different contributor’s PR, built on the previous interface, fails unexpectedly. You didn’t catch the breaking change because you had no compatibility tests enforcing the plugin contract.

What I learned

Treat extension points as public contracts

If you support a plugin API, document it as a minimal, stable contract. Keep it small, reduce optional features, and define explicit versioning. Favor additive changes; avoid changing semantics quietly.

Minimal contract (illustrative)

# your_project/core.py
from dataclasses import dataclass
@dataclass(frozen=True)
class EvidenceResult:
type: str  # "present" | "missing" | "unverifiable"
note: str = ""
score_delta: float = 0.0
class EvidenceSource:
name: str
def evaluate(self, profile: dict) -> EvidenceResult:
"""
Input: dictionary of profile fields (synthetic in demo)
Output: EvidenceResult with type, note, and optional score_delta
"""
raise NotImplementedError

Version the interface explicitly

Add an interface version constant in the core module. Plugins import and declare the version they target. CI checks that plugin declarations match the core version and that any bump is accompanied by a deprecation window.

Versioning example:

# your_project/core_version.py
INTERFACE_VERSION = "1.2.0"  # semantic versioned
# plugin declares support
SUPPORTED_VERSIONS = {"1.2.0"}  # plugins can add multiple supported versions

Deprecation windows — commit to dates and behavior When you change the interface, announce deprecations with a grace window: two minor releases or a set number of weeks. During the window, keep the old behavior working and warn at runtime (without breaking tests). At the end, remove the old path and bump the interface version.

Contract tests in CI

Unit tests validate plugin internals. Contract tests validate extension points. Write tests that load every plugin and execute a baseline scenario: call evaluate() with a standard profile, assert return types and fields, and assert compatibility with the current core version. Fail the build on mismatch.

Contract test skeleton:

import importlib

from your_project.core import EvidenceResult
from your_project.core_version import INTERFACE_VERSION

PLUGIN_MODULES = ["your_project.plugins.example_plugin"]

def test_all_plugins_implement_contract():
    for module_name in PLUGIN_MODULES:
        module = importlib.import_module(module_name)
        plugin = module.ExamplePlugin()
        result = plugin.evaluate({"example": True})
        assert isinstance(result, EvidenceResult)
        assert result.status in {"present", "missing", "unverifiable"}
        assert INTERFACE_VERSION in module.SUPPORTED_VERSIONS

Your CI should run both unit and contract tests. If the core changes, test the change against every supported plugin. If external plugins live in separate repositories, publish a small test harness they can add to their own CI.

Communicate interface changes in release notes Semantic version your interface. A breaking change to the plugin API is a major bump. The release notes should list exactly what changed and a minimal migration example. They should also link deprecation windows and final removal dates.

Example release note

## Plugin API 2.0.0 (breaking)
- Keep the result status field unchanged.
- Make the score delta field required.
- Migration: provide a zero delta when a plugin has no score change.
- Publish a deprecation window and removal date.

Keep extension points narrow Every new method or field in a plugin contract increases fragility. Keep it small. If contributors want flexibility, let them model complexity inside their plugin; keep the core contract simple, documented, and testable.

Trade-offs and failure modes

Trade-offs

Strict contract tests vs contributor agility: Some contributors will feel slowed by extra checks. Counterbalance with templates and examples.

Deprecation windows vs maintenance overhead: Maintaining old paths for a window adds complexity. Keep windows short and focused for small projects.

Central testing vs distributed plugins

If plugins live elsewhere, you can’t run their tests pre-merge. Provide a harness and encourage adoption.

Failure modes

“Silent break”

A contributor changes the contract; unit tests pass; no compatibility suite fails. Solution: contract tests that load plugins and validate the core expectations.

“Major bump fatigue”: Frequent major bumps erode trust. Solution: batch breaking changes and make additive changes where possible.

“Ambiguous release notes”: Vague notes cause confusion. Solution: concrete migration steps and example code in release notes.

Try it yourself

Add an INTERFACE_VERSION constant and require plugins to declare supported versions.

Implement CI contract tests that import and execute all plugins against the core contract.

Document a deprecation window policy in CONTRIBUTING.md. Commit to timelines and stick to them.

Mutual TLS to Autonomous Database needs a wallet. In practice that wallet ends up in a chat thread, in a repo, or baked into a container image, and rotating it means finding every copy.

I wanted the wallet to have one home, a controlled way out, and a short life on disk.

Options and trade-offs

• Copying the wallet files around: fast, hard to rotate and hard to audit.
• Wallet files as Vault secrets: access is governed by IAM and logged. The files need encoding and a rebuild step.
• Server-authenticated TLS without a wallet: fewer artifacts. It depends on the database allowing TLS connections without mTLS.

My recommendation

Generate the wallet from the database service, store its files as Vault secrets, and let the runtime principal read them at startup. The application recreates the files in a protected directory, sets TNS_ADMIN, and connects through the TNS alias.

Worked example

Generate the wallet and push the files into Vault. The CLI needs base64 content, while the Console encodes plain text for you.

oci db autonomous-database generate-wallet \
  --autonomous-database-id <db_ocid> \
  --file wallet.zip \
  --password "<wallet_password>"
 
unzip wallet.zip -d wallet
base64 -w0 wallet/cwallet.sso > cwallet.sso.b64
 
oci vault secret create-base64 \
  --compartment-id <compartment_ocid> \
  --vault-id <vault_ocid> \
  --key-id <symmetric_key_ocid> \
  --secret-name db_wallet_cwallet_sso \
  --secret-content-content "$(cat cwallet.sso.b64)" \
  --secret-content-stage CURRENT

Repeat for the other files the connection needs, usually tnsnames.ora, sqlnet.ora and ojdbc.properties. At startup, the app reads each secret, decodes it and writes it into a directory only its own user can access. The sketch below shows the permission handling.

Path dir = Files.createTempDirectory("wallet",
    PosixFilePermissions.asFileAttribute(PosixFilePermissions.fromString("rwx------")));
for (var e : walletSecrets.entrySet()) {          // name -> base64 content
    Files.write(dir.resolve(e.getKey()), Base64.getDecoder().decode(e.getValue()));
}
// start the JVM or pool with -Doracle.net.tns_admin=<dir> or TNS_ADMIN=<dir>

The ojdbc.properties file that ships in the wallet refers to the TNS_ADMIN directory for its wallet location, so it works unchanged when you rebuild the directory.

Checklist

1. Generate the wallet from the database service and track when it expires.
2. Store each file as a secret, using base64 for CLI and API paths.
3. Give the runtime principal read access to those secrets only.
4. Rebuild the wallet directory at runtime with owner-only permissions, and never bake it into an image.
5. Confirm the server certificate name check is on, and log wallet reads.
6. Keep the previous secret version for a short time in case rotation goes wrong.

Where this doesn't apply

If your policy allows server-authenticated TLS, you may not need a wallet at all. That removes the distribution problem entirely.

One honest limitation

Wallets are multi-file, and the cleanup of temporary files is easy to forget. Test the rebuild path and the deletion path as carefully as the connection itself.

Sources

• OCI Documentation: Downloading Client Credentials (Wallets) for Autonomous Database
• OCI CLI Reference: oci vault secret create-base64
• OCI Documentation: Connecting with JDBC and a Wallet

Without limits, a hot container can starve its neighbours. With limits that are too tight, it gets killed and restarts in a loop. Either way, the failure spreads: one tier is slow, upstream timeouts fire, retries pile up, and a small problem becomes an outage. On the JVM there is an extra trap. The heap is only part of the memory a process uses. Metaspace, thread stacks, the code cache, direct buffers and GC structures all live outside it, so a container limit equal to the heap size is almost guaranteed to be exceeded.

Options and trade-offs

• No limits: nothing gets killed by the runtime, but one runaway service can degrade the whole host.
• A hard limit with a fixed -Xmx: predictable, but easy to get wrong when the limit changes and the heap flag does not.
• A hard limit with a percentage-based heap: the JVM sizes the heap from the container limit, and you leave headroom for everything else.

My recommendation

Set a memory limit, let the JVM size its heap from it with a percentage, and leave real headroom for native memory. Bound every queue so overload becomes a fast, explicit refusal instead of a slow death. Make the out-of-memory case easy to recognise.

Worked example

The Compose file sets the limit and tells the JVM to use 70 percent of it for the heap. The rest is headroom.

services:
  api:
    image: demo/api:1.4.2
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      JAVA_TOOL_OPTIONS: "-XX:MaxRAMPercentage=70 -XX:+ExitOnOutOfMemoryError"
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M

A tiny program shows what the JVM actually sees inside the container, which is the first thing to check when sizing goes wrong:

public class WhatDoIHave {
    public static void main(String[] args) {
        Runtime rt = Runtime.getRuntime();
        System.out.printf("processors=%d maxHeap=%dMB%n",
                rt.availableProcessors(), rt.maxMemory() / (1024 * 1024));
    }
}

For back-pressure, use a bounded queue and refuse work when it is full, instead of accepting everything and falling over later:

import java.util.concurrent.*;

public class BoundedWorkers {
    private final ThreadPoolExecutor pool = new ThreadPoolExecutor(
            4, 4, 0L, TimeUnit.MILLISECONDS,
            new ArrayBlockingQueue<>(100),
            new ThreadPoolExecutor.AbortPolicy());

    /** Returns an HTTP-style status: 202 accepted, 503 over capacity. */
    public int submit(Runnable task) {
        try {
            pool.execute(task);
            return 202;
        } catch (RejectedExecutionException e) {
            return 503;
        }
    }
}

When a container disappears with exit code 137, check whether the kernel killed it:

docker inspect --format '{{.State.OOMKilled}} {{.State.ExitCode}}' api
docker stats --no-stream api

Checklist

1. Set memory and CPU limits, and write down why each number was chosen.
2. Use a heap percentage, and leave headroom for native memory.
3. Bound every queue, and return a clear over-capacity response when it is full.
4. Log queue depth, rejected work and memory use so the problem is visible before the kill.
5. Load test with the real limits applied, and watch what the upstream tier does.

Where this doesn't apply

These numbers are starting points for a small service. Large heaps, many threads or heavy direct-buffer use all change the headroom you need, and only measurement on your own workload settles it.

One honest limitation

Limits shrink the blast radius but do not prevent incidents. A service that is out of memory still fails; the point is that it fails predictably and recovers quickly.

Sources

• Docker Docs: Resource constraints
• Oracle Java Documentation: Java in containers (JVM ergonomics)
• Docker Docs: docker container inspect

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonInclude;

import java.util.List;
import java.util.Objects;
import java.util.Optional;

@JsonInclude(JsonInclude.Include.NON_ABSENT)
@JsonIgnoreProperties(ignoreUnknown = true)
public record Profile(String id, Optional<String> headline, List<String> skills) {

    public Profile {
        Objects.requireNonNull(id, "id");
        headline = headline == null ? Optional.empty() : headline;
        skills = skills == null ? List.of() : List.copyOf(skills);
    }
}

• The compact constructor makes id required and normalizes missing optional data, so no code downstream handles null for headline or skills.
• Optional<String> marks headline as optional in the type itself. (Records with Optional need Jackson's jackson-datatype-jdk8 module registered.)
• NON_ABSENT leaves an absent optional out of the JSON instead of writing an explicit null.
• ignoreUnknown = true lets this reader tolerate fields that newer producers add.

ObjectMapper mapper = new ObjectMapper()
        .registerModule(new Jdk8Module());

Why not plain POJOs?

The silent coercion trap

Profile p = mapper.readValue("{\"id\": 42}", Profile.class);
// p.id() is "42", with no error

import com.fasterxml.jackson.databind.cfg.CoercionAction;
import com.fasterxml.jackson.databind.cfg.CoercionInputShape;
import com.fasterxml.jackson.databind.type.LogicalType;

ObjectMapper strict = new ObjectMapper().registerModule(new Jdk8Module());
strict.coercionConfigFor(LogicalType.Textual)
      .setCoercion(CoercionInputShape.Integer, CoercionAction.Fail)
      .setCoercion(CoercionInputShape.Float, CoercionAction.Fail);

Null, absent, and present

Evolving schemas safely

• Add new fields as optional, with a defined default.
• Never change a field's type in place. Add a new field and deprecate the old one.
• Never reuse a removed field's name for different data.
• Treat enum changes with care. Configure an unknown-value fallback (for example READ_UNKNOWN_ENUM_VALUES_USING_DEFAULT_VALUE with a default) so old clients survive new symbols.

Property-based round-trip tests

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module;
import net.jqwik.api.ForAll;
import net.jqwik.api.Property;

import java.util.List;
import java.util.Optional;

import static org.junit.jupiter.api.Assertions.assertEquals;

class ProfileRoundTripTest {

    private final ObjectMapper mapper = new ObjectMapper().registerModule(new Jdk8Module());

    @Property
    void roundTripPreservesData(@ForAll String id,
                                @ForAll Optional<String> headline,
                                @ForAll List<String> skills) throws Exception {
        Profile original = new Profile(id, headline, skills);

        String json = mapper.writeValueAsString(original);
        Profile parsed = mapper.readValue(json, Profile.class);

        assertEquals(original, parsed);
    }
}

Failure modes

• Silent type coercion hiding a producer's breaking change.
• Explicit null written by one serializer and omitted by another.
• Enums that fail on unknown values in old clients.
• A required field added without a default, breaking old documents.

Demo to build

Takeaways

• Put invariants and null normalization in the record's compact constructor.
• Turn off silent coercion where the contract is strict.
• Evolve schemas additively, and keep golden files from every release.
• Property-based round-trip tests guard the serialization invariants.

Teams over-assign roles because it is faster. One person ends up able to create projects, deploy models and publish agents because it was easier to grant a broad role once than to think about it. Foundry has built-in roles that separate these tasks, and using them well is the cheapest security improvement available. One thing to know up front: the roles were renamed. What used to be called Azure AI User is now Foundry User, and Azure AI Project Manager is now Foundry Project Manager. If your infrastructure code still uses the old names, update it.

Options and trade-offs

• Foundry User: the least-privilege developer role. It gives reader access to the project and resource plus the data actions needed to build and test agents. It does not allow deploying or managing models.
• Foundry Project Manager: everything Foundry User can do, plus creating and managing projects, publishing agents, and assigning the Foundry User role to others. It does not include model deployment.
• Foundry Account Owner and Foundry Owner: manage the account and projects, including deploying models. These are broad. Use them sparingly and for platform owners only.
• Agent consumers: callers who only need to invoke a published agent should get access to the agent endpoint, not a build or admin role.

My recommendation

Give developers Foundry User and nothing else on the resource. Give team leads Foundry Project Manager so they can create projects and invite developers without waiting on a platform team. Reserve Foundry Account Owner for the few people who actually deploy models. Review assignments on a schedule, because over-privilege creeps back.

Minimal working example

Assign roles by name at the narrowest scope that works:

SUB=<subscription-id>
RG=<resource-group>
ACCOUNT=<foundry-resource-name>
SCOPE="/subscriptions/$SUB/resourceGroups/$RG/providers/Microsoft.CognitiveServices/accounts/$ACCOUNT"

# Developers: least privilege
az role assignment create \
  --assignee <developer-object-id> \
  --role "Foundry User" \
  --scope "$SCOPE"

# Team lead: can create projects, publish agents, invite developers
az role assignment create \
  --assignee <lead-object-id> \
  --role "Foundry Project Manager" \
  --scope "$SCOPE"

# Platform owner who deploys models
az role assignment create \
  --assignee <platform-owner-object-id> \
  --role "Foundry Account Owner" \
  --scope "$SCOPE"

Audit who has what:

az role assignment list --scope "$SCOPE" \
  --query "[].{principal:principalName, role:roleDefinitionName}" -o table

Test it: sign in as a Foundry User and confirm you can build and run an agent but cannot deploy a model. Sign in as a Project Manager and confirm you can publish an agent but still cannot deploy a model.

Checklist

• Start every developer at Foundry User on the resource, and elevate only for a named task.
• Check the permission tables for "manage models" and deployment before granting Account Owner or Owner.
• Remember that Project Manager can assign only the Foundry User role, which makes delegated onboarding safe.
• Give service principals and end users that only invoke agents the narrowest role that allows it.
• Update scripts and templates that still reference the old Azure AI role names.
• Review role assignments at a regular interval, and remove access when people change teams.

Where this doesn't apply

Older hub-based (classic) projects have a different role surface. If you run both, map roles carefully and do not assume names carry across.

Sources

• Role-based access control for Microsoft Foundry
• Elevated-role tasks in Microsoft Foundry
• Azure built-in roles for AI + machine learning

---

GenAI agents must resist prompt injection, avoid harmful outputs, and maintain provenance. Azure AI Content Safety offers moderation for text/images and safety features (Prompt Shields, groundedness detection, protected material) with configurable thresholds and blocklists.

Options and trade-offs

• Prompt Shields (direct and indirect injection): Detect jailbreak attempts and document-borne attacks; low false-positive rates; add latency; configurable block vs annotate modes.
• Groundedness detection (Preview): Detect ungrounded completions relative to sources; helps provenance and QA; preview scope limits.
• Protected material detection (text GA, code preview): Identifies copyrighted segments in completions; supports compliance; English scope.
• Blocklists and custom categories: Enforce organization-specific terms and custom detection; stored encrypted at rest with CMK options.

Minimal working example

Text moderation (Python) Requirements: requests or SDK; endpoint; subscription key or Entra auth.

import requests, json
endpoint = "https://YOUR-CONTENT-SAFETY.cognitiveservices.azure.com"
headers = {"Ocp-Apim-Subscription-Key": "KEY","Content-Type":"application/json"}
payload = {"text": "Sample text", "blocklistNames": []}
r = requests.post(f"{endpoint}/contentsafety/text:analyze?api-version=2024-09-01",
                  headers=headers, data=json.dumps(payload))
print(r.json())

Test: Inspect categoriesAnalysis severities; adjust thresholds; add blocklists.

Checklist

• Configure separate input/output filters; turn on Prompt Shields for inputs; add groundedness and protected material for outputs.
• Confirm API rate limits and payload size limits per modality/tier.
• Use blocklists for brand/PII terms; understand storage and deletion controls.

Where this doesn't apply

Offline moderation pipelines with proprietary classifiers; integrate but consider Content Safety for complementary coverage.

Sources

• What is Azure AI Content Safety? (GA)
• Documentation hub (GA)
• Data, privacy, and security (GA)

---

In aceu-edge-infer, I keep a small harness that compares FP32 and INT8 inference and records latency and output deltas. The sample inputs are synthetic, so its results are method scaffolding—not evidence of production performance. The measurement discipline is what I want the demo to teach.

Decide what question the benchmark answers

Are you measuring cold start, steady-state single-request latency, or throughput under concurrent work? Those are different measurements. State the workload, model artifact, runtime and device, input shape, batch size, thread/concurrency settings, and whether preprocessing and post-processing are included.

Warm up—and report cold start separately

The first call may include model compilation, memory allocation, cache fill, or runtime initialization. If the question is steady-state latency, run a planned warm-up and keep those samples out of the measured set. If cold start matters to users, report it as its own metric instead of mixing it into steady-state numbers.

Report percentiles as well as averages

Users feel slow requests, not just the mean. Report p50, p95, and p99 latency for each model variant, along with the sample count and run duration. Tail percentiles need enough samples to be meaningful; do not imply precision the benchmark cannot support.

Compare the same inputs and check outputs

Run the same representative inputs through FP32 and INT8. Compare both the score deltas and task-level decisions. For classification, inspect per-class changes and examples near decision boundaries; a low average absolute difference can still hide an important decision flip.

Here is a small summarizer for arrays collected by a benchmark harness. It assumes both latency arrays contain milliseconds and each output array corresponds to the same input order, with one row per request and one column per model output:

import numpy as np


def latency_summary(samples_ms):
    values = np.asarray(samples_ms, dtype=np.float64)
    return {
        "p50_ms": float(np.percentile(values, 50)),
        "p95_ms": float(np.percentile(values, 95)),
        "p99_ms": float(np.percentile(values, 99)),
    }


def parity_summary(outputs_fp32, outputs_int8):
    fp32 = np.asarray(outputs_fp32, dtype=np.float64)
    int8 = np.asarray(outputs_int8, dtype=np.float64)
    per_request_delta = np.mean(np.abs(fp32 - int8), axis=-1)
    return {
        "mean_absolute_delta": float(np.mean(per_request_delta)),
        "p95_absolute_delta": float(np.percentile(per_request_delta, 95)),
    }

These summaries do not replace task-specific evaluation. For example, also record whether top-class predictions or other decision thresholds changed.

Control and record variance

Use the same machine and runtime configuration for each comparison. Note background load, CPU power state, affinity or isolation settings if you use them, and any other workload sharing the device. Repeat runs and compare run-to-run variation. If the host cannot be isolated, state that plainly.

Use a representative validation set for accuracy and parity. Synthetic inputs are valuable for exercising a harness or pipeline, but they cannot establish how a model behaves on the real input distribution.

Keep calibration and benchmark series distinct

Calibration data influences the quantization parameters. Record its source and important characteristics, and keep the evaluation set separate. If calibration changes, treat the quantized model as a new artifact and rerun the full comparison rather than blending results from different model versions.

Make precision and execution path observable

If some requests fall back from INT8 to FP32, label them. Do not combine them into one “INT8” result. Emit structured output—JSON works well—with model hashes, precision, device, warm-up configuration, sample count, latency distribution, output deltas, and run metadata.

Trade-offs and failure modes

• Simple harness vs. realistic load: a single-request loop is easy to understand, but it does not characterize concurrency or resource contention.
• Synthetic inputs vs. representative data: synthetic inputs make a repeatable demo; production claims need appropriate evaluation inputs.
• Tail reporting vs. small samples: percentiles reveal latency spread, but only when there are enough measurements to support them.
• Mixed precision vs. clean comparison: unreported fallbacks make speed and parity comparisons untrustworthy.

Try it yourself

1. Write down whether you are testing cold start, latency, throughput, or a combination.
2. Warm up deliberately and record the sample count and test conditions.
3. Report FP32 and INT8 percentiles alongside output-delta and decision-change summaries.
4. Repeat on an idle host and under representative load; keep those results separate.

Closing thought

Benchmarking honestly is a habit. Warm up, report the distribution, compare outputs, and describe the machine and workload. Then “it’s faster” becomes a claim another engineer can reproduce and assess.

A small open-source project may be maintained with limited time, so contributor experience matters. A tested quickstart, a narrow first change, and clear expectations make it easier for someone to turn an idea into a useful pull request.

The failure mode

A new contributor lands on your repo, reads a long CONTRIBUTING.md, and tries to set up the environment. They get halfway through a failing install, lose an hour chasing Python versions, and give up. Their idea never materializes, and you lose a potential collaborator. The contributions that do arrive are sporadic and often misaligned with your project norms.

What I learned

A “quickstart” that is truly quick

The first section of CONTRIBUTING.md should be a script that succeeds on a fresh machine in under 10 minutes. No optional flourishes; no “you can also…” — one path, tested regularly. In a Python project, that means creating an isolated environment, installing exact dependencies, running tests, and starting the server. If you support multiple OSes, state the best-supported environment up front and link alternatives after the main path.

Example quickstart

# clone + setup
git clone https://github.com/yourname/your-project && cd your-project
python3 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
# run tests
pytest -q
# start service
uvicorn app.main:app --host 0.0.0.0 --port 8000

A first PR path that doesn’t require knowing the whole codebase

If your project supports plugins, design the first contribution as “add a simple plugin” — a single file with a single function, plus a minimal test. Scaffold the directory structure in the quickstart and provide a template file with TODOs. Make it obvious what a valid plugin looks like and how it integrates.

Minimal plugin template

# your_project/plugins/example_plugin.py
from your_project.core import EvidenceSource, EvidenceResult
class ExamplePlugin(EvidenceSource):
    name = "example_plugin"
def evaluate(self, profile: dict) -> EvidenceResult:
    # TODO: basic rule: if profile has 'example' flag, return a small boost
if profile.get("example", False):
    return EvidenceResult.present(score_delta=0.05, note="example flag")
    return EvidenceResult.missing(reason="no example flag")

Tests are not optional — make them the easiest path

Require a test in your PR template, but help contributors succeed by including a ready-made test template that imports the plugin and checks a tiny scenario. Fewer than 20 lines is perfect, and the test should run fast.

PR checklist and matching test

### Checklist
- [ ] Added a test covering the plugin behavior
- [ ] Ran `pytest -q` locally
- [ ] Updated plugin docs (one paragraph + example)
Test template:
# tests/test_example_plugin.py
from your_project.plugins.example_plugin import ExamplePlugin
from your_project.core import EvidenceResult
def test_example_plugin_with_flag():
plugin = ExamplePlugin()
result = plugin.evaluate({"example": True})
assert isinstance(result, EvidenceResult)
assert result.type == "present"
assert result.note == "example flag"

Labeling matters more than exhortations

A “good first issue” label is a real signal. Curate it weekly: choose small, well-bounded tasks with a single file change and a single test. Avoid issues that depend on strategic decisions or require deep context. If you don’t have time to curate, do fewer but better — five solid good-first-issues beat twenty vague ones.

“How to ask for help” — define the format, not the channel

New contributors often ask for help with missing context. Provide a brief template for help requests: environment, exact command, full error output, and what they already tried. Keep it short and clear. This reduces back-and-forth and helps you respond faster.

- OS and Python version:
- Exact command:
- Full error output (paste as a code block):
- What I’ve tried so far:

The social contract: response-time expectations

State expected response windows for maintainers. If you maintain the project in spare time, say so. Set realistic targets for issue triage and pull request reviews, then keep them visible. Set expectations, and you’ll reduce anxiety on both sides.

CI that reduces guesswork

Your CI should run tests and sanity checks on every PR with clear messages. If a plugin interface requires a test, fail with a helpful note when none is detected. If the project uses a formatting or linting standard, automate the fix script and link it from the failure output.

Trade-offs and failure modes

Trade-offs

Prescriptive quickstart vs flexibility: You’ll help more contributors by picking one supported path; you’ll disappoint a few who prefer different tools. Document alternatives briefly but keep the main path simple.

Template rigor vs contributor creativity: Templates guide good behavior but can feel constraining. Invite advanced contributions after the first PR; keep the first path narrow and friendly.

CI strictness vs PR throughput: Strict checks reduce maintainers’ manual work but increase initial friction. Counterbalance with crystal-clear failure messages and one-line fix commands.

Failure modes

“Wall of text”: Long CONTRIBUTING.md that reads more like a policy than a path. Solution: front-load the quickstart and the first PR template; move policies to the bottom.

“Passes locally, fails in CI”: Version and environment differences. Solution: pin dependencies, provide a lock or requirements file, and document the supported Python version.

“Invisible expectations”: Unstated response windows or required tests. Solution: PR templates and CI checks that make expectations explicit.

Try it yourself

Rewrite the top of your CONTRIBUTING.md into a tested quickstart that succeeds in under 10 minutes.

Add a one-file plugin template and a matching test template. Wire CI to require a test for new plugins.

Curate five “good first issue” tickets with single-file changes and clear acceptance criteria. Measure issue and review response times, then adjust your maintainer schedule accordingly.

A database password in an environment variable or a config file is a leak waiting for a screenshot, a log line or a copied image. The harder question is operational: who creates the secret, how does the app read it, and what happens when it changes?

I wanted one flow that answers all three, so rotation is a normal event instead of a project.

Options and trade-offs

• Secrets in files or environment variables: fast to start, and prone to leaks and drift.
• A non-OCI secret store: a good option if you already run one, with extra integration and policy work.
• OCI Vault secrets: IAM-governed, versioned, with rotation options. It needs deliberate policies and a read path.

My recommendation

Create the secret in a vault, with a master encryption key in the same vault, and let the application read the current version at runtime using an instance or resource principal. Rotate sensitive credentials on a schedule and watch the audit log for reads.

Worked example

Reading a secret from Java with an instance principal. Secret content comes back base64 encoded, so decode it and keep it in memory.

var provider = InstancePrincipalsAuthenticationDetailsProvider.builder().build();
try (var secrets = SecretsClient.builder().build(provider)) {
    var req = GetSecretBundleRequest.builder()
            .secretId(System.getenv("DB_SECRET_OCID"))
            .stage(GetSecretBundleRequest.Stage.Current)
            .build();
    var content = (Base64SecretBundleContentDetails)
            secrets.getSecretBundle(req).getSecretBundle().getSecretBundleContent();
    byte[] password = java.util.Base64.getDecoder().decode(content.getContent());
    // use the bytes, then discard them; do not log them
}

The policy that lets only the runtime read that one secret:

Allow dynamic-group app-runtimes to read secret-bundles in compartment apps
  where target.secret.id = '<db_secret_ocid>'

Rotation can be configured on the secret with an interval, or driven through a function that updates the target system, such as an Autonomous Database user, and then writes a new secret version. Because every version has a stage such as current or previous, you can roll back by pointing readers at the earlier version while you investigate.

Checklist

1. Create the secret with a key in the same vault.
2. Authorize only the runtime principal to read secret bundles.
3. Decode at runtime, hold the value in memory and never log it.
4. Turn on rotation for credentials that matter.
5. Alert on unusual GetSecretBundle activity in the audit log.
6. Write down how to restore the previous version if a rotation fails.

Where this doesn't apply

One-time tokens that never need to be read again do not need a vault entry. Very large blobs may also exceed the secret size limit, so check the current limit before choosing Vault for them.

One honest limitation

Rotation and cross-region replication add moving parts. I rehearse a rotation in a test environment before the first production run.

Sources

• OCI Documentation: Vault, Creating a Secret
• OCI Documentation: Updating a Secret and Secret Rotation
• OCI Documentation: Retrieving a Secret Bundle (Java SDK)

The aceu-trust-demo uses a multi-stage build, a slim base, a non-root user, a health endpoint, and Compose for local runs. The checklist below is designed to make those decisions repeatable.

1. Keep build tools out of the runtime stage

Compilers, headers, and test tools belong in a build stage when the running application does not need them. Copy only the installed runtime dependencies and application code into the final stage. A smaller image can reduce transfer time and remove packages the service does not need.

# syntax=docker/dockerfile:1
FROM python:3.12-slim AS builder
WORKDIR /app
RUN python -m venv /venv
ENV PATH="/venv/bin:$PATH"
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY aceu ./aceu

FROM python:3.12-slim AS runtime
RUN useradd --uid 10001 --create-home appuser
WORKDIR /app
COPY --from=builder --chown=10001:10001 /venv /venv
COPY --from=builder --chown=10001:10001 /app/aceu ./aceu
ENV PATH="/venv/bin:$PATH" \
    PYTHONDONTWRITEBYTECODE=1
USER 10001
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD python -c 'import urllib.request; urllib.request.urlopen("http://localhost:8000/health", timeout=2)' || exit 1
CMD ["uvicorn", "aceu.app:app", "--host", "0.0.0.0", "--port", "8000"]

Check the actual base image and application requirements before using this verbatim. For example, a health endpoint must exist and return a failing status when the process is unhealthy.

2. Choose a base image for the runtime

A minimal base can reduce unnecessary packages, but it may omit a system library your app needs. Add required runtime packages explicitly, then measure the result. Pick an image the team can patch and operate; “smallest possible” is not the only goal.

3. Run as a non-root user with a known UID

Use an explicit user ID and make sure files copied into the image are readable by that user. If a writeable directory is needed, grant access only to that path. Fix ownership in the Dockerfile instead of switching back to root when a permissions error appears.

4. Keep the build context narrow

Docker can only build from the context it receives. Exclude local caches, virtual environments, repository history, and secrets with a .dockerignore file:

__pycache__/
*.py[cod]
.env
.git/
node_modules/
venv/
.venv/
dist/
build/

Review COPY statements as carefully as ignore rules. Do not copy test fixtures or private configuration into the runtime image unless the application needs them.

5. Keep secrets out of image layers

Do not place credentials in Dockerfile ARG or ENV instructions, and do not copy secret files into a layer. Prefer platform-managed runtime secrets. When a build genuinely needs a secret, use BuildKit’s secret mount rather than baking it into the image.

6. Check for weakened isolation and noisy health checks

Review Dockerfiles and deployment manifests for settings that weaken isolation, such as an unconfined seccomp profile. Keep health checks bounded and inexpensive; an overly aggressive check can add load or mark a service unhealthy during a temporary resource spike. Set log levels intentionally so normal operation does not bury useful signals.

Trade-offs and failure modes

• Slim image vs. hidden dependencies: add the libraries you need, then verify the runtime stage starts and serves its critical paths.
• Non-root vs. file permissions: set ownership during build and grant write access only where needed.
• Health checks vs. extra load: probe a fast path and use a reasonable timeout and interval.
• Runtime secrets vs. rotation work: secure injection adds operational responsibilities, so document ownership and rotation clearly.

Common surprises include a credential accidentally left in an image layer, a slim runtime missing a required shared library, or a non-root process unable to read a root-owned file. A deliberate image review catches these before deployment.

Try it on your next image

1. Separate build and runtime stages, then compare the resulting image size.
2. Run the application under a non-root UID and verify file access.
3. Add a .dockerignore file and inspect the build context.
4. Add a CI lint step for known unsafe Dockerfile or manifest settings.

Closing thought

A smaller, quieter, safer image is easier to move and less likely to surprise the team in production. The demo remains illustrative; the container practices are the reusable part.

• Untyped: a raw JSON tree (JsonNode), whatever shape the producer chose.
• Typed: a Java class or record bound to a fixed schema.

Principle: translate the status, preserve the details

1. Map a Throwable to an HTTP status and message. This is policy, and it belongs in one place.
2. Carry the error details through untouched. If the producer's contract is untyped, leave it untyped.

A small translator

import com.fasterxml.jackson.databind.JsonNode;

import java.util.NoSuchElementException;
import java.util.concurrent.TimeoutException;

public final class ApiException extends RuntimeException {
    private final int status;
    private final String code;
    private final JsonNode details;          // untyped on purpose

    public ApiException(int status, String message, String code, JsonNode details, Throwable cause) {
        super(message, cause);
        this.status = status;
        this.code = code;
        this.details = details;
    }

    public int status() { return status; }
    public String code() { return code; }
    public JsonNode details() { return details; }

    public static ApiException translate(Throwable t) {
        if (t instanceof ApiException existing) {
            return existing;                                   // never re-wrap
        }
        if (t instanceof IllegalArgumentException) {
            return new ApiException(400, t.getMessage(), "BAD_REQUEST", null, t);
        }
        if (t instanceof NoSuchElementException) {
            return new ApiException(404, t.getMessage(), "NOT_FOUND", null, t);
        }
        if (t instanceof TimeoutException) {
            return new ApiException(504, "Upstream timeout", "TIMEOUT", null, t);
        }
        return new ApiException(500, "Internal error", "INTERNAL", null, t);
    }
}

• Return an existing ApiException unchanged. Wrapping it again discards its status and details.
• Hide internals for 500s. The message is generic; the cause is kept for server logs.
• Keep the mapping table small and tested. Every exception type you map is a promise to clients.

One place to build the response

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;
import jakarta.ws.rs.ext.ExceptionMapper;
import jakarta.ws.rs.ext.Provider;

@Provider
public class ApiExceptionMapper implements ExceptionMapper<Throwable> {

    private static final ObjectMapper MAPPER = new ObjectMapper();

    @Override
    public Response toResponse(Throwable t) {
        ApiException e = ApiException.translate(t);
        return Response.status(e.status())
                .type(MediaType.APPLICATION_JSON)
                .entity(body(e).toString())
                .build();
    }

    static ObjectNode body(ApiException e) {
        ObjectNode node = MAPPER.createObjectNode();
        node.put("status", e.status());
        node.put("message", e.getMessage() == null ? "" : e.getMessage());
        node.put("code", e.code());
        if (e.details() != null) {
            node.set("errorDetails", e.details());             // passed through as-is
        }
        return node;
    }
}

Batch operations need extra care

public static <K> Map<K, ApiException> errorsByKey(Map<K, Throwable> failures) {
    Map<K, ApiException> result = new LinkedHashMap<>();
    failures.forEach((key, t) -> result.put(key, ApiException.translate(t)));
    return result;
}

Prove the wire does not change

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.*;

class ErrorDetailsWireTest {

    private final ObjectMapper mapper = new ObjectMapper();

    record ErrorInfo(String field, String reason) { }

    @Test
    void untypedDetailsSurviveUnchanged() throws Exception {
        JsonNode details = mapper.readTree(
                "{\"field\":\"email\",\"reason\":\"format\",\"custom\":\"kept\"}");

        ApiException e = new ApiException(422, "invalid", "VALIDATION", details, null);
        JsonNode body = mapper.readTree(ApiExceptionMapper.body(e).toString());

        assertEquals(details, body.get("errorDetails"));
    }

    @Test
    void typedConversionLosesUnknownFields() throws Exception {
        JsonNode details = mapper.readTree(
                "{\"field\":\"email\",\"reason\":\"format\",\"custom\":\"kept\"}");

        ObjectMapper lenient = new ObjectMapper()
                .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        JsonNode roundTripped = lenient.valueToTree(lenient.convertValue(details, ErrorInfo.class));

        assertNotEquals(details, roundTripped);               // "custom" is gone
    }
}

Status codes are contract too

Failure modes

• Typed envelopes rejecting or dropping unknown fields.
• Double-wrapping an exception that already has a status.
• Whole-batch failure when only one key failed.
• Leaking stack traces or internal messages in 500 responses.

Demo to build

Takeaways

• Separate "which status" from "which details".
• Do not convert untyped details to typed ones unless you are deliberately versioning the contract.
• Add a wire-level test for error responses.

In aceu-edge-infer, I convert a lightweight classifier from ONNX for use with OpenVINO. One custom activation exposed an operator gap and required a conservative CPU fallback. The demo’s ACE-U output is illustrative, not an evaluation of real people; the model-delivery practices here are the focus.

Check the graph before you convert

Inspect the ONNX graph, its opset, input names, and shapes. Look for custom operators and operations that may not be supported by the runtime version or target device. If the model has dynamic dimensions, decide which shapes production must support and test those explicitly.

OpenVINO can convert supported ONNX models to IR, and it can also read ONNX directly at runtime. Conversion is useful when you want a prepared artifact and predictable model-load behavior; first confirm the format is supported and that conversion solves a real deployment need.

Use the current conversion API

For a supported ONNX model, the OpenVINO Python API can convert and save an IR artifact:

import openvino as ov
from pathlib import Path

model = ov.convert_model("models/classifier.onnx")
output = Path("build/ir_fp32/classifier.xml")
output.parent.mkdir(parents=True, exist_ok=True)
ov.save_model(model, output)

The saved IR consists of an XML graph and a weights file. Alternatively, the ovc command-line tool can convert supported input models. Keep the exact command or script, OpenVINO version, input shapes, and conversion logs with the build so another engineer can reproduce it.

Read conversion errors as signals

• Unsupported or partially supported operators: check the runtime’s supported operations and determine whether a supported equivalent preserves behavior.
• Shape inference failures: inspect dynamic dimensions and provide explicit input shapes only when they match the model’s intended use.
• Precision-specific differences: validate the FP32 conversion first; quantization adds a separate source of numerical change.
• Custom layers: decide early whether to replace, isolate, or route around the operation.

Choose a fallback deliberately

If an operator cannot use the preferred execution path, there are several options:

• Replace it with a supported equivalent, then measure output parity and confirm the change fits the model’s accuracy budget.
• Keep a documented CPU implementation for the unsupported portion when the runtime and architecture support that boundary.
• Split a larger graph into a supported model and a small, explicit post-processing step in application code.

Do not label a fallback as “optimized” without measurement. Surface the selected runtime path in logs or response metadata, and make clear whether the service can continue safely when the preferred path is unavailable.

Validate outputs before optimizing precision

Create a golden evaluation set and compare the source ONNX model with the converted FP32 model first. Define acceptance criteria that fit the task—for example, maximum per-class delta or mean absolute error. If outputs differ, compare intermediate tensors where possible to locate the first divergence.

Only after conversion parity is understood should you evaluate INT8. This keeps operator-conversion differences separate from quantization differences and makes failures much easier to diagnose.

Make conversion part of the build

Script input shapes, conversion flags, substitutions, and output paths. Version IR artifacts with the model and code revision that produced them; record file hashes and conversion logs rather than overwriting a vague latest artifact. Run conversion in CI when the model, conversion script, or runtime version changes.

Trade-offs and failure modes

• Operator replacement vs. behavioral fidelity: an equivalent-looking function can change output; verify parity against the task’s acceptance criteria.
• CPU fallback vs. uniform execution: a fallback can preserve service behavior but adds a path to test and monitor.
• Fast manual fix vs. reproducibility: one-off flags may unblock a conversion, while a scripted fix makes future builds repeatable.

A model that converts and runs can still produce different outputs. A fallback that is not disclosed can make performance results misleading. A runtime update can change what converts successfully, so keep conversion inside the change-review process.

Try it yourself

1. Inventory operators, opset, shapes, and custom layers in your ONNX file.
2. Convert to IR with a scripted, versioned command and retain its logs.
3. Compare outputs on a golden set before you introduce quantization.
4. Document and test every fallback path, including how the service reports it.

Closing thought

The conversion gap is manageable when operator compatibility, model parity, and fallbacks are treated as product requirements. Script the path, version what it creates, and validate the outputs before optimizing further.

I develop on an arm64 laptop, and a lot of the places I deploy to are x86. If I push an image built only for my laptop, the failure shows up later and far from the cause, usually as an unhelpful exec format error on a server. Docker can build for several platforms in one command, but there are a few traps: emulation is slow and sometimes subtly wrong, vendor binaries downloaded during a build may be for the wrong CPU, and a manifest that is missing a platform fails on pull.

Options and trade-offs

• Build only for the laptop architecture: quick, but it breaks the first time the image runs elsewhere.
• Build every platform under emulation: one command, but slow, and it only proves the code starts, not that it behaves.
• Build natively per architecture, or avoid per-architecture work entirely: fastest and most faithful, but it needs either runners for each architecture or a build that does not depend on the CPU.

My recommendation

Be explicit about platforms and publish a manifest list. For JVM services, I go one step further: compile the jar once on the build platform, because bytecode is architecture independent, and only the runtime image differs per platform. That removes emulation from the expensive step.

Worked example

The build stage is pinned to the platform doing the build, so it runs natively. Only the small runtime stage is produced per target architecture.

# syntax=docker/dockerfile:1
FROM --platform=$BUILDPLATFORM eclipse-temurin:21-jdk AS build
WORKDIR /src
COPY . .
RUN ./gradlew --no-daemon installDist -x test

FROM eclipse-temurin:21-jre
ARG TARGETARCH
WORKDIR /opt/app
COPY --from=build /src/build/install/app/ ./
RUN echo "runtime image for ${TARGETARCH}"
ENTRYPOINT ["/opt/app/bin/app"]

docker buildx create --name multiarch --use
docker buildx inspect --bootstrap

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t registry.example.com/demo/app:1.4.2 \
  --push .

docker buildx imagetools inspect registry.example.com/demo/app:1.4.2

A tiny smoke test confirms which architecture you are actually running on:

public class Arch {
    public static void main(String[] args) {
        System.out.println(System.getProperty("os.name") + " / " + System.getProperty("os.arch"));
    }
}

docker run --rm --platform linux/amd64 registry.example.com/demo/app:1.4.2
docker run --rm --platform linux/arm64 registry.example.com/demo/app:1.4.2

Checklist

1. State --platform explicitly, every time.
2. Run the compile step on the build platform when the output is architecture independent.
3. Check any downloaded binary against the target architecture.
4. Inspect the pushed manifest list and confirm every platform you need is present.
5. Run real tests on native hardware for each platform you ship.

Where this doesn't apply

If you ship native code (JNI libraries, native-image binaries, or CPU-specific agents), the compile step is no longer architecture independent, and you need native builds for each target.

One honest limitation

Emulation is useful for a smoke test and not much else. It can hide timing and system-call differences, and a JIT-heavy workload will behave differently from the real thing.

Sources

• Docker Docs: Multi-platform builds
• Docker Docs: buildx imagetools inspect
• Docker Docs: Multi-stage builds (BUILDPLATFORM and TARGETARCH)

Microsoft Graph change notifications tell your system when something changes, for example a user is created, updated or deleted. The default delivery method is a webhook, and webhooks are brittle at scale. Endpoints time out, retries pile up, cold starts add latency, and a missed lifecycle event means a subscription quietly expires and you stop hearing about changes. Event Grid partner topics replace the webhook with managed delivery, retries, dead-lettering and fan-out to serverless handlers.

Options and trade-offs

• Direct webhook to a Function or web app: the simplest to build. You own scaling, retry behavior and response-time limits, and a slow validation response can lose the subscription.
• Event Grid partner topic: Graph delivers to a topic in your Azure subscription, and you attach event subscriptions to route the events. There is more setup, but delivery, retry and dead-lettering are handled for you.
• Event Hubs: the right choice for very high-volume streams where you want partitioned, replayable consumption rather than per-event routing.

My recommendation

Use an Event Grid partner topic for anything that must not lose events: identity lifecycle changes that drive access removal or deprovisioning. Subscribe to lifecycle notifications as well as change notifications, and treat renewal as part of the design, not an afterthought.

Minimal working example

Step 1: create the Graph subscription. The notification URL points at Event Grid instead of a web address:

POST https://graph.microsoft.com/v1.0/subscriptions
Content-Type: application/json
Authorization: Bearer <token>

{
  "changeType": "created,updated,deleted",
  "notificationUrl": "EventGrid:?azuresubscriptionid=<subscription-id>&resourcegroup=<resource-group>&partnertopic=<topic-name>&location=<region>",
  "lifecycleNotificationUrl": "EventGrid:?azuresubscriptionid=<subscription-id>&resourcegroup=<resource-group>&partnertopic=<topic-name>&location=<region>",
  "resource": "users",
  "expirationDateTime": "2026-12-01T00:00:00Z",
  "clientState": "<random-secret-value>"
}

Step 2: activate the partner topic that Graph created in your resource group, then route it:

az eventgrid partner topic activate \
  --name <topic-name> \
  --resource-group <resource-group>

Then add an event subscription on the topic that sends events to an Azure Function (portal or CLI).

Step 3: handle events asynchronously. The function should do as little as possible before returning:

using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Logging;
using System.Text.Json;

public class OnGraphEvent
{
    private readonly ILogger<OnGraphEvent> _log;
    public OnGraphEvent(ILogger<OnGraphEvent> log) => _log = log;

    [Function("OnGraphEvent")]
    public void Run([EventGridTrigger] string eventJson)
    {
        using var doc = JsonDocument.Parse(eventJson);
        var type = doc.RootElement.GetProperty("type").GetString();

        if (type != null && type.Contains("lifecycleNotification", StringComparison.OrdinalIgnoreCase))
        {
            _log.LogInformation("Lifecycle event received, renew the subscription: {Type}", type);
            // enqueue a renewal job
            return;
        }

        _log.LogInformation("Change event: {Type}", type);
        // enqueue the change for processing, do not do slow work here
    }
}

Check the exact event schema and type names against your own partner topic before you rely on them, since they differ by resource and Graph version.

Test it: create a test user, confirm an event reaches the function, then delete the user and confirm the delete event arrives. Let a short-lived subscription run to expiry in a test tenant and confirm the lifecycle event fires.

Checklist

• Subscribe to lifecycle notifications as well as change notifications, and renew subscriptions when the lifecycle events say so.
• Check the maximum subscription lifetime for the resource type, and set renewal well before it.
• Put a clientState secret in the subscription and validate it on receipt where your handler can see it.
• Keep handlers fast and idempotent. Queue work and return, since events can be delivered more than once.
• Configure dead-lettering on the event subscription and alert when messages land there.
• Use encrypted rich notifications where the resource supports them, and manage the certificates as documented.

Where this doesn't apply

Extremely high-volume streams, where Event Hubs is a better fit. Also, resources that Graph does not support through Event Grid partner topics still need a webhook.

Sources

• Change notifications overview
• Microsoft Graph Event Grid sample (.NET)
• Microsoft Graph Event Grid sample (Java)

---

Dependencies change faster than a small team can review them. Left alone, Dependabot opens a pull request per package, reviewers stop looking, and updates either pile up unmerged or get approved on autopilot. Both outcomes are dangerous. The first leaves known vulnerabilities in place. The second lets a bad release in with a rubber stamp.

The risk is real and it is not limited to libraries. In March 2025, the widely used GitHub Action tj-actions/changed-files was compromised: attackers pointed existing version tags at malicious code that printed secrets into workflow logs, and CISA added the issue to its Known Exploited Vulnerabilities catalog. Workflows that referenced a tag, not a commit, picked up the malicious code without anyone changing a line. The lesson applies to every ecosystem: what you depend on, and how you pin it, is part of your attack surface.

Options and trade-offs

• Individual update pull requests: easy to review in isolation, and noisy enough that reviewers fatigue.
• Grouped updates: Dependabot batches related updates into one pull request. Fewer pull requests, slightly larger diffs, and better reviewer throughput.
• Auto-merge for low-risk updates: fast, and only safe where tests are good. Limit it to patch releases or development-only dependencies.
• Code owners on sensitive paths: guarantees someone accountable reviews changes to build files, workflows and security code. It adds latency where it matters.
• Security advisories: Dependabot alerts tell you what is vulnerable. Repository security advisories let you coordinate a fix privately before disclosure, which matters if you maintain a library others depend on.
• Pinning actions by tag versus by commit SHA: tags are readable and movable, and SHAs are immutable. Dependabot can keep SHA pins current.

My recommendation

Group routine minor and patch updates into one weekly pull request per ecosystem. Auto-merge only patch updates, and only when the required checks pass. Keep major versions and anything touching build or workflow files manual and owner-reviewed. Pin third-party GitHub Actions to full commit SHAs, and let Dependabot raise the update pull requests. Treat security alerts separately from routine updates, with their own faster path.

Minimal working example

The Dependabot configuration, covering a Gradle project and the workflows themselves:

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "gradle"
    directory: "/"
    schedule:
      interval: "weekly"
      day: "monday"
    open-pull-requests-limit: 5
    groups:
      minor-and-patch:
        update-types: ["minor", "patch"]
    labels: ["dependencies"]

  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "weekly"
    groups:
      actions:
        patterns: ["*"]
    labels: ["dependencies", "ci"]

Auto-merge for patch updates only, using Dependabot's own metadata action:

# .github/workflows/dependabot-automerge.yml
name: dependabot-automerge
on: pull_request

permissions:
  contents: write
  pull-requests: write

jobs:
  automerge:
    if: github.actor == 'dependabot[bot]'
    runs-on: ubuntu-latest
    steps:
      - id: meta
        uses: dependabot/fetch-metadata@v2
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

      - name: Enable auto-merge for patch updates
        if: steps.meta.outputs.update-type == 'version-update:semver-patch'
        run: gh pr merge --auto --squash "$PR_URL"
        env:
          PR_URL: ${{ github.event.pull_request.html_url }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Auto-merge waits for the required checks configured in your ruleset or branch protection, so make sure they exist. Without them, "auto-merge when checks pass" merges immediately.

Owners for the files that control your build and your supply chain:

# .github/CODEOWNERS
/.github/              @my-org/security
/build.gradle*         @my-org/maintainers
/gradle/               @my-org/maintainers
/.github/dependabot.yml @my-org/security

Pin an action by commit SHA, with a readable comment that Dependabot updates along with the SHA:

- uses: actions/checkout@<full-40-character-commit-sha>  # v4.x.x

Test it: let Dependabot open a grouped pull request, confirm a patch-only update merges after checks pass, confirm a major update waits for review, and confirm a change to the workflows folder asks for the security group.

Checklist

• Enable Dependabot version updates and alerts, and keep security updates on.
• Group minor and patch updates per ecosystem, and cap open pull requests.
• Restrict auto-merge to patch updates with passing required checks.
• Make sure required checks exist before enabling auto-merge.
• Add CODEOWNERS for build files, workflows and the Dependabot config.
• Pin third-party actions to full commit SHAs.
• Enable private vulnerability reporting if you maintain a public library, and use repository security advisories to coordinate fixes.
• Write the triage rule down: who looks at a security alert, within what time, and what counts as done.

Where this doesn't apply

Projects with frozen or vendored dependencies see few updates and may not need grouping. Ecosystems governed by an external policy engine or an internal mirror may enforce these rules elsewhere.

One honest limitation

Grouping reduces noise, but it can also hide an interaction: one risky update buried in a batch of ten harmless ones. Keep majors and anything touching sensitive paths out of the groups, and read the lockfile diff when it matters.

Sources

• Dependabot options reference
• Automating Dependabot with GitHub Actions
• About code owners
• About repository security advisories
• CISA alert on the tj-actions/changed-files compromise

Turning on MFA is the right call, and it is also the change most likely to produce a locked-out admin at 5 p.m. on a Friday. Automation that signs in as a user can break as well.

My goal was a rollout order that protects administrators first, leaves one deliberate escape hatch while things settle, and then closes that hatch.

Options and trade-offs

• Default MFA settings alone: simple, but they may not cover administrator-specific rules, so validate which factors are really enabled.
• Custom sign-on policies: fine-grained control by group and application, at the cost of more configuration.
• The pre-seeded console security policy: a good starting point aimed at administrators. It still needs testing and a plan for exceptions.

My recommendation

Enable the factors first, including a phishing-resistant one such as FIDO2 or a passkey. Then enforce through a sign-on policy with the administrator rule at the top and an all-users rule beneath it. Exempt one administrator temporarily, confirm everyone else can sign in, and remove the exemption quickly.

Worked example

The rollout order I use. Rule order matters, because sign-on rules are evaluated from the top.

1. Enable factors: FIDO2/passkey, mobile app notification/passcode.
   Generate a bypass code and store it somewhere offline.
2. Open the sign-on policy for the console (or create one).
3. Rule 1 (top):  group = Administrators  -> require MFA
   Rule 2:        all users                -> require MFA
4. Exclude ONE named admin while you test. Keep a second session open.
5. Sign in with a private browser window as an admin and as a normal user.
6. Remove the exclusion. Re-test enrollment and re-authentication.
7. Confirm service accounts and automation use API keys or a client
   application, not interactive user sign-in.

Before you flip enforcement, check that you can still reach the tenancy through a path that does not depend on the new rule, such as a confidential application that calls the REST API.

Checklist

1. Enable phishing-resistant factors, and keep a bypass code stored offline.
2. Put the administrator rule first, then the all-users rule.
3. Exclude a single administrator temporarily, and remove that exclusion on a set date.
4. Test enrollment and re-authentication in a private window.
5. Write the support steps for a user who loses their device.
6. Watch sign-in failures for the first few days and adjust.

Where this doesn't apply

Non-interactive automation should not go through user MFA. Give it API keys, instance principals or a client application with its own controls.

One honest limitation

Each additional factor adds friction at sign-in and at recovery. I weigh how easy it is to enroll and recover against the security gain, especially for infrequent users.

Sources

• OCI Documentation: Managing Multifactor Authentication in Identity Domains
• OCI Documentation: Managing Sign-On Policies
• OCI Documentation: Security Policy for OCI Console

In my public demo, aceu-edge-infer, I use a small ONNX classifier converted to OpenVINO IR and maintain FP32 and INT8 variants. The service reports which precision it used. It returns four ACE-U dimensions—Authenticity, Credibility, Empathy, and Uniqueness—but these are illustrative outputs from a demo, not validated judgments of real people. The engineering questions are the useful part.

Why quantization can help—and why it changes behavior

Quantization represents selected model weights and activations using lower-precision values. With fewer bits to move and process, an inference path may use less memory and run faster on compatible hardware. The actual speedup depends on the model, operators, runtime, and target device; INT8 is not a universal fast switch.

Lower precision also introduces rounding. Small changes in early layers can flow through later layers, and a score close to a decision threshold may move across it. Some operators quantize cleanly; others may need different quantization settings, to remain in higher precision, or to run through a fallback path.

Look for where the errors gather

Aggregate accuracy can hide concentrated regressions. Compare FP32 and INT8 outputs by class and by meaningful input slice. Inspect examples near decision boundaries, rare patterns, and values near the extremes of the calibration distribution. A useful audit asks not only “How much did the average score move?” but also “Where did the model change its decision?”

Calibrate on data that resembles use

Post-training quantization uses calibration samples to estimate ranges for activations. If the samples do not resemble the inputs the service will see, those ranges can be a poor fit. Include representative cases and the edge cases you care about. When real data cannot be used, diverse synthetic examples can help exercise the pipeline, but they do not replace evaluation on an appropriate held-out set.

A minimal OpenVINO/NNCF sketch looks like this. calibration_samples must yield preprocessed examples in the model’s expected input format:

import nncf
import openvino as ov

core = ov.Core()
model = core.read_model("model_fp32.xml")
calibration_dataset = nncf.Dataset(calibration_samples)

model_int8 = nncf.quantize(model, calibration_dataset)
ov.save_model(model_int8, "model_int8.xml")

This produces a quantized model; it does not establish that the model is accurate enough for a particular use. Keep calibration data and evaluation data separate, and record the evaluation results alongside the model artifact.

Measure more than a mean

• Latency distribution: report p50, p95, and p99 for both precisions. A mean alone can hide a worse tail.
• Output deltas: compare score changes and decision changes across the evaluation set.
• Warm-up: exclude initialization and compilation work from steady-state measurements, but report cold-start behavior separately if it matters in production.
• Representative conditions: use the same device, request shape, batch size, concurrency, and preprocessing path.

The demo contains a small harness that runs synthetic requests against both variants. Its results are scaffolding until measured on an appropriate workload; synthetic measurements should never be presented as production performance.

Know when not to ship INT8

If the latency improvement is marginal while the output changes affect a high-importance class, the trade-off may be negative. For sensitive applications, stability can matter more than a modest speed gain. Compare other options too: batching, request scheduling, a different CPU pool, or optimizing a costly preprocessing step may achieve the goal with less model-level change.

Make precision visible and roll out in steps

Record the precision and execution device in service logs or response metadata so a change is diagnosable. Keep the public response contract minimal and avoid exposing sensitive implementation details.

Before a full rollout, run FP32 and INT8 in shadow for a representative sample and alert on meaningful divergence. If one slice regresses, route it back to FP32 or disable the INT8 path while you investigate. Maintaining both variants adds operational work, but gives the team a controlled recovery option.

Try it yourself

1. Build FP32 and INT8 variants and disclose the precision used for each request.
2. Measure warm and cold paths separately, then report latency percentiles and output-delta distributions.
3. Evaluate by class and edge-case slice, not only by aggregate accuracy.
4. Run a shadow comparison and define in advance what divergence will trigger a rollback.

Closing thought

Quantization is powerful, but faster is not free. Percentiles, output deltas, and slice-level evaluation tell you whether INT8 belongs in the production path—or whether FP32 is the more responsible choice.

Many repositories run code scanning and still merge vulnerable code. The scan runs, the results land in the Security tab, and the pull request turns green anyway, because nothing connects the scan's findings to the decision to merge. Scanning that cannot block a merge is a report, and reports get read late.

GitHub offers three separate mechanisms that people often blur together. Required status checks say a named job must succeed. Code scanning merge protection, configured in a ruleset, says a pull request cannot merge if a required tool finds alerts above a chosen severity, or has not finished, or is not set up. Code owner review says the right people must approve changes to sensitive paths. The GitHub documentation is explicit that merge protection through rulesets is not the same thing as status checks. Using the right one for each job is the whole skill.

Options and trade-offs

• Report only: scanning runs and alerts appear in the Security tab. There is no friction and no enforcement.
• Required status check on the scan job: the job must pass. It is easy to set up, but it checks that the job ran, not what it found, unless the job itself fails on findings.
• Code scanning rule in a ruleset: blocks merges based on alert severity thresholds and also when analysis is incomplete or missing. This is the closest match to "do not merge known problems".
• Code owner review: guarantees that people who understand a directory look at changes to it. It adds latency, so reserve it for paths that carry real risk.
• Branch protection rules versus rulesets: both can require checks and reviews. Rulesets layer, can target tags as well as branches, and allow granular bypass lists. Rulesets are available on public repositories for every plan, and on private repositories with Pro, Team or Enterprise Cloud.

My recommendation

Use a ruleset on the default branch that combines four things: a pull request with at least one approval, a required build check, the CodeQL code scanning rule with a severity threshold you can live with, and code owner review for the directories that matter. Start the severity threshold at high or critical so the gate earns trust, then tighten it as the backlog shrinks. Keep the bypass list short and limited to a role you audit.

Minimal working example

First, a CodeQL workflow that runs on pull requests and on a schedule:

name: codeql
on:
  pull_request:
    branches: [main]
  push:
    branches: [main]
  schedule:
    - cron: "17 3 * * 1"

permissions:
  contents: read
  security-events: write

jobs:
  analyze:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: github/codeql-action/init@v3
        with:
          languages: java-kotlin
          build-mode: autobuild
      - uses: github/codeql-action/analyze@v3

Next, code owners for sensitive paths (.github/CODEOWNERS):

# Default reviewers
*                      @my-org/maintainers

# Security-sensitive paths need the security group
/src/main/java/**/auth/   @my-org/security
/.github/workflows/       @my-org/security

Then a ruleset, saved as ruleset.json and created with the REST API. Verify the field names against the current API reference before relying on it:

{
  "name": "main-protection",
  "target": "branch",
  "enforcement": "active",
  "conditions": {
    "ref_name": { "include": ["~DEFAULT_BRANCH"], "exclude": [] }
  },
  "bypass_actors": [
    { "actor_id": 5, "actor_type": "RepositoryRole", "bypass_mode": "pull_request" }
  ],
  "rules": [
    {
      "type": "pull_request",
      "parameters": {
        "required_approving_review_count": 1,
        "dismiss_stale_reviews_on_push": true,
        "require_code_owner_review": true,
        "require_last_push_approval": false,
        "required_review_thread_resolution": true
      }
    },
    {
      "type": "required_status_checks",
      "parameters": {
        "strict_required_status_checks_policy": true,
        "required_status_checks": [ { "context": "build" } ]
      }
    },
    {
      "type": "code_scanning",
      "parameters": {
        "code_scanning_tools": [
          { "tool": "CodeQL", "security_alerts_threshold": "high_or_higher", "alerts_threshold": "errors" }
        ]
      }
    }
  ]
}

gh api -X POST repos/OWNER/REPO/rulesets --input ruleset.json

Test it in a throwaway branch. Open a pull request that introduces a known issue (for example SQL built by string concatenation) and confirm the merge is blocked. Then fix it and confirm the block lifts. Finally, open a pull request that touches a code-owned path and confirm it waits for the owning team.

Checklist

• Enable code scanning with the CodeQL workflow, and confirm the first run on the default branch finishes before turning on enforcement.
• Add the code scanning rule to a ruleset, with a severity threshold chosen from your current alert backlog.
• Keep a required build check and a pull request rule in the same ruleset.
• Add CODEOWNERS entries only for paths where review adds real value, and turn on code owner review.
• Use enforcement: evaluate first if your plan supports it, to see what would have been blocked before you enforce.
• Keep the bypass list to one audited role.
• Be careful with path filters on workflows that feed required checks. A skipped workflow never reports a status, and a required check that never reports blocks the merge forever.
• Organization-wide code scanning rules are limited to GitHub Enterprise plans. Smaller teams configure the rule per repository.

Where this doesn't apply

Repositories that use an external scanner and never upload SARIF results to GitHub cannot use code scanning merge protection, so the gate has to live in that tool's own CI check. Monorepos that gate through a separate policy engine may prefer to keep one source of truth.

One honest limitation

A severity gate catches what the scanner finds, and scanners miss things. The gate also creates pressure to dismiss alerts rather than fix them, so review dismissals as carefully as you review merges.

Sources

• Code scanning merge protection
• Available rules for rulesets
• About code owners
• REST API for repository rules
• Configuring default setup for code scanning

---

Teams want code, secret and dependency scanning wired into the pull request, not bolted on afterwards. GitHub Advanced Security (GHAS) does that, and for most repositories it runs fine on GitHub-hosted runners. The trouble starts when a build needs custom images, private network access or specialized hardware. Then you are running your own runners, and with them come token handling, scaling and isolation problems that did not exist before.

Options and trade-offs

• GitHub-hosted runners with GHAS: the fastest path. There is no infrastructure to patch and each job gets a clean VM. The limit is that you take the images and network position GitHub gives you.
• Self-hosted runners on Azure Virtual Machine Scale Sets (VMSS): you control the image, the network and the hardware. In return you own scaling, patching, and the credential used to register runners.
• Persistent self-hosted runners: simple to set up, and the worst option for security. State and credentials from one job can leak into the next. Avoid them for anything that builds untrusted code.
• Scope the GHAS features separately: secret scanning and push protection, code scanning with CodeQL, and dependency review. They have different costs and different operational weight, so turn them on in the order that gives the most value for the least friction.

My recommendation

Start on hosted runners and turn on code scanning and secret scanning. Only add self-hosted capacity for a specific reason you can name, and when you do, make every runner ephemeral: one job, then the VM is gone. Keep the registration credential out of the image and fetch it at boot through a managed identity.

Minimal working example

The flow: a VMSS instance boots, signs in with its managed identity, reads a registration secret from Key Vault, registers itself as an ephemeral runner, runs one job, and exits.

Boot script (cloud-init or a custom script extension):

#!/usr/bin/env bash
set -euo pipefail

ORG="my-org"
VAULT="my-runner-vault"
RUNNER_DIR=/opt/actions-runner

# 1. Sign in with the VM's managed identity (no secret on disk)
az login --identity --output none

# 2. Read the credential used to request registration tokens
GH_TOKEN=$(az keyvault secret show --vault-name "$VAULT" --name gh-runner-admin --query value -o tsv)

# 3. Ask GitHub for a short-lived registration token
REG_TOKEN=$(curl -fsS -X POST \
  -H "Authorization: Bearer $GH_TOKEN" \
  -H "Accept: application/vnd.github+json" \
  "https://api.github.com/orgs/$ORG/actions/runners/registration-token" | jq -r .token)

# 4. Register as an ephemeral runner in a dedicated group, then run
cd "$RUNNER_DIR"
./config.sh --url "https://github.com/$ORG" --token "$REG_TOKEN" \
  --ephemeral --unattended --runnergroup azure-vmss --labels azure-vmss,linux
./run.sh

Target the pool from a workflow, and add code scanning alongside it:

name: ci
on: [pull_request]

jobs:
  build:
    runs-on: [self-hosted, linux, azure-vmss]
    steps:
      - uses: actions/checkout@v4
      - run: ./build.sh

  codeql:
    runs-on: ubuntu-latest        # hosted runner is enough for scanning most repos
    permissions:
      security-events: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: github/codeql-action/init@v3
        with:
          languages: java
      - uses: github/codeql-action/analyze@v3

Test it: queue a job, confirm a new VM registers, runs the job, and disappears from the runner list afterwards. Then check that a second job lands on a different VM.

Checklist

• Confirm which GHAS features your plan includes and how active committers are counted before you roll out widely.
• Make every self-hosted runner ephemeral, and put them in a dedicated runner group restricted to the repositories that need them.
• Fetch the registration credential at boot through a managed identity and Key Vault. Never bake it into the image.
• Prefer a GitHub App over a personal access token for requesting registration tokens, so access is scoped and not tied to a person.
• Isolate the runner network. Allow only the egress the builds need.
• Autoscale on queue depth, and cap the maximum instance count so a burst cannot become a bill.
• Use hosted runners for CodeQL unless the build needs something they cannot provide.

Where this doesn't apply

Standard build stacks fit GitHub-hosted runners, and you should use them. Self-hosted capacity is justified by a real constraint (private network, custom hardware, compliance), not by habit.

Sources

• GitHub security features overview
• Self-hosted runners
• GH-500 study guide

---

That kind of drift is easy to miss. A repeatable build takes more than a Dockerfile: it needs a locked dependency graph, a deliberate base-image update policy, and a way to inspect the artifact that actually shipped.

Why “same Dockerfile” can produce a different image

Version ranges allow resolvers to choose newer compatible releases. A mutable base-image tag can also point to a newer image later. Build caches may keep an older layer locally while CI fetches fresh inputs. The source recipe is unchanged, but the resolved inputs differ.

Lock the dependency graph

Use exact direct-dependency pins and commit a lock file that records the resolved transitive versions. Where the package manager supports it, include hashes and install in locked mode. A short input file might look like this:

# requirements.in — human-maintained direct dependencies
fastapi==0.141.1
uvicorn==0.30.1

# Resolve and lock transitive dependencies, including hashes.
pip-compile --generate-hashes --output-file requirements.txt requirements.in

Commit both files. Install from the generated lock in CI rather than re-resolving ranges during every image build:

pip install --require-hashes -r requirements.txt

Pins make change deliberate; they do not remove the need to update packages for fixes.

Choose tags or digests deliberately

Tags are convenient and can move to newer images. Digests identify a specific image immutably. A digest gives a reproducible base, but it will not advance to security fixes on its own—you must monitor updates and refresh it deliberately.

# Illustrative digest placeholder: replace with a verified digest
FROM python:3.12-slim@sha256:<verified-digest>

If you use tags, track when they change and refresh the base during scheduled builds. --pull tells Docker to attempt to pull referenced images before the build:

docker build --pull --tag aceu-trust-demo:main .

Make updates reviewable

Use a dependency update bot or scheduled update pull requests, then gate those changes with tests and image scanning. For production-critical dependencies, document which updates need an explicit reviewer. Prefer a visible, tested update over a silent change arriving through a floating version range.

Keep registry access safe and documented

CI needs a clear path to authenticate, tag, and push. Store credentials in the CI secret manager, use a short-lived token where available, and send passwords over standard input instead of exposing them in command arguments:

# REGISTRY_USER and REGISTRY_TOKEN come from the CI secret store.
printf '%s' "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" \
  --username "$REGISTRY_USER" --password-stdin

IMAGE="$REGISTRY_HOST/aceu-trust-demo:$GIT_SHA"
docker build --pull --tag "$IMAGE" .
docker push "$IMAGE"

Document who owns the credentials, how rotation works, and what a failed login looks like. Never print secret values into build logs.

Compare artifacts when behavior changes

If local and cloud behavior diverge, inspect the image’s package inventory or generate an SBOM. Compare the resolved versions and base-image digest, then decide whether to re-lock, refresh the image, or accept the change with an appropriate test update. Looking at the built artifact beats guessing from the Dockerfile alone.

Trade-offs and failure modes

• Exact pins vs. security updates: pins improve repeatability, while scheduled updates and scans keep them current.
• Tags vs. digests: tags are easier to refresh; digests make the selected base explicit and immutable.
• Locks vs. setup time: generating a lock adds a step, but saves time when builds need to be compared.
• Registry authentication: expired tokens or changed scopes should produce an explicit CI failure, not a misleading successful pipeline.

Try it in your pipeline

1. Replace floating version ranges with exact direct pins and commit a full lock file.
2. Pick a base-image policy: tag plus scheduled refresh, or digest plus an explicit update cadence.
3. Build with --pull when you want CI to check for updated base images.
4. Record the image tag or digest produced by CI and inspect its packages when behavior changes.

Closing thought

“Same Dockerfile” rarely guarantees “same runtime.” Lock files, explicit base-image policy, and artifact inspection turn hidden drift into a change the team can review.

A safe baseline configuration

-XX:FlightRecorderOptions=repository=/var/tmp/jfr
-XX:StartFlightRecording=name=continuous,delay=2m,disk=true,maxage=30m,maxsize=250m,settings=default,dumponexit=true,filename=/var/tmp/jfr/exit.jfr

• delay=2m waits until the application has started and warmed up before recording, so startup noise does not fill your window.
• disk=true writes data to disk, which allows long windows without holding everything in memory.
• maxage=30m and maxsize=250m bound the recording. The data is discarded when either limit is hit, which is what keeps disk use predictable.
• settings=default selects the standard event template, which is designed to be low overhead. Use the heavier profile template only for short, targeted sessions.
• dumponexit=true saves the recording when the JVM exits, which helps with crashes and deliberate restarts.

Capturing data during an incident

jcmd <pid> JFR.dump name=continuous filename=/var/tmp/jfr/incident-$(date +%s).jfr

Defaults per service class

Analyze a recording in Java

import jdk.jfr.consumer.RecordedEvent;
import jdk.jfr.consumer.RecordedFrame;
import jdk.jfr.consumer.RecordedStackTrace;
import jdk.jfr.consumer.RecordingFile;

import java.io.IOException;
import java.io.PrintWriter;
import java.nio.file.Path;
import java.util.*;

public final class JfrAllocationFolder {

    public static void main(String[] args) throws IOException {
        Path jfr = Path.of(args[0]);
        Path out = Path.of(args[1]);

        Map<String, Long> bytesByStack = new HashMap<>();

        for (RecordedEvent event : RecordingFile.readAllEvents(jfr)) {
            if (!"jdk.ObjectAllocationSample".equals(event.getEventType().getName())) {
                continue;
            }
            RecordedStackTrace trace = event.getStackTrace();
            if (trace == null) {
                continue;
            }
            List<RecordedFrame> frames = trace.getFrames();
            List<String> names = new ArrayList<>();
            for (int i = frames.size() - 1; i >= 0; i--) {         // root first
                var method = frames.get(i).getMethod();
                names.add(method.getType().getName() + "." + method.getName());
            }
            String key = String.join(";", names);
            bytesByStack.merge(key, event.getLong("weight"), Long::sum);
        }

        try (PrintWriter w = new PrintWriter(out.toFile())) {
            bytesByStack.forEach((stack, bytes) -> w.println(stack + " " + bytes));
        }
    }
}

Gate your visualizations

Failure modes

• Oversized recordings. Always set maxage and maxsize.
• Mismatched JDK versions. A recording from one JDK may not parse with a very different tool version. Keep your analysis tools at or above the JDK you run.
• Using the heavy template by default. Reserve profile for short investigations.
• No access plan. Decide in advance who can dump recordings and where they are stored, because recordings can contain sensitive data such as stack traces and thread names.

Demo to build

Takeaways

• Continuous JFR is safe when it is bounded by age and size.
• Dump on demand with jcmd and analyze with the consumer API or a flame graph.
• Verify that data exists before visualizing it.

Most teams already have an identity provider such as Entra ID (formerly Azure AD) or Okta, and nobody wants a second set of passwords for the cloud console. Federation solves that, but a careless setup can lock out administrators or leave group membership drifting away from what the IdP says.

I wanted a repeatable order of operations that keeps a way back in at every step.

Options and trade-offs

• Console setup: straightforward and visible, and it needs careful mapping and testing.
• API or automation: repeatable, but there are more moving parts to get wrong the first time.
• Default sign-on and IdP policies versus custom rules: the defaults are a fine start, but confirm what they do to administrators before you rely on them.

My recommendation

Use the Identity Domains flow with a SAML identity provider. Exchange metadata, decide which assertion attribute identifies the user, map IdP groups to IAM groups, and test with a staged non-admin account before an administrator. Change federation and MFA separately so a problem can be traced to one change.

Worked example

The sequence I follow in the Console. It is a checklist, not a script, because the exact screens differ a little between tenancies.

1. Keep a local break-glass admin that does NOT use the new IdP.
2. Identity Domains > Security > Identity providers > Add SAML IdP.
   - Upload the IdP metadata XML (or enter the endpoints and signing cert).
   - Choose the attribute that carries the username (NameID or email).
3. Export the domain's service provider metadata and import it into the IdP.
4. Map IdP groups to IAM groups (group mapping / JIT provisioning).
5. Add the IdP to an identity provider policy rule and activate it.
6. Test: non-admin user first, then an admin. Confirm group-based access.
7. Record the metadata expiry date and who owns the mapping table.

For Entra ID the pattern is the same. You create an enterprise application, upload the service provider metadata, download the federation metadata XML and import it into the domain. OIDC-based identity providers follow the same ideas, with a client ID and secret in place of metadata files.

Checklist

1. Keep a break-glass local administrator outside federation.
2. Document the group mappings and name an owner for each.
3. Test a non-admin sign-in before you test an admin sign-in.
4. Watch for attribute mismatches, and fix the mapping instead of widening a policy.
5. Track the IdP signing certificate expiry and the metadata refresh process.
6. Do not change federation and MFA policy in the same maintenance window.

Where this doesn't apply

A very small tenancy with no external IdP is better off with local users and a strong sign-on policy. Federation adds value when there is already an identity source to follow.

One honest limitation

An expired or rotated IdP certificate can break sign-in without warning. I keep a calendar entry for it and a short, written refresh procedure.

Sources

• OCI Documentation: Federating with Identity Domains (SAML identity providers)
• OCI Documentation: Identity Provider Policies in Identity Domains
• OCI Documentation: Manually Federating Your Tenancy

A release asset is a promise: this file came from this project. Most consumers have no way to check it. If a binary is rebuilt outside CI, swapped in transit, or built from a different commit than the tag suggests, the file name looks identical and nothing flags the difference. Checksums help only if you trust the place you got the checksum from. Build-pipeline compromises such as the 2020 SolarWinds attack are the reason this problem is taken seriously.

GitHub artifact attestations address this with a signed statement that binds an artifact's digest to its build provenance: the repository, workflow, commit and runner that produced it. The statements follow the in-toto and Sigstore formats. You can verify them with the GitHub CLI, online or offline. An SBOM (software bill of materials) can be attached in the same way, so a consumer can see what is inside the artifact as well as where it came from.

Options and trade-offs

• Do nothing: no friction, and no way for anyone to verify anything.
• Checksums only: catches accidental corruption, but proves nothing about origin if the checksum file sits next to the artifact.
• SBOM only: shows contents, but without a signed link to a build it is just another unverified file.
• Build provenance attestation: a signed statement tying the artifact digest to the workflow run. This is the strongest single step for most projects.
• Provenance plus an SBOM attestation: adds contents transparency on top of origin. Verifying the SBOM requires naming its predicate type explicitly, because verification defaults to provenance.
• Custom predicates: record your own assertions (test results, policy checks) in the in-toto format when provenance and SBOM are not enough.

My recommendation

Attest provenance for every released binary and container image, and add an SBOM attestation when consumers care about contents. Treat verification as part of the release process, not an afterthought: run gh attestation verify in the release workflow's final step so a broken attestation fails the release. Be honest in the README about what this proves. A valid attestation says the artifact came from a particular workflow run. It does not say the workflow was uncompromised or the build reproducible.

Minimal working example

A release workflow for a Java project. It builds a jar, creates an SBOM, attests both, builds and attests a container image by digest, and publishes the release:

name: release
on:
  push:
    tags: ["v*"]

permissions:
  contents: write        # create the release
  id-token: write        # sign with the workflow's OIDC identity
  attestations: write    # store the attestation
  packages: write        # push the image

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "21"

      - name: Build
        run: ./gradlew --no-daemon build

      - name: Generate SBOM
        uses: anchore/sbom-action@v0
        with:
          path: .
          format: spdx-json
          output-file: sbom.spdx.json

      - name: Attest build provenance (jar)
        uses: actions/attest-build-provenance@v2
        with:
          subject-path: build/libs/*.jar

      - name: Attest SBOM
        uses: actions/attest-sbom@v2
        with:
          subject-path: build/libs/*.jar
          sbom-path: sbom.spdx.json

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push image
        id: push
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.ref_name }}

      - name: Attest image provenance (by digest)
        uses: actions/attest-build-provenance@v2
        with:
          subject-name: ghcr.io/${{ github.repository }}
          subject-digest: ${{ steps.push.outputs.digest }}
          push-to-registry: true

      - name: Publish release
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: gh release create "${{ github.ref_name }}" build/libs/*.jar sbom.spdx.json --generate-notes

Verify as a consumer:

# Provenance (the default)
gh attestation verify build/libs/app-1.0.0.jar --repo OWNER/REPO

# SBOM: name the predicate type explicitly
gh attestation verify build/libs/app-1.0.0.jar --repo OWNER/REPO \
  --predicate-type https://spdx.dev/Document/v2.3

# Container image
gh attestation verify oci://ghcr.io/OWNER/REPO:v1.0.0 --repo OWNER/REPO

Test it: release a tag, run the three commands against the published assets, then change one byte of the jar and confirm verification fails.

Checklist

• Grant the job id-token: write and attestations: write. Add packages: write when pushing images.
• Attest container images by digest, not by tag, and leave the tag out of subject-name.
• Pin third-party actions to a full commit SHA and let Dependabot update them. The attest actions move between major versions, so check the current release before copying versions from any article.
• Verify the artifact in CI before publishing, and document the verification command in the README.
• Publish the SBOM next to the artifact, and tell users to pass the SBOM predicate type when verifying it.
• Practice offline verification (download the attestation bundle and the trusted root with the CLI) for environments without internet access.
• State plainly in the README what an attestation does and does not prove.

Where this doesn't apply

If you only ship source and no built artifacts, there is little to attest. Attestations for private and internal repositories need GitHub Enterprise Cloud, and the feature is not available on GitHub Enterprise Server, so check your plan first. Fully air-gapped environments need their own trust anchors.

One honest limitation

Provenance is necessary but not sufficient. A signed statement that a workflow built a file does not make the build hermetic or reproducible, and a compromised workflow produces a perfectly valid attestation. Higher assurance depends on build isolation and organizational policy, which the SLSA levels describe.

Sources

• Artifact attestations
• actions/attest-build-provenance
• actions/attest-sbom
• Verifying attestations offline
• SLSA specification, build levels

---

When a container build takes minutes, people stop iterating and start batching changes, which is when bugs hide. Teams usually respond by turning caching on and hoping. That helps a little, but the real gains come from what you send to the builder and the order you ask it to do things in. A Java service is a good case. The dependencies change rarely and the source changes constantly, yet a naive Dockerfile downloads the world on every edit.

Options and trade-offs

• Do nothing and rely on the default layer cache: fine until the first COPY of the whole repo, after which every code change re-downloads dependencies.
• Copy only the build files first, resolve dependencies, then copy the source: much better, but a changed build file still re-downloads everything.
• Add a BuildKit cache mount for the Gradle home: dependency downloads survive across builds, even when the layer itself is invalidated.

My recommendation

Start with a .dockerignore, because it is the cheapest win. Order layers from least to most volatile, use a cache mount for the dependency cache, and use a multi-stage build so the JDK and build tools stay out of the runtime image. Pin dependency versions so speed does not come at the cost of repeatability.

Worked example

A two-stage Dockerfile for a Gradle project. The cache mount is only available during the RUN step, so the build that needs it happens inside the same step.

# syntax=docker/dockerfile:1
FROM eclipse-temurin:21-jdk AS build
WORKDIR /src

# Build files first: this layer only changes when dependencies change
COPY gradlew settings.gradle.kts build.gradle.kts ./
COPY gradle gradle
RUN --mount=type=cache,target=/root/.gradle \
    ./gradlew --no-daemon dependencies > /dev/null

# Source last: this is the layer that changes on every edit
COPY src src
RUN --mount=type=cache,target=/root/.gradle \
    ./gradlew --no-daemon installDist -x test

FROM eclipse-temurin:21-jre
WORKDIR /opt/app
RUN useradd --uid 10001 --create-home appuser
COPY --from=build /src/build/install/app/ ./
USER 10001
EXPOSE 8080
ENTRYPOINT ["/opt/app/bin/app"]

And the context file that keeps the build fast before any layer is considered:

.git
.gradle/
build/
out/
*.iml
.idea/
.env

To see the effect, time a cold build, change one line of source, and time it again:

time DOCKER_BUILDKIT=1 docker build -t demo/app:dev .
echo '// touch' >> src/main/java/App.java
time docker build -t demo/app:dev .

The second build should skip the dependency steps entirely. Bump a dependency version and confirm that step does re-run.

Checklist

1. Add a .dockerignore before changing anything else.
2. Copy build files, resolve dependencies, then copy source.
3. Use a cache mount for the Gradle home, and run the build in the same RUN step.
4. Use a multi-stage build and run as a non-root user.
5. Pin dependency versions, and rebuild without cache now and then to catch drift.

Where this doesn't apply

If your dependencies change on every commit, or the project is tiny, none of this will be noticeable. A remote build service with its own cache may also change which of these steps matter.

One honest limitation

Cache mounts live on the builder, not in the image, so a fresh CI runner starts cold unless you export and import the cache. A fast local build says little about a CI build.

Sources

• Docker Docs: Optimize cache usage in builds
• Docker Docs: Multi-stage builds
• Docker Docs: Dockerfile reference (RUN --mount)
• Docker Docs: .dockerignore

Every OpenVINO application reads a model and compiles it for a specific device when it starts. For a small model on CPU the cost is easy to ignore. For larger models, or for GPU and NPU targets, compilation can dominate the time before the first request completes. If a rollout restarts your service, the first user request can pay that price.

OpenVINO's model caching stores the compiled result so that a later start can import it instead of recompiling. You enable it by pointing OpenVINO at a cache folder. The catch is that not every device supports importing and exporting compiled models. OpenVINO's documentation says such devices work normally but do not get the compilation speed-up, so a cache you configured can quietly do nothing.

That quiet failure is the reason to measure rather than assume. The friction points are confirming device support, understanding how caching interacts with AUTO and the NPU, and measuring a restart in a way that is not fooled by process state.

Options and trade-offs

• Application-level caching: set the CACHE_DIR property on the OpenVINO core before compiling. The first compile has extra work because it also stores the result, so it can be slightly slower. Later compiles can import the stored model.
• Capability check: ask the device whether it supports import and export (the EXPORT_IMPORT capability). If it does not, caching will not shorten that device's compile step, whatever you configure.
• AUTO device selection: the AUTO documentation says AUTO starts inference on the CPU by default unless there is a cached model for the best-suited device, because the CPU can begin right away. While the CPU works, AUTO keeps loading the model onto the better device and switches over when ready. A warm cache can therefore change which device serves your first request.
• NPU: the NPU documentation and plugin README describe a driver-level compiled-model cache enabled by default in current drivers, and OpenVINO's own caching is enabled when you set cache_dir. Read how the two interact in the documentation for your driver and version.
• Model Server: OpenVINO Model Server provides a --cache_dir option and a documentation page on its model cache. The goal is the same as in application code: faster loading on later starts.
• Prefix caching is a different mechanism that works per request instead of per process. Model Server's performance tuning guide lists --enable_prefix_caching true as the default and describes it as reusing work for repeated prompt prefixes, which is common when chat clients resend conversation history.
• General rule for any compiled artifact: it is tied to the device, the software version, and the model and its input shapes. Do not share one cache folder across versions or machine types.
• Recommendation: turn caching on wherever the device supports it, keep the cache folder on storage that survives restarts, and treat the capability check as a gate. If a device does not report EXPORT_IMPORT, plan for the full compile time there. Measure restarts with fresh processes, because a second compile call inside one process does not tell you what a restart costs.
• Decision lens: restart speed against operational risk. A persistent cache is one more thing to manage, and a stale artifact left after an upgrade is a confusing failure. Put the OpenVINO version, model version, and device name in the folder path so you can delete a whole generation at once. Example setup: a container with a volume mounted at the cache path, a startup script that logs compile time, and a health check that confirms the folder contains files after the first run.

Minimal working example

Prereqs

• Python 3 with the openvino package installed (pip install openvino).
• An OpenVINO IR model (model.xml and model.bin) or an ONNX file you already use.
• A writable folder for the cache, on the same filesystem the production service will use.

capability_check.py

import openvino as ov
import openvino.properties.device as device

core = ov.Core()
for name in core.available_devices:
    supported = "EXPORT_IMPORT" in core.get_property(name, device.capabilities)
    print(name, "supports caching" if supported else "no import/export")

cold_vs_cached.py (run it in a new process each time)

import sys, time
import openvino as ov

model_path, device_name = sys.argv[1], sys.argv[2]
core = ov.Core()
core.set_property({"CACHE_DIR": "./model_cache"})

start = time.perf_counter()
compiled = core.compile_model(model_path, device_name)
print(f"compile_model on {device_name}: {time.perf_counter() - start:.3f}s")

Delete the model_cache folder to get a cold run. Run the script again without deleting it to get a cached run.

Adjust file names, model names, and paths to your own setup.

Test: Run capability_check.py and note which devices report support. Run cold_vs_cached.py ten times with the cache folder deleted before each run (cold), then ten times without deleting it (cached), each in a new process. If the cached median is clearly lower, caching is working on that device. If the two medians are close, the device probably does not support caching or the folder is not being used, which is also a useful answer.

Record: device, OpenVINO version, operating system, and model; capability check result; cold compile time as median and 95th percentile; cached compile time as median and 95th percentile; cache folder size; time to first token cold versus cached if you serve a language model; whether outputs match between the two.

Checklist

• Run the capability check for every device you plan to use.
• Set CACHE_DIR (or --cache_dir in Model Server) and confirm files appear after the first run.
• Put version, model, and device in the cache path and store it on persistent storage.
• Measure in separate processes and report percentiles.
• Compare outputs between a cold run and a cached run.
• Avoid: assuming caching works without checking the device's capabilities.
• Avoid: timing two compiles in one process and calling it a restart test.

Where this doesn't apply

Workloads where the model is compiled once for a long-running process and restarts are rare, cases where first-token latency is dominated by something other than compilation (network time or input parsing), and devices that do not support import and export.

Sources

• OpenVINO Model Caching Overview
• Automatic Device Selection documentation
• Automatic Device Selection and Configuration (OpenVINO blog)
• OpenVINO NPU device documentation
• Intel NPU plugin README
• Model Server performance tuning guide
• OpenVINO Model Server parameters

---

ONNX Runtime provides optimized inference across multiple accelerators. In production, AKS GPU node pools must be prepared (drivers/plugins), readiness and health checks must reflect model load state, and autoscaling must consider GPU allocation and throughput constraints.

Options and trade-offs

• NVIDIA Triton with ONNX backend: Multi-framework server with health/readiness endpoints; simplifies model repos and supports gRPC/HTTP. Operationally richer; larger image footprint; excellent for multi-model serving.
• ORT server containers: Leaner; you must select images/runtime versions and wire health probes yourself; good for single model or simple stacks.
• Node pool strategy: Dedicated GPU node pools with NVIDIA device plugin; enforce nvidia.com/gpu limits; separate system and user pools for stable scheduling.

Minimal working example

Prereqs

• AKS cluster; GPU node pool with NVIDIA device plugin; kubectl; Triton image; PVC for model repository.
• Model: ResNet-50 in ONNX.

Deployment (Triton + ONNX)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: triton-server
  namespace: inference
spec:
  replicas: 1
  selector:
    matchLabels:
      app: triton-server
  template:
    metadata:
      labels:
        app: triton-server
    spec:
      containers:
      - name: triton
        image: nvcr.io/nvidia/tritonserver:24.08-py3
        args: ["tritonserver","--model-repository=/models/repository","--strict-model-config=false"]
        ports:
        - containerPort: 8000
          name: http
        - containerPort: 8001
          name: grpc
        readinessProbe:
          httpGet:
            path: /v2/health/ready
            port: 8000
          initialDelaySeconds: 60
          periodSeconds: 10
          failureThreshold: 6
        resources:
          limits:
            nvidia.com/gpu: 1
        volumeMounts:
        - name: model-vol
          mountPath: /models
      volumes:
      - name: model-vol
        persistentVolumeClaim:
          claimName: triton-model-repository-pvc

Test: kubectl get pods -n inference; wait for readiness; send inference to port 8000; confirm GPU assignment with kubectl describe. Version and runtime assumptions

• Triton 24.08; AKS GPU plugin installed; ONNX Runtime on server side via Triton; CUDA matching driver for your GPU.

Checklist

• Validate CUDA/cuDNN compatibility with the ORT/Triton image; mismatches silently fallback to CPU in some cases.
• Pin readinessProbe to /v2/health/ready; initialDelaySeconds accounts for model load.
• Use separate GPU node pools; confirm nvidia.com/gpu schedulable capacity.
• Cap replicas by GPU capacity; ensure PVC throughput supports model load.

Where this doesn't apply

CPU-only inference; use ORT CPU builds and different autoscaling signals.

Sources

• AI Inference on AKS with Triton + ONNX (GA blog)
• ONNX Runtime install and CUDA requirements (GA)
• Use GPUs on AKS Hybrid/Edge node pools (GA)

---

A dynamic group rule that says "every instance in compartment X" is easy to write and easy to regret. The day someone launches a test box or a bastion in that compartment, it silently picks up the same permissions as your production services.

I wanted membership that is predictable and explainable: this instance is in the group because it is in this compartment and carries this tag.

Options and trade-offs

• Compartment-only rules: simple, with broad membership that grows by accident.
• Instance OCID rules: exact, but you edit the rule every time an instance is replaced.
• Compartment plus tag rules: precise enough for most teams, as long as tagging is governed.

My recommendation

I combine a compartment OCID with a defined tag namespace, key and value, joined with ALL so both conditions must be true. Then I treat the tag like an access-control input and restrict who can apply it. Without that last step, anyone who can tag an instance can effectively add it to the group.

Worked example

A rule that requires both the compartment and the tag. The tag here uses a namespace called department and a key called runtime.

ALL {instance.compartment.id = '<apps_compartment_ocid>',
     tag.department.runtime.value = 'payments'}

When a group should cover two compartments, use ANY around the compartment checks and keep the tag condition outside it:

ALL {
  ANY {instance.compartment.id = '<apps_compartment_ocid>',
       instance.compartment.id = '<shared_compartment_ocid>'},
  tag.department.runtime.value = 'payments'
}

The Console rule builder covers the simple cases, but you will type tag rules by hand. Changes to a rule or to a tag are not instant, so I verify the effective membership rather than assuming it, and I keep the rule text in version control next to the policy it supports.

Checklist

1. Define the tag namespace, key and allowed values before writing the rule.
2. Use ALL so the compartment and the tag must both match.
3. Limit who may apply or change the tag through IAM.
4. Keep the rule text and the reason for it in version control.
5. Plan for a delay after rule or tag changes, and verify membership before cutting over.
6. Re-check the effective membership after each deployment.

Where this doesn't apply

Short-lived experiments are better served by an instance-specific rule, or by a throwaway compartment that will be deleted. A tagging scheme is overhead you do not need there.

One honest limitation

The delay before membership changes take effect can surprise a rollout. I stage the change, test that the policy works for the new instances, and only then retire the old rule.

Sources

• OCI Documentation: Managing Dynamic Groups (matching rules)
• OCI Documentation: Tagging Overview
• OCI Documentation: Calling Services from an Instance

• G1 is the default collector since JDK 9. It balances throughput and pause time, and works well across a wide range of heap sizes.
• ZGC is built for low pause times that stay short regardless of heap size, using concurrent work. On JDK 17 it is non-generational, so it examines the whole heap each cycle. JDK 21 added a generational mode (JEP 439), which lowers memory and CPU overhead compared with the single-generation version JEP 439.

# JDK 17
-XX:+UseG1GC
-XX:+UseZGC

# JDK 21+ (generational ZGC)
-XX:+UseZGC -XX:+ZGenerational

Where the cost shows up

• Plan for more memory when upgrading. Moving a service from G1 on an older JDK to ZGC can require a noticeably bigger heap and more native memory. Budget for it in your container limits, or the kernel's OOM killer will make the decision for you.
• Watch RSS, not just heap. Total process memory includes metaspace, thread stacks, code cache, GC data structures, and direct buffers. A container limit equal to -Xmx is almost always too small.

When G1 still wins

• Small to medium heaps where G1 pauses are already well inside your latency budget.
• Throughput-oriented batch jobs, where a few longer pauses cost less than constant concurrent overhead.
• CPU-constrained containers with very few cores, since concurrent GC threads need cores to run on.

When ZGC is worth it

• Large heaps or strict tail-latency targets where G1's pauses show up in your p99.
• Services with enough spare CPU and memory to give the collector room.
• Workloads where a measured test shows fewer long pauses and no allocation stalls.

Heap sizing rules of thumb

1. Measure the live set (memory still reachable after a full collection) from GC logs.
2. Start with a heap at roughly 2x to 3x the live set for concurrent collectors, then adjust from measurement.
3. Set -Xms equal to -Xmx for steady services, to avoid resize pauses.
4. Keep the container limit well above -Xmx to leave room for native memory.
5. Always turn on GC logging in production:

-Xlog:gc*:file=/var/log/app/gc.log:time,uptime,level,tags:filecount=5,filesize=20m

A load generator you can run

import java.util.*;
import java.util.concurrent.ThreadLocalRandom;

public final class AllocationLoad {

    public static void main(String[] args) {
        int cacheEntries = Integer.parseInt(args.length > 0 ? args[0] : "2000000");
        int operations   = Integer.parseInt(args.length > 1 ? args[1] : "5000000");

        Map<Integer, byte[]> cache = new HashMap<>();
        for (int i = 0; i < cacheEntries; i++) {
            cache.put(i, new byte[64]);                       // long-lived data
        }

        long[] latencyNanos = new long[operations];
        ThreadLocalRandom rnd = ThreadLocalRandom.current();

        for (int i = 0; i < operations; i++) {
            long start = System.nanoTime();

            byte[] shortLived = new byte[512 + rnd.nextInt(2048)];     // garbage
            shortLived[0] = 1;
            if ((i & 0x3FF) == 0) {                                    // occasional churn in the cache
                cache.put(rnd.nextInt(cacheEntries), new byte[64]);
            }
            blackhole += shortLived.length;

            latencyNanos[i] = System.nanoTime() - start;
        }

        Arrays.sort(latencyNanos);
        System.out.printf("p50=%dus p99=%dus p99.9=%dus max=%dus%n",
                latencyNanos[operations / 2] / 1_000,
                latencyNanos[(int) (operations * 0.99)] / 1_000,
                latencyNanos[(int) (operations * 0.999)] / 1_000,
                latencyNanos[operations - 1] / 1_000);
    }

    static volatile long blackhole;
}

for heap in 2g 4g 8g; do
  java -Xms$heap -Xmx$heap -XX:+UseG1GC  -Xlog:gc*:file=g1-$heap.log  AllocationLoad
  java -Xms$heap -Xmx$heap -XX:+UseZGC   -Xlog:gc*:file=zgc-$heap.log AllocationLoad
done

What to chart

Failure modes

• A heap sized to the old collector's needs, causing allocation stalls under ZGC.
• A container limit too close to the heap size, causing out-of-memory kills from native memory.
• Judging by averages. GC problems live in the tail.

Takeaways

• ZGC reduces pauses by spending CPU and memory. G1 remains a strong default.
• Size the heap from the measured live set, and size the container from RSS.
• Decide with your own tail-latency numbers, not slogans.

IAM policies tend to grow the same way: someone is blocked, someone writes a broad statement to unblock them, and that statement lives forever. Tenancy-wide grants and "manage all-resources" lines are the usual result, and they are hard to walk back once other things depend on them.

I wanted a default that is easy to follow under time pressure. Scope every statement to a compartment, name the resource family, and pick the verb that matches the job.

Options and trade-offs

• Tenancy-level policies: quick to write and a large blast radius. I keep them for a few shared administrative duties with tightly limited verbs.
• Compartment-first policies: they line up with ownership and cost boundaries. The price is that you need a compartment structure and consistent names.
• Conditional policies: precise, using where clauses on bucket, secret or operation. They also need documentation and testing, or they turn into puzzles.

My recommendation

Start in the compartment that owns the workload. OCI's verbs run from inspect to read to use to manage, so give runtime principals read or use and keep manage for operators. Prefer resource families such as instance-family or object-family over all-resources, and review the policy set on a schedule so it does not drift.

Worked example

Here is the shape I aim for with an application compartment called apps. Runtime access is narrow, and the platform team gets a separate, more powerful statement.

# Runtime: what the app itself needs
Allow dynamic-group app-runtimes to read objects in compartment apps
  where target.bucket.name = 'app-config'
Allow dynamic-group app-runtimes to use log-content in compartment apps
 
# Operators: separate group, separate statements
Allow group platform-admins to manage instance-family in compartment apps
Allow group platform-admins to read policies in compartment apps

Compare that with what usually shows up in a review:

Allow group developers to manage all-resources in tenancy

The second line is shorter and works on day one, which is why it keeps appearing. It is also impossible to explain to an auditor. When delegating to a sub-team, use a compartment path such as A:B so the grant follows the hierarchy.

Be careful with the ability to manage policies. Whoever can manage policies in a compartment can usually grant themselves more, so I keep that permission with a very small group.

Checklist

1. Use read or use for runtime principals, and keep manage for people who operate the service.
2. Scope by compartment, and by compartment path when delegating.
3. Write one purpose per statement, and add a short comment on who, what and why.
4. Use resource families instead of all-resources.
5. Put a condition on the statements that touch sensitive buckets or secrets.
6. Review grants on a schedule and remove the stale ones.

Where this doesn't apply

Incident response across many compartments is a case where a temporary tenancy-level grant is sensible. Time-box it, write down who approved it, and delete it when the incident closes.

One honest limitation

Narrow policies mean more statements, and more statements need maintenance. A naming convention and a template for new compartments, set up early, keep that from becoming the bottleneck.

Sources

• OCI Documentation: Policy Syntax
• OCI Documentation: Policy Reference (verbs and resource families)
• OCI Documentation: Managing Compartments

This article uses a small public demo, aceu-trust-demo, as a running example. It uses a multi-stage image, a non-root user, a health endpoint, and a container deployment behind external HTTPS ingress. Its ACE-U output is illustrative and based on synthetic evidence; it is not a validated assessment of real people. The operational lessons apply more broadly.

The “green” deployment that still fails

A deployment reports “running” and the dashboard looks healthy, but requests time out. A dependency is still warming up. The process is alive, so the liveness signal stays green; the service is simply not ready to serve traffic yet. The failure began when one signal was expected to answer two different questions.

Give each signal one job

Startup: has initialization finished?

Startup covers work such as loading configuration, opening required connections, or priming a cache. Slow startup should not be mistaken for a dead process.

Readiness: can this instance take traffic now?

Readiness is a promise to callers. Keep it false until the dependencies required for the service’s critical path are available. A TCP connection alone may not be enough if the application depends on a loaded index or credentials.

Liveness: is the process stuck in a state that needs a restart?

Liveness should detect an unrecoverable condition, such as a deadlock. It should not fail merely because a downstream system is briefly unavailable: repeatedly restarting healthy processes can amplify an outage.

Docker’s HEALTHCHECK records a container health state. It does not, by itself, define how a particular load balancer or orchestrator routes traffic. In Kubernetes, configure startup, readiness, and liveness probes for their distinct purposes.

A bounded Docker health check

If Python is already part of the runtime image, a tiny local endpoint check avoids adding a separate curl package just for health status:

HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD python -c 'import urllib.request; urllib.request.urlopen("http://localhost:8000/health", timeout=2)' || exit 1

Keep this endpoint cheap and local. Put stricter dependency checks in a separate readiness endpoint, then configure your platform to use that endpoint to gate traffic.

Roll out in stages and predefine rollback

Do not move all traffic at once. Start with a small cohort, hold it under real load for a defined soak period, then expand. Write the rollback condition before the release—for example, a measured latency threshold sustained for a defined duration, or an error rate that crosses a set limit. The thresholds should come from the service’s baseline and risk profile, not be guessed after the rollout starts.

Shut down gracefully

When a platform scales down an idle container, the process may receive SIGTERM. Stop accepting new work, let in-flight requests finish within a deadline, close outbound clients, flush metrics, and exit. Treat SIGKILL as the forced fallback. Without a drain period, a routine scale-down can become partial writes, dangling sessions, or noisy retries.

Make the dependency chain visible

Under pressure, teams often blame the topmost service. Keep a runbook that follows dependencies below the application: runtime, network, storage, cache or index, and external services. Document how each layer fails and how to check it without adding load. Infrastructure problems such as CPU starvation, memory pressure, or network saturation can surface as application timeouts and backpressure.

A green health endpoint can also miss a heavy or rarely used path. Pair alerts with lightweight, non-sensitive transition logs such as ready=false reason=index_warming and ready=true checks=passed. A weekly on-call handoff—what broke, the likely cause, and which mitigation helped—turns recurring cold starts and slow dependency warm-ups into work the team can fix.

Trade-offs to make explicit

• Strict readiness vs. first-green speed: better gating can delay traffic, while reducing user-facing failures.
• Longer soak vs. delivery pace: more observation lowers rollout risk but delays full exposure.
• Graceful drain vs. implementation effort: draining takes work; skipping it risks incomplete operations.
• Frequent liveness checks vs. flapping: overly sensitive checks can restart a service during resource spikes and worsen an incident.

Try it in your service

1. Add a separate /ready endpoint that checks the critical dependency before returning success.
2. Write down a staged rollout, soak period, and rollback trigger before your next release.
3. Handle SIGTERM by draining requests and closing resources cleanly.
4. Extend your runbook with infrastructure checks and first suspects for latency spikes.

Closing thought

Separating startup, readiness, and liveness reduces confusion during incidents and helps limit their blast radius. The demo illustrates the distinction with a local health endpoint and a reversible deployment path. Its ACE-U dimensions—Authenticity, Credibility, Empathy, and Uniqueness—remain conceptual in this context, not a judgment of real people.

API keys are static credentials that must be distributed, rotated, and audited; they are unsuitable for high-assurance environments and agent frameworks at scale. Azure OpenAI supports Microsoft Entra ID authentication, enabling keyless access via managed identities and workload identities. The friction points are: enabling identity where the app is hosted, scoping roles correctly, and handling client capability differences (service vs AKS vs user-interactive).

Options and trade-offs

• System-assigned managed identity (App Service/Functions/Container Apps): Azure hosts present a managed identity; applications obtain Entra access tokens (DefaultAzureCredential) and call Azure OpenAI via the resource's custom subdomain endpoint. Minimal credential handling; role assignment required ("Cognitive Services OpenAI User").
• User-assigned managed identity (App Service/ACA/VMs): Provides explicit identity reuse across multiple hosts; requires client_id configuration and role assignment at resource, RG, or subscription scope. Slightly more setup, clearer scoping.
• AKS Workload Identity: Pods use a Kubernetes service account projected token; a federated identity credential links the service account to a managed identity. This enables short-lived, passwordless authentication to Azure OpenAI from AKS without storing secrets in pods. Requires enabling OIDC issuer, Workload Identity, service account annotations, and pod labeling.
• User-delegated (interactive apps): Use user-based authentication in frontends/worker apps; appropriate when the user context is required (for example, per-user metering or consent). More complexity around token flows, less suitable for daemon agents.

Minimal working example

Prereqs

• .NET 8 or Python 3.9+, Azure.Identity 1.10+ and Azure.AI.OpenAI/Microsoft.Extensions.AI for .NET, or openai 1.x with azure-identity for Python.
• Azure OpenAI resource with a custom subdomain endpoint (required for Entra tokens).
• Role: Cognitive Services OpenAI User assigned to the identity at the resource scope.

A) .NET (App Service with system-assigned MI) Packages: Azure.Identity, Azure.AI.OpenAI, Microsoft.Extensions.AI

using Azure.Identity;
using Azure.AI.OpenAI;

var credential = new DefaultAzureCredential();
var client = new OpenAIClient(new Uri("https://YOUR-RESOURCE.openai.azure.com/"), credential);

var resp = await client.GetChatCompletionsAsync("gpt-4o", new ChatCompletionsOptions
{
    Messages = { new ChatMessage(ChatRole.User, "Hello from Entra auth!") }
});
Console.WriteLine(resp.Value.Choices[0].Message.Content);

Test: Deploy to App Service with identity enabled; assign Cognitive Services OpenAI User on the OpenAI resource; call endpoint and inspect server logs for successful token acquisition. B) AKS Workload Identity (secure access sample) Steps: Enable OIDC issuer and Workload Identity; create user-assigned MI; create federated credential; annotate service account; label pods. Pod manifest snippet:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ai-service
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ai-service
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: workload-identity-sa
      containers:
      - name: ai-service
        image: ghcr.io/azure-samples/aks-store-demo/ai-service:latest
        env:
        - name: USE_AZURE_OPENAI
          value: "True"
        - name: USE_AZURE_AD
          value: "True"
        - name: AZURE_OPENAI_DEPLOYMENT_NAME
          value: "DEPLOYMENT"
        - name: AZURE_OPENAI_ENDPOINT
          value: "https://YOUR-RESOURCE.openai.azure.com/"

Test: kubectl apply; confirm tokens acquired via Workload ID; verify successful calls to Azure OpenAI.

Checklist

• Use the OpenAI resource's custom subdomain endpoint for Entra tokens.
• Assign the Cognitive Services OpenAI User role to the managed identity at the correct scope.
• On AKS, enable Workload Identity and OIDC issuer; configure service account annotations and azure.workload.identity/use label.
• Prefer DefaultAzureCredential for host portability; provide client_id for user-assigned MI.
• Map workloads to least-privilege scopes (resource vs RG vs subscription).

Where this doesn't apply

Purely external, non-Azure hosting lacking managed identity support; use service principals with key vault-backed secrets only as a transitional measure (with rotation and audit).

Sources

• Authenticate an Azure hosted .NET app to Azure OpenAI using Entra ID (GA)
• Foundry (classic) Entra auth for Azure OpenAI (GA)
• Secure access to Azure OpenAI from AKS (GA)

---

Teams adopting OpenVINO for generative workloads meet the same set of switches: compiled-model caching, prefix caching, device selection, KV-cache sizing, and device-specific properties. Each can help. Switched on in different combinations by different people, they produce behavior that nobody can explain.

A playbook turns those switches into decisions with owners and evidence. It starts from budgets, derives defaults from measurements, and records for each default what was measured and on which hardware.

The friction points are deciding the budgets, keeping defaults tied to evidence, and reviewing them when software or hardware changes.

Options and trade-offs

• Budgets: define time to first token at a stated percentile, throughput, and a memory limit for each service. Without a target, tuning has no end.
• Cold-start policy: use compiled-model caching where devices support it, and verify it with the capability check and the output comparison from earlier in the series.
• Device policy: choose explicit devices for predictable behavior, use AUTO where portability matters, and log the executing device. AUTO may serve early requests on the CPU unless a cached model exists for the better device.
• Memory policy: set the KV-cache size explicitly, keep prefix caching on for conversational traffic, limit prompt size and concurrency at the gateway, and define what happens to an oversize request.
• NPU gate: require a compile smoke test, a recorded driver version, a list of the device's supported properties, and a CPU fallback before enabling the NPU for users.
• Evidence rule: every default points to a measurement with the hardware and versions it came from. A default without evidence is a guess and should be labeled as one.
• Recommendation: write the playbook as configuration plus a short decision table in your repository, not as a document nobody reads. Each setting gets a reason, an owner, and a link to the measurement that justified it. Review it whenever OpenVINO or Model Server versions, models, hardware, or traffic change.
• Decision flow in four questions: Is cold start a risk? Enable caching and verify it. Are contexts long or repeated? Keep prefix caching on and set an explicit KV limit. Is an NPU planned? Pass the readiness checklist first. Do you need portability across machines? Use AUTO for routing and log where work ran.
• Decision lens: consistency and defensibility. Example setup: a platform team with one configuration repository, a benchmark job that reruns the key tests on each version upgrade, and a one-page decision flow that new services follow.

Minimal working example

Prereqs

• Results from the earlier articles for the services you run.
• A repository or folder where configuration and results files can live together.

defaults.yaml (outline of a policy file; fill every value from your own measurements)

service: chat-assistant
budget:
  ttft_p95_seconds: <your target>
  memory_limit_gb: <your limit>
caching:
  compiled_model_cache: true
  cache_path: /var/cache/ov/<ov-version>/<model-version>/<device>
  evidence: results/cold-vs-cached.md
device:
  strategy: explicit   # or auto
  log_execution_device: true
  evidence: results/device-strategy.md
kv_cache:
  cache_size_gb: <from observation>
  prefix_caching: true
  overflow_behavior: reject-with-message
  evidence: results/kv-cache.md

Every evidence entry should point to a results file you produced from the earlier articles.

Adjust file names, model names, and paths to your own setup.

Test: Pick one default in defaults.yaml and find the results file that justifies it. Upgrade OpenVINO or Model Server in a test environment and rerun the relevant measurement. If the numbers changed, update the default and its evidence. A default you cannot link to a measurement should be marked as a guess.

Record: the budget for each service; cold start before and after the caching policy; first-token percentile before and after the memory policy; the device strategy outcome on your hardware; the result of the rerun after the last upgrade; the evidence file for every default.

Checklist

• Define budgets for each service.
• Enable and verify compiled-model caching, with an output comparison for warm starts.
• Choose a device policy and log the executing device.
• Set an explicit KV-cache limit from observation.
• Require a readiness checklist for the NPU, attach evidence to every default, and review after upgrades.
• Avoid: copying another team's defaults without measuring.
• Avoid: tuning without budgets.

Where this doesn't apply

Small experiments on one machine where defaults are adequate, and teams that cannot run repeatable measurements.

Sources

• OpenVINO Model Caching Overview
• Automatic Device Selection documentation
• Model Server performance tuning guide
• OpenVINO Model Server parameters
• OpenVINO NPU device documentation
• onnxruntime-openvino on PyPI

Everyone has had the local stack that mostly works. The database is still initialising when the API connects, a worker grabs a port something else expected, and a few minutes later the fans are screaming because three containers decided they own every core. None of this is exotic. It comes from three habits: relying on start order instead of readiness, letting ports be whatever they turn out to be, and leaving resource limits at their defaults. All three are cheap to fix in a Compose file.

Options and trade-offs

• Fixed sleeps in an entrypoint: easy to write, wrong on a slow day and wasteful on a fast one.
• Wait-for scripts: they work, but you maintain a script per dependency and each one hides a different definition of ready.
• Healthchecks plus depends_on with condition service_healthy: Compose does the waiting, and readiness is defined once, next to the service it describes.

My recommendation

I keep a small default stack (database and API) and put the heavy pipeline workers behind a profile, so day-to-day work does not pay for ETL. Every service gets a real healthcheck, downstream services wait on health rather than on container start, ports are bound to 127.0.0.1 and written down, and each service has a CPU and memory ceiling.

Worked example

The example below starts a Postgres database and an API by default. The worker only starts when you ask for the etl profile, and it will not start until the database reports healthy.

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: local-only-password
    ports:
      - "127.0.0.1:5433:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 3s
      retries: 10
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M

  api:
    build: .
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      DB_URL: jdbc:postgresql://db:5432/postgres
    depends_on:
      db:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M

  etl-worker:
    build: ./worker
    profiles: ["etl"]
    environment:
      DB_URL: jdbc:postgresql://db:5432/postgres
    depends_on:
      db:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "0.5"
          memory: 256M

Start the base stack and wait until everything is healthy, then add the pipeline when you need it:

docker compose up -d --wait
docker compose --profile etl up -d
docker compose ps
docker stats --no-stream

The last command is the quick check that the limits are actually applied. If a container sits at its memory ceiling, that is information, not a bug in the cap.

Checklist

1. Put a real readiness check on every service that others depend on.
2. Use condition: service_healthy instead of sleeps or start-order assumptions.
3. Bind ports explicitly, to 127.0.0.1, and keep them in a small documented range.
4. Set CPU and memory limits on every service, and make workers back off or fail with a clear message when they hit them.
5. Mark throwaway volumes clearly and prune them on a schedule.

Where this doesn't apply

This is a local-development pattern. Profiles are a convenience that can drift from how the real system is orchestrated, so do not treat a passing local stack as proof that production wiring is right.

One honest limitation

Healthchecks add startup time, and a heavy check makes it worse. Keep them to a cheap probe, and remember that a limit set too low turns a slow job into an out-of-memory kill.

Sources

• Docker Docs: Control startup and shutdown order in Compose
• Docker Docs: Using profiles with Compose
• Docker Docs: Compose deploy specification (resources)

• A producer compiles with JDK 17 and publishes Java 17 class files (major version 61). A consumer on JDK 11 (which reads up to major version 55) gets UnsupportedClassVersionError.
• Gradle notices first. Each published variant declares the Java version it supports, and a consumer asking for Java 11 refuses a Java 17 variant with an "incompatible" resolution error.
• Hidden transitive dependencies or build plugins still assume Java 8 and fail on a newer toolchain.

The pattern: build with the new JDK, target the old bytecode

// build.gradle.kts (producer library)
plugins { `java-library` }

java {
    toolchain { languageVersion.set(JavaLanguageVersion.of(17)) }   // compile with JDK 17
}

tasks.withType<JavaCompile>().configureEach {
    options.release.set(11)    // emit Java 11 bytecode and restrict APIs to Java 11
}

• It sets the class-file version, so consumers on Java 11 can load the jar.
• It limits you to the Java 11 API surface, so you cannot accidentally call a Java 17-only method. With plain -source/-target you could compile against JDK 17's class library and fail at runtime on an older JVM.

The consumer side

// build.gradle.kts (consumer app)
java { toolchain { languageVersion.set(JavaLanguageVersion.of(11)) } }
dependencies { implementation(project(":producer")) }   // or a published artifact

A rollout sequence that works

1. Inventory consumers, including transitive ones. This step catches most surprises. Search your dependency graph, not just your team's repos.
2. Unlock producers with the toolchain and release pattern above. Nothing changes for consumers.
3. Gate CI so no module accidentally publishes newer bytecode than planned (see below).
4. Upgrade consumers one at a time, starting with leaf services.
5. Raise release in producers once nobody depends on the older level.
6. Upgrade build plugins early. Old Gradle plugins and annotation processors are the most common blockers.

Enforce it with a bytecode gate

import java.io.DataInputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;

public final class BytecodeLevelCheck {

    public static void main(String[] args) throws IOException {
        String jarPath = args[0];
        int maxMajor = Integer.parseInt(args[1]);
        int worst = 0;

        try (ZipFile zip = new ZipFile(jarPath)) {
            var entries = zip.entries();
            while (entries.hasMoreElements()) {
                ZipEntry entry = entries.nextElement();
                if (!entry.getName().endsWith(".class") || entry.getName().startsWith("META-INF/versions/")) {
                    continue;
                }
                try (var in = new DataInputStream(zip.getInputStream(entry))) {
                    in.readInt();                       // magic 0xCAFEBABE
                    in.readUnsignedShort();             // minor
                    worst = Math.max(worst, in.readUnsignedShort());   // major
                }
            }
        }
        if (worst > maxMajor) {
            System.err.println("Bytecode major " + worst + " exceeds allowed " + maxMajor);
            System.exit(1);
        }
        System.out.println("OK: highest major version " + worst);
    }
}

Reproducing the problem in a toy project

Failure modes to expect

• A dependency compiled for Java 8 that uses internal APIs removed in later JDKs.
• Build plugins that read JDK internals reflectively and break on newer versions (look for InaccessibleObjectException).
• Containers and JVM flags: some legacy GC flags are removed or ignored after Java 8, so audit your startup options.

What you gain

Takeaways

• Separate the compile JDK from the bytecode target with a toolchain plus options.release.
• Inventory transitive consumers first, because the graph decides your pace.
• Add a bytecode gate to CI so a policy becomes a failing build.

Every OCI SDK sample starts the same way: drop an API key and a config file on the host. That is fine on a laptop. On a fleet of compute instances it becomes a quiet liability, because those keys never expire on their own, they get copied into images, and nobody remembers which ones are still in use.

Instance principals remove that whole class of problem. The instance proves who it is with a certificate that OCI issues and rotates for it, and IAM decides what it may do through a dynamic group and a policy. There is nothing to ship, nothing to rotate by hand, and nothing to leak from a config file.

Options and trade-offs

• API keys in a config file: the simplest option, but it leaves a long-lived secret on disk and puts rotation on you.
• Session or delegation tokens: shorter lived, but you still have to mint, distribute and refresh them from somewhere.
• Instance principals with a dynamic group: no client-side secret at all. The cost is that you have to think carefully about which instances land in the group.

My recommendation

For code running on OCI compute, I make instance principals the default and write the dynamic group rule before anything else. The rule should name a compartment and a tag, and the policy should name a compartment, a resource family and, where it is practical, a single bucket. Tenancy-wide verbs are the exception and need a reason.

Worked example

The Java below runs only on an instance that belongs to the dynamic group. It reads from one bucket and has no credentials in code, environment variables or files. The namespace and bucket name come from configuration.

import com.oracle.bmc.auth.InstancePrincipalsAuthenticationDetailsProvider;
import com.oracle.bmc.objectstorage.ObjectStorageClient;
import com.oracle.bmc.objectstorage.requests.ListObjectsRequest;
 
public class ListConfigObjects {
    public static void main(String[] args) throws Exception {
        var provider = InstancePrincipalsAuthenticationDetailsProvider.builder().build();
        try (var os = ObjectStorageClient.builder().build(provider)) {
            var req = ListObjectsRequest.builder()
                    .namespaceName(System.getenv("OCI_NAMESPACE"))
                    .bucketName(System.getenv("CONFIG_BUCKET"))
                    .build();
            os.listObjects(req).getListObjects().getObjects()
              .forEach(o -> System.out.println(o.getName()));
        }
    }
}

The matching rule and policy for the dynamic group. The tag makes membership an explicit decision rather than a side effect of where an instance was created.

ALL {instance.compartment.id = '<apps_compartment_ocid>',
     tag.department.runtime.value = 'payments'}
 
Allow dynamic-group app-runtimes to read objects in compartment apps
  where target.bucket.name = 'app-config'
Allow dynamic-group app-runtimes to use log-content in compartment apps

If your dynamic group lives in a non-default identity domain, prefix the group name with the domain, for example dynamic-group MyDomain/app-runtimes.

Checklist

1. Write the dynamic group rule with both a compartment and a tag.
2. Grant the narrowest verb and resource family, and add a bucket or resource condition where you can.
3. Remove every API key and config file from the host once the SDK uses the instance principal provider.
4. Review the audit log and confirm the principal on each call is the instance you expected.
5. Control who can apply the tag, because tagging an instance is what adds it to the group.
6. Write down the rollback: edit the rule, then adjust the policy.

Where this doesn't apply

Serverless functions and other managed resources have their own identity, so use resource principals there. Anything running outside OCI has no instance principal at all and needs a different authentication path.

One honest limitation

Anyone who can get a shell on the instance inherits its permissions. Instance principals take away the secret, not the access, so SSH access and the policy have to be designed together.

Sources

• OCI Documentation: Calling Services from an Instance
• OCI Documentation: Managing Dynamic Groups
• OCI Documentation: OCI SDK for Java, Instance Principals authentication

Personal access tokens are broad. They grant access across every repository the account can reach. GitHub Apps define explicit permissions, install on chosen repositories, and issue installation tokens that expire. Octokit libraries handle token generation and refresh, and actions/create-github-app-token creates scoped tokens inside CI.

Options and trade-offs

• GitHub App versus OAuth app: Apps offer per-permission and per-repository control. OAuth relies on user-granted scopes.
• Octokit versus manual token handling: Octokit handles installation tokens and refresh. Manual handling gives more control and more code to maintain.
• CI token action: mints a scoped token for each job, so you avoid a broad long-lived credential.

My take

Give the app the smallest permission that does the job, install it only on the repositories it needs, and avoid write access to contents unless the bot must change code.

Decision lens: security risk and maintenance cost.

Example setup: a triage bot installed on two repositories with issue write access only. Each CI run mints a token limited to those repositories, adds a comment or label, and never gets contents write access.

Template

src/appClient.js:

import { Octokit } from "@octokit/core";
import { createAppAuth } from "@octokit/auth-app";

export function installationOctokit(appId, privateKey, installationId) {
  return new Octokit({
    authStrategy: createAppAuth,
    auth: { appId, privateKey, installationId }
  });
}

CI: use actions/create-github-app-token with the narrowest permissions and repository list the job needs. Check its README for the current input names.

Checklist

1. Register a GitHub App and generate a private key.
2. Set the minimum permissions the bot needs.
3. Install the app on the target repositories only.
4. Use Octokit's app authentication for installation tokens.
5. Mint scoped tokens in CI for each job.
6. Avoid long-lived personal tokens.

Common mistakes

• Granting contents write when only issue write is needed.
• Using a personal access token, which gives the bot a wider blast radius.
• Assuming tokens never expire. Installation tokens are short-lived, so refresh them.

Where this doesn't apply

• Organizations that do not allow GitHub Apps or require a different authentication flow.

How to get involved

• Good first issue: reduce the permissions of an existing bot and add repository scoping.
• Discussion: what else should a triage bot be allowed to do?

Sources

• Generating an installation access token for a GitHub App: https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app. Installation tokens, permissions, and expiry.
• Authenticating as a GitHub App installation: https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app-installation.
• Octokit.js: https://github.com/octokit/octokit.js/. App authentication and installation behavior.
• create-github-app-token action: https://github.com/actions/create-github-app-token.

The neural processing unit introduced with Intel Core Ultra processors is a low-power device for AI inference. OpenVINO exposes it as the NPU device, but it has requirements and limits that the CPU and GPU do not.

Teams meet the same surprises: a model with dynamic shapes that will not compile, compile times that dwarf the first inference, driver differences between machines, and caching that behaves differently from other devices.

The friction points are driver and platform requirements, the static-shape constraint, and knowing which properties your driver actually exposes.

Options and trade-offs

• Platform and driver: the NPU needs a supported platform and an NPU driver. Record the driver version with every result, because available properties and behavior depend on it.
• Static shapes: Intel community guidance and published research on NPU inference state that the NPU in OpenVINO supports models with static shapes. If your model has dynamic dimensions, plan to fix them before compiling, and check the release notes of your OpenVINO version for any newer capability.
• Driver-level cache: the plugin README and documentation say a user-mode-driver model cache is enabled by default in current NPU drivers to improve time to first inference. OpenVINO's own caching is enabled when you set cache_dir. Read how the two interact in the documentation for your driver and version.
• Property discovery: instead of relying on remembered property names, ask the device. Querying the NPU device for its supported properties lists what your driver and OpenVINO version expose. Record that list with your results.
• Compile time: the first compile can take much longer than the first inference. Use compiled-model caching so a restart does not repeat it, and measure it as described in the caching article.
• Fallback: provide a CPU path for machines where the NPU is absent, the driver is outdated, or compilation fails, and log when the fallback is used.
• Recommendation: treat NPU support as something your build proves. Add a smoke test that compiles a small static-shape model on the NPU and fails loudly if it cannot, and record the platform, driver, and OpenVINO versions. Fix input shapes before compiling, enable caching, and run twice in separate processes to confirm the second start is faster.
• Decision lens: whether the NPU is the right target at all. If your model needs dynamic shapes or more memory than the device offers, the CPU or GPU may be simpler. Example setup: a laptop application that runs a small encoder on the NPU for power efficiency and falls back to the CPU if the smoke test fails.

Minimal working example

Prereqs

• An Intel Core Ultra system with an NPU and a current NPU driver.
• Python 3 with the openvino package installed.
• A static-shape model.

npu_probe.py

import sys, time
import openvino as ov

core = ov.Core()
print("Devices:", core.available_devices)
if "NPU" not in core.available_devices:
    sys.exit("No NPU found; use the CPU fallback")

print("NPU supported properties:")
for prop in core.get_property("NPU", "SUPPORTED_PROPERTIES"):
    print(" ", prop)

core.set_property({"CACHE_DIR": "./npu_cache"})
start = time.perf_counter()
compiled = core.compile_model(sys.argv[1], "NPU")  # a static-shape model
print(f"compile on NPU: {time.perf_counter() - start:.3f}s")

Adjust file names, model names, and paths to your own setup.

Test: Run npu_probe.py twice in separate processes and keep the printed property list. Compare compile time between the first and second run. Then try a model with a dynamic dimension and record what the NPU does. If there is no NPU on the machine, the script exits with a message, which is the fallback case your application also needs to handle.

Record: platform, NPU driver version, and OpenVINO version; the property list the NPU reports; compile time with an empty and a populated cache as median and 95th percentile over at least ten fresh processes; first-inference time cold versus cached; behavior with a dynamic-shape model; fallback behavior when the NPU is unavailable.

Checklist

• Record platform, driver, and OpenVINO versions.
• Add an NPU compile smoke test to your build.
• Fix input shapes before compiling.
• Enable caching and confirm the second start is faster.
• List the properties the device exposes, provide a logged CPU fallback, and re-run everything after a driver or OpenVINO upgrade.
• Avoid: compiling a dynamic-shape model and expecting it to work.
• Avoid: assuming compile time on a developer laptop matches the target machine.

Where this doesn't apply

Machines without an NPU, models that cannot be expressed with static shapes, and workloads where the GPU or CPU already meets latency and power targets.

Sources

• OpenVINO NPU device documentation
• Intel NPU plugin README
• Intel community: NPU supports static shapes
• NITRO: LLM Inference on Intel Laptop NPUs (arXiv)
• OpenVINO Model Caching Overview

---

Platform teams spend much of their day in a shell. The gh CLI covers many everyday GitHub operations: issues, pull requests, projects, releases, and workflow runs. In CI, the GH_TOKEN environment variable provides authentication. With a few scripts, triage and review stop depending on tab-switching.

Options and trade-offs

• Pull requests: list, check out, review, and merge from the terminal. Fast, but comments need discipline.
• Issues: create, list, assign, and comment. Templates can be applied through body files.
• Workflows: run, watch, and rerun. Rerunning only failed jobs saves time.
• gh api: reach any REST or GraphQL endpoint and filter the output with jq. Powerful, and easy to misuse without authentication.

My take

Write a short list of scripts for the actions you repeat most, keep them in the repository, and teach the team a small core set of commands before introducing anything more complex.

Decision lens: review latency and maintenance cost.

Example setup: three scripts for approving and squash-merging a pull request, rerunning failed jobs, and opening a bug issue from a template, each logging its output to a file for an audit trail.

Template

scripts/approve_and_squash.sh:

#!/usr/bin/env bash
set -euo pipefail
gh pr review "$1" --approve
gh pr merge "$1" --squash --delete-branch

scripts/rerun_failed.sh:

#!/usr/bin/env bash
set -euo pipefail
gh run rerun "$1" --failed
gh run watch "$1"

scripts/new_bug.sh:

#!/usr/bin/env bash
set -euo pipefail
gh issue create --title "$1" --body-file .github/ISSUE_TEMPLATE/bug.md --label bug --assignee @me

In CI, set GH_TOKEN so gh commands authenticate.

Checklist

1. Install gh and run gh auth login.
2. Set GH_TOKEN in any workflow that calls gh.
3. Add scripts for pull request review and workflow reruns.
4. Create issues from templates to keep them consistent.
5. Log script output for audit trails.
6. Teach a short list of core commands first.

Common mistakes

• Missing GH_TOKEN in workflows, so gh calls fail.
• Merging without deleting the branch, so stale branches pile up.
• Rerunning an entire workflow when only the failed jobs need it.

Where this doesn't apply

• Locked-down environments where the CLI is not available.

How to get involved

• Good first issue: add a script that opens the logs for a failing run.
• Discussion: which ten gh commands should new team members learn first?

Sources

• Using GitHub CLI in workflows: https://docs.github.com/en/actions/using-workflows/using-github-cli-in-workflows. GH_TOKEN and examples.
• gh pr checks manual: https://cli.github.com/manual/gh_pr_checks. Behavior and exit codes.

---

A vision-language model turns images or video frames into tokens and processes them together with text. A handful of frames can contribute far more tokens than an ordinary chat message. That raises both the time to read the prompt and the memory held for it.

As a result, a service that handles video behaves differently from a text chat service even when the same server software runs both. A cache limit chosen from chat traffic may not hold, and neither may a latency target set for text.

The friction points are setting a frame budget, re-sizing memory with video in the mix, and not assuming that features which help text, such as prefix caching, help video.

Options and trade-offs

• Input types: check the documentation for your OpenVINO GenAI and Model Server versions for which image and video inputs the vision-language pipeline accepts and how frames are sampled. This area changes quickly, so use the documentation for the version you install.
• Frame budget: limit the number of frames and their resolution per request. Because each frame contributes tokens, this is usually the most direct control over both latency and memory.
• Cache limit: the Model Server --cache_size option sets KV-cache memory. A limit that works for text-only traffic should be re-examined with video requests in the mix.
• Prompt reuse: prefix caching reuses work for repeated prompt prefixes. Whether it helps when the changing part of a conversation is video is something to test directly and report either way.
• Device choice: use --target_device to select a device explicitly, and record the device and driver with every result.
• Recommendation: treat video as its own workload with its own limits. Choose a frame budget first, then size memory from measurements that include video requests. Measure the same text prompt with no image, one image, several images, and a short clip; the differences are the cost of each input type.
• Decision lens: input cost versus feature value. Example setup: an internal tool that accepts clips of a few seconds, with a fixed sampling rate, a fixed memory limit, and a clear error message for longer clips.

Minimal working example

Prereqs

• Model Server serving a vision-language model, set up as its documentation for your version describes.
• Docker and Python 3 with requests.
• A sample image and a short video clip.

modality_ladder.py

import time, requests

URL = "http://localhost:8000/v3/chat/completions"
MODEL = "your-vlm-model-name"  # the name you deployed

def ask(content):
    start = time.perf_counter()
    r = requests.post(URL, json={"model": MODEL, "messages": [{"role": "user", "content": content}], "max_tokens": 16})
    return r.status_code, round(time.perf_counter() - start, 3)

text = {"type": "text", "text": "Describe what you see."}
print("text only:", ask([text]))
# Add image and frame entries in the format your server version documents,
# then compare request time as you add more of them.

Follow your Model Server version's documentation for the exact request format for images and video frames.

Adjust file names, model names, and paths to your own setup.

Test: Send the same text prompt with no image, one image, several images, and a short clip, at least twenty times each. Watch the server process's memory. Then raise the number of frames until the server refuses or slows sharply, and note where that happens.

Record: model, precision, device, driver, and software versions; frame sampling and resolution; request time per input type as median and 95th percentile; peak memory per input type; the frame count at which the server refuses or slows; whether prefix caching helps with a stable text history and changing video; your chosen frame budget.

Checklist

• Use the documentation for your exact versions for input formats.
• Set a frame budget.
• Re-size memory using video requests.
• Measure each input type separately and record device and driver with every result.
• Test prefix reuse with changing video, and return a clear error for inputs over budget.
• Avoid: sizing memory from text-only traffic.
• Avoid: assuming prefix caching helps when the changing part is video.

Where this doesn't apply

Text-only services, models or versions without video support, and single-image tools where the cost difference is small.

Sources

• OpenVINO Model Server parameters
• Agentic AI with OpenVINO Model Server (example commands)
• Model Server continuous batching demo
• Model Server performance tuning guide

---

Static docs should update when the code does. GitHub Pages can deploy from a branch or from a workflow. The Actions-based path uses configure-pages, upload-pages-artifact, and deploy-pages, with explicit permissions and a concurrency group that cancels outdated deploys.

Options and trade-offs

• Deploy from a branch: the simplest setup, with limited build control. Jekyll runs by default unless you add a .nojekyll file.
• Deploy with Actions: supports any static site generator and explicit permissions and concurrency controls.
• Environment protection: require checks before the github-pages environment deploys.

My take

Use the Actions path for any project that needs a build step. Protect the deployment environment so a failing build cannot publish.

Decision lens: maintenance cost and review latency.

Example setup: a documentation site built with a JavaScript toolchain, split into a build job that uploads an artifact and a deploy job that publishes it.

Template

.github/workflows/deploy-pages.yml:

name: Deploy static site to Pages
on:
  push:
    branches: [ main ]
  workflow_dispatch:

permissions:
  contents: read
  pages: write
  id-token: write

concurrency:
  group: pages
  cancel-in-progress: true

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/configure-pages@v5
      - name: Build site
        run: |
          npm ci
          npm run build
      - uses: actions/upload-pages-artifact@v3
        with:
          path: ./dist
  deploy:
    needs: build
    runs-on: ubuntu-latest
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    steps:
      - id: deployment
        uses: actions/deploy-pages@v4

Adjust the build commands and the artifact path for your generator.

README: describe the directory layout and note that Pages should be set to deploy from GitHub Actions.

Checklist

1. Set Pages to deploy from GitHub Actions.
2. Add the workflow above and adjust it for your build.
3. Protect the github-pages environment.
4. Add .nojekyll if you deploy from a branch with a non-Jekyll generator.
5. Test the build on a pull request before merging.

Common mistakes

• Missing the pages and id-token permissions, so the deploy fails.
• No concurrency group, so an older deploy can overwrite a newer one.
• Expecting Pages to host dynamic backends. It serves static files only.

Where this doesn't apply

• Projects that need server-side rendering or a dynamic backend.

How to get involved

• Good first issue: convert a manual deploy to a Pages workflow.
• Discussion: which build steps does your docs site need in CI?

Sources

• Creating a GitHub Pages site: https://docs.github.com/articles/creating-project-pages-manually. Branch versus Actions deployment and .nojekyll.
• Pages deploy actions: the actions/deploy-pages and actions/upload-pages-artifact repositories on GitHub. Current action versions and inputs.

---

A language model has to read the whole prompt before it can produce a first token. That prefill step grows with prompt length, so time to first token depends on the prompt lengths your users actually send.

Intel community guidance and published research on NPU inference say that the NPU in OpenVINO works with static shapes. A prompt of varying length does not fit that directly, so a deployment has to choose a strategy: pad to a fixed length, process in fixed-size pieces, or keep several compiled shapes. Newer OpenVINO and GenAI releases may add dedicated support, so read the release notes for your version before designing around the limitation.

The friction points are choosing a strategy, defining prompt-length buckets that match real use, and measuring the tail instead of the average.

Options and trade-offs

• Pad to one fixed length: simple and compatible with static shapes. Short prompts pay for the padding, so it favors uniform prompt lengths.
• Fixed-size chunks: process the prompt piece by piece with a chunk size you choose. It avoids padding everything to the longest length, at the cost of more steps for long prompts. Check whether your OpenVINO and GenAI versions provide this directly.
• Several compiled shapes: keep one compiled model per length bucket and route each request by its prompt length. It is predictable but uses more memory and more cache entries.
• Streaming: returning tokens as they are generated improves perceived latency but does not shorten prefill, so track perceived first-token time and true first-token time separately.
• Measurement unit: use prompt-length buckets such as short, medium, long, and very long, defined in tokens. An average across all prompts hides the long-prompt tail that users notice.
• Recommendation: begin with your real prompt-length distribution; if you have none, make a reasoned guess, label it as a guess, and plan to replace it. Test each strategy your stack supports against every bucket, changing one thing at a time. Adopt a strategy only if it improves the 95th percentile for the buckets that matter, and keep the simpler setup if it does not.
• Decision lens: user-visible first-token latency versus implementation complexity. Example setup: an assistant whose prompts are mostly short uses a small fixed length, with a separate path for the rare long prompt.

Minimal working example

Prereqs

• An NPU machine with a language-model stack that runs on the NPU for your OpenVINO version, set up as that version's documentation describes.
• Python 3 with transformers for the tokenizer.
• Your model's tokenizer.

make_prompts.py (build prompts of controlled token length)

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("your-model-name")  # use your model's tokenizer
filler = "The quick brown fox jumps over the lazy dog. " * 2000
ids = tokenizer(filler)["input_ids"]

buckets = {"short": 64, "medium": 512, "long": 2048}
for name, n in buckets.items():
    prompt = tokenizer.decode(ids[:n])
    print(name, len(tokenizer(prompt)["input_ids"]))

Use the prompts with whichever generation API your stack provides, and record the time from sending the request to the first generated token.

Adjust file names, model names, and paths to your own setup.

Test: Build prompts for each bucket with make_prompts.py. For each strategy your stack supports, warm up, then send at least thirty requests per bucket and record time to first token. Report the median and 95th percentile per bucket, with streaming on and off. Keep every other setting the same between runs.

Record: NPU, driver, OpenVINO, and GenAI versions; model and precision; bucket definitions in tokens; time to first token per bucket and strategy as median and 95th percentile; streaming on versus off; your chosen strategy and the reason.

Checklist

• Read the release notes for your version about NPU and variable prompts.
• Define buckets from real or clearly labeled estimated lengths.
• Warm up before measuring and change one setting per run.
• Report median and 95th percentile per bucket.
• Track perceived and true first-token time separately, and keep the simpler strategy when a change does not help.
• Avoid: averaging across prompt lengths.
• Avoid: assuming a feature from release notes exists in your installed version.

Where this doesn't apply

Pipelines where all prompts have almost the same length, deployments on CPU or GPU, and versions that provide native handling of variable prompts (follow that documentation instead).

Sources

• OpenVINO NPU device documentation
• Intel community: NPU supports static shapes
• NITRO: LLM Inference on Intel Laptop NPUs (arXiv)
• Intel NPU plugin README

---