Actuator and Observability: Health, Metrics and Monitoring

Actuator is the dashboard Spring Boot installs in your application. Once the program runs, you cannot see "how busy am I, are my dependencies fine, how much heap is left, which build is deployed". Actuator packages that internal state into a set of HTTP addresses (called endpoints); you or your ops system asks with a browser or curl, and it answers. It draws no charts and fires no alerts — it just answers honestly.

Five terms, one line each:

• Endpoint: a fixed URL such as /actuator/health; request it and you get JSON back
• Exposure: which endpoints are actually reachable over HTTP; Boot ships only health and info by default
• HealthIndicator: a tiny self-check function meaning "the part I own is UP or DOWN"; the container aggregates all of them into one status
• Micrometer: the metrics facade (what SLF4J is to logging) — your code only instruments, and swapping Prometheus for Datadog changes no business code
• Probe: the two doors Kubernetes knocks on periodically — liveness asks "is the process alive", readiness asks "can it take traffic now"

That tree is the map of this article: one branch is safe to publish, one is internal-only, one stays closed by default. Section 3 walks through each entry.

After this article you should be able to answer three questions:

• Why does /actuator/health say DOWN without telling you which part broke — and how do you make it tell you?
• What exactly is dangerous about include: "*", and which lines belong in a production config?
• Can you spot "the endpoints are getting slow" ten minutes early from a metric? Which one?

After launch, the sentence you dread most is "the system feels a bit slow". Behind it are three concrete questions: what is happening now, what changed, and why. The industry answers them with the "three pillars of observability":

Logs are great at "pinpointing one incident", but to compute "the error rate over the last hour" you would have to count millions of log lines — far too slow. Metrics precompute "aggregatable numbers": QPS, latency percentiles, error rate, connection pool usage — time series by nature, ideal for dashboards and alerts. Tracing then stitches together cross-service call chains.

Spring Boot's Actuator is precisely the entry point for the latter two (especially metrics): it exposes the app's internal state through a set of HTTP endpoints. This article goes from "enabling Actuator" to "wiring up Prometheus and Grafana".

Add one starter and it works with no other change:

By default only health and info are exposed; the rest are not visible over HTTP — a safe default. To use more, list them explicitly:

Run the kernel lab first to see how this gate actually opens and closes:

Now push the gate fully open and watch what an attacker collects within thirty seconds:

You do not have to memorise the difference between those two configs — flip the switches and see it. Here is this article's exposure sandbox: on one side the "just show me everything" config people shout for in chat, on the other the pre-launch shape:

One more thing worth confirming: which file these settings live in, and who overrides whom, decides why "I changed the yml locally but production ignored it".

And the two endings side by side, as one picture:

Actuator has many endpoints, but a dozen matter in practice. Organized by purpose and risk:

Endpoint names are not the thing to memorise — the question you are asking comes first. Here are six everyday questions, each matched with the endpoint to open before anything else. Match one wrongly and it will describe the price you just paid:

/actuator/health aggregates several built-in indicators (DB, Redis, disk, ping) into structured JSON:

For a third-party service that a health check should not blanket-cover, write a custom HealthIndicator:

In Kubernetes, health maps to probes, and liveness and readiness have completely different meanings. You must group them, or you invite the disaster of "a DB hiccup removes every instance":

The aggregation deserves a live look: with three indicators and one DOWN, how is the top-level status voted out, and under what condition does it bother showing the details?

The act health lab showed you the outcome, but the word overall hides a real vote. Spread that voting rule into a single-step debugger and walk it line by line — click Next and watch how the severity ranking flips the whole verdict at step 5:

Then zoom out to the rollout itself — what each probe answers at startup, during a wobble, on recovery and at shutdown:

Watch the graceful-shutdown chain next to it and you will see why draining traffic must come before killing the process:

Spring Boot uses Micrometer as its metrics facade (like SLF4J for logging): business code depends only on MeterRegistry, and the backend can be swapped to Prometheus, Datadog and so on. Know the four fundamental meters:

Built-in metrics by source, the common ones:

Those numbers do not materialise on a dashboard — they are pulled out of /actuator/metrics one question at a time:

Technical metrics tell you "the system is healthy"; business metrics tell you "the business is healthy". Inject MeterRegistry to instrument:

Method-level timing is also available via @Timed (requires a registered TimedAspect):

First turn metrics into a format Prometheus can read — add a registry dependency:

Hitting /actuator/prometheus yields text like this (excerpt):

Prometheus scrapes this endpoint on a schedule and stores it as time series:

Grafana turns the time series into dashboards, and alert rules notify you automatically:

A production endpoint errors out, but the log level is INFO and hides the detail. Restarting to change config is too costly — use the loggers endpoint to change it online:

What does this endpoint actually change? The level field on a Logback logger object — run the whole pipeline once and it is obvious:

/actuator/info shows build info, the Git commit and custom properties, answering "which version is actually running". First make Maven produce build metadata, then expose it:

Now hitting /actuator/info shows the version, build time and Git commit id — "is production new or old" is no longer a guessing game.

Actuator is useful but double-edged by default. Check each item before launch:

You do not have to memorise the checklist — tick the lines that matter and let the generator assemble a yml skeleton you can take straight to production:

With the tools known and the traps marked, string together the order in which your hands should reach for things on the night it breaks — every cell uses one endpoint from this article, and no cell may be skipped:

And the summary that goes with it: observing endpoints make the story tellable, diagnosing endpoints make it findable, and hardening decides who may even ask — miss any of the three and the four-capability picture in Section 0 stays a capability table instead of a triage workflow.

Every row below can be copied and searched as it stands. Beginners stall in the same four places: the endpoint 404s before you have even begun, health says DOWN without naming who, /env leaks a secret, and the management port gets confused with the business port.

The rows above can all be rescued from the error text alone. One class of incident is less kind: the error does not name the culprit — the clue hides in the numbers inside the parentheses. This stack is an excerpt thrown on a probe thread during a real Pod-cycling incident — do not read the conclusion yet, point out the frame you believe is the culprit:

Goal: put together a minimal monitoring project that runs entirely on your laptop and whose config is already a production skeleton — and see for yourself the 200-versus-404 difference a whitelist makes, the components block inside /health, and one line of curl switching a package to DEBUG.

Before you type anything, run the five labs in this kernel console, in order — each one is a conclusion from this article and every echo is computed by the kernel itself:

Step one, pom.xml:

Step two, src/main/resources/application.yml:

Step three, one custom health indicator and one order-count instrument (package com.example.monitorlab):

Step four, start the app and run these curls in order (watch which ones are 8080 and which are 9090):

Expected output (the default shape of ② when you are unauthorized, the two groups from ③, and the status codes of ④ and ⑤):

Acceptance checklist: ① you can explain why there is no db inside liveness; ② change show-details to always, call ② again, and the custom inventoryCache indicator now shows up in components; ③ you can say what causes each of the two 404s in ④ and ⑤ (the first is port isolation, the second is the whitelist).

One line at a time — and the conclusion flips:

1. Replace include with "*" and restart. What you will observe: the _links object from curl :9090/actuator suddenly has a dozen or more keys; inside env the value of spring.datasource.password is masked, but the app.my.access-key-alias=plain-secret you just added prints verbatim — that is the actual boundary of the sanitising rules. Put the config back when you are done.
2. Comment out the management.server.port line. What you will observe: ④ turns into 200, because the management endpoints are back on the business port — and if your gateway then forwards by path, /actuator is public. This one is what makes "port isolation" click.
3. Add db to the liveness group, then break startup with a wrong URL (jdbc:h2:mem:lab;INIT=RUNSCRIPT FROM 'nonexistent.sql'). What you will observe: /actuator/health/liveness goes straight to DOWN — a rehearsal of "Kubernetes repeatedly restarts an instance that only needed to be pulled out of rotation".
4. Change the threshold in InventoryCacheHealthIndicator from 60_000 to -1. What you will observe: the top-level status turns DOWN and the HTTP code becomes 503, but by default you still cannot see why — so you are pushed into changing show-details, which is exactly the second row of Section 12.
5. After step ⑥, hit an endpoint a dozen or more times. What you will observe: DEBUG logs pour out; set the level back to null (inherit from the parent) and it goes quiet at once. The value and the risk of "debugging without a restart", in one minute.

Hint: after item 3, go back to the probe animation in Section 4 — the two timelines should line up exactly.

Build yourself an "on-call self-service panel": one browser screen that shows the key conclusions, with no Grafana installed.

Requirements:

• A custom endpoint @Endpoint(id = "opsboard") whose @ReadOperation returns one aggregated JSON: JVM heap usage, the GC count delta, pending connections in the pool, the 5xx ratio over the last 5 minutes, the row count of t_order, the running Git commit, and uptime
• All of it comes from an injected MeterRegistry, DataSource and HealthContributor; no new database tables allowed
• A @TransactionalEventListener(AFTER_COMMIT) or an interceptor records the business success rate, as one tile of the panel
• The panel endpoint is not exposed by default; it can only be opened through the include list of an internal profile, and your documentation says plainly how to open it and when to close it again
• A README.md with three curls that perform the standard sequence: "spot the anomaly → locate the component → raise the log level temporarily"

Acceptance checklist: ① every field in the output of curl :9090/actuator/opsboard is traceable to a metric name you can point at; ② make the database unavailable on purpose (a wrong URL, or cut the network with Testcontainers) — the matching tile must turn red, /health/readiness must be DOWN and /health/liveness must still be UP; ③ after 60 seconds of load, the 5xx ratio on the panel is within 5% of the figure you compute by hand from http_server_requests_seconds_count{status="500"}; ④ with the internal profile switched off, the endpoint returns 404.