Logs: finding the one line that matters

journalctl reads the logs systemd collects. -u narrows to one service, -p err keeps errors and worse, grep finds the line that explains the failure.

Your site is down and nginx will not start. The answer is already written down, sitting in a pile of lines about DNS, cloud agents and somebody in another country guessing your password. Find the line that explains it. Three filters is all it takes.

Unit
Priority Search

These are real lines from a real box, hostname swapped for the lab's. The two nginx access lines at the top are strangers probing an open proxy about 90 seconds after the machine existed, and the ssh lines are somebody guessing usernames. That is the noise. Your answer was three lines below it.

What you just learned

Most services no longer write their own log file. They print to the screen, systemd catches it, and journalctl reads it back. That is why -u nginx works without you knowing any path: you are asking systemd about a service it already owns.

The older world still exists underneath. /var/log holds real files: auth.log for logins, syslog for everything, and per-app directories like /var/log/nginx/ for access and error logs. Those you read with the tools from block 04, which is why tail, grep and pipes keep showing up.

Logs also get rotated, which is why you find files ending in .1 and .gz beside the live one: yesterday and the day before, compressed, not gone. And for scale, the box these lines came from wrote 1390 journal lines in ten minutes while doing nothing at all. You are never going to read the log. You are going to filter it.

Security camera footage. Nobody watches all of it. You jump to the camera, then to the minute, then to the frame.
You can name it now: say this to your AI
nginx will not start. Show me the last 50 journal lines for the nginx unit, errors only, and tell me what is actually failing.
Naming the unit and the priority is the difference between an AI guessing and an AI reading. You just learned both words.

Seen on a real server

journalctl -u nginx -p err
The two flags that cut this journal from 26 lines to 7: one unit, errors and worse.
nginx: [emerg] still could not bind()
nginx giving up, not the reason it failed. The cause is three lines above it, quieter and more specific.

You can find the line now. Last block: reading the symptom well enough to know what it means before you go looking.