Docker for a small website: useful or unnecessary?

For one small application on one VPS, would you use Docker or install the stack directly?

Containers can make deployments more repeatable, but there are still volumes, networking, updates and backups to look after. For a project maintained by one person, that trade-off seems worth discussing.

What makes you choose one approach over the other? And if you use containers, what do you do about persistent data and recovery?