@jialin.huang
FRONT-ENDBACK-ENDNETWORK, HTTPOS, COMPUTERCLOUD, AWS, Docker
To live is to risk it all Otherwise you are just an inert chunk of randomly assembled molecules drifting wherever the Universe blows you

© 2024 jialin00.com

Original content since 2022

back
RSS

brutal solution — docker system prune

WHY

Sometimes Dockerfiles have too many layers or remote dependencies (e.g., ADD), making it difficult to identify which layer is causing problems. This is especially annoying when some behaviors come from remote sources.

When you're waiting for JUST the final CMD to start development, but find that rebuilding the Docker image doesn't help, it can be frustrating. It's even more frustrating when you can't recognize the numerous untagged images when using the image command.
If a build takes 10-15 minutes and still fails, it becomes extremely frustrating.

In such cases, a brute way can be very nice.

dangling image v.s. unused image

  1. Unused: Images NOT USED by any container, including stopped containers.
  1. Dangling: Images that exist but have no tag or name.
    On the other hand, a dangling image just means that you've created the new build of the image, but it wasn't given a new name. So the old images you have becomes the dangling image". Those old image are the ones that are untagged and displays "<none>" on its name when you run docker images.

dangling images is unused images

Unused images are not necessarily dangling images.

docker system prune

REMOVE:

  • Stopped containers
  • Unused networks
  • Dangling images
  • Dangling build cache

docker system prune -a

In addition to the docker system prune, it also removes:

  • Unused images

docker system prune --volumes

In addition to the docker system prune, it also removes:

  • Unused volumes
    docker system prune --volumes
# - Deleted Containers
# - Deleted Networks
# - Deleted Volumes
# - Deleted Images
# - Deleted build cache objects

docker-compose down -v --rmi local

Removes containers, networks, volumes, and local images within the scope of the docker-compose.yml file. Local means it only removes images built locally, not those pulled from remote repositories.

docker system prune vs docker builder prune

only discuss builder here.

docker builder prune is more broader and includes:

  1. unused builder cache
  1. unused Intermediate Image Layers (part of builder cache)
    • docker system prune only delete dangling
    • docker builder prune builder is more thorough. DELETE ALL.
  1. Other build-related cache data (source code cache, dependency download cache, temporary files generated during compilation, unused stage results in multi-stage builds)

Intermediate Image Layers (IIL) and Builder Cache

Intermediate Image Layers (IIL) are actual layers that make up the final image. Each instruction in a Dockerfile typically creates an IIL.

Builder cache has a broader scope and includes IILs as well as other data not part of image layers.

FROM ubuntu:20.04
RUN apt-get update && apt-get install -y python3
COPY ./app /app
RUN pip install -r /app/requirements.txt
CMD ["python3", "/app/main.py"]

Use docker history <image_name> will know that

each instruction in a Dockerfile typically creates an IIL

IMAGE          CREATED        CREATED BY                                      SIZE
abc123def456   2 minutes ago  CMD ["python3" "/app/main.py"]                  0B
789ghi101112   2 minutes ago  RUN /bin/sh -c pip install -r /app/require...   10MB
jklmno131415   2 minutes ago  COPY ./app /app # buildkit                      5MB
pqrstu161718   3 minutes ago  RUN /bin/sh -c apt-get update && apt-get i...   100MB
Builder cache has a broader scope and includes IILs as well as other data not part of image layers.

https://stackoverflow.com/questions/30604846/docker-error-no-space-left-on-device/56147993#56147993

EOF