Breakpoint

Image Processor OOM Errors

An image processing service is crashing with Out of Memory (OOM) errors. This happens sporadically, but seems to be related to high-resolution image uploads. Find the memory leak.

System Overview

A web application allows users to upload images. Uploaded images are sent to a queue. An "Image Processor" service picks up jobs from the queue, resizes the images into several formats (thumbnail, medium, large), and saves them to a cloud storage bucket.

Logs


[2023-11-01 14:20:05] INFO: New job received. Image: 'large-image-a.jpg', size: 15MB.
[2023-11-01 14:20:05] INFO: Starting processing for 'large-image-a.jpg'.
[2023-11-01 14:20:06] INFO: Reading image into memory.
[2023-11-01 14:20:08] INFO: Resizing to thumbnail.
[2023-11-01 14:20:09] INFO: Resizing to medium.
[2023-11-01 14:20:12] INFO: Resizing to large.
[2023-11-01 14:20:13] INFO: Processing complete for 'large-image-a.jpg'.
[2023-11-01 14:21:00] INFO: New job received. Image: 'small-image-c.png', size: 1MB.
[2023-11-01 14:21:01] INFO: Processing complete for 'small-image-c.png'.
[2023-11-01 14:22:10] INFO: New job received. Image: 'huge-image-b.tiff', size: 150MB.
[2023-11-01 14:22:11] INFO: Starting processing for 'huge-image-b.tiff'.
[2023-11-01 14:22:12] INFO: Reading image into memory.
[2023-11-01 14:22:15] WARN: Cloud storage latency detected. Upload may be slow.
[2023-11-01 14:22:18] INFO: Resizing to thumbnail.
[2023-11-01 14:22:25] INFO: Resizing to medium.
[2023-11-01 14:22:35] FATAL: Process terminating due to Out of Memory.

Diagnosis & Plan

Based on the logs, what do you think the root cause is, and what are your proposed next steps?