I recently started to take a look at OpenCV for doing some (programmatic) image processing for a small project I’ll maybe talk about later on.

My problem: in my program I receive JPEG-images in a buffer over a network connection and not by opening a file. Now my question was: how to create an OpenCV Mat(rix) from this buffer? Normally should not fill a whole post, but it took me too much time to develop to not document it now.

Strange enough, even on Stackoverflow I only found partial answers. I did a half-hearted web-search and found nothing really complete. Here are the facts I gathered (no guarantee for their correctness, but this is my current state of understanding)

1. OpenCV does not directly support the importation of JPEGs from a buffer (but it does support the reading of a file).
2. You need to use a libjpeg-variant to create an uncompressed image which then can be imported into the Matrix
3. OpenCV  needs images in BGR-colorspaces to be processed further on, by default images are in RGB-colorspace

When doing this kind of  processing I want to limit copies and processing time. Here’s the code I came up with:

class ImageProcessing
{
  struct jpeg_decompress_struct cinfo;
  struct jpeg_error_mgr jerr;

public:
  ImageProcessing()
  {
    cinfo.err = jpeg_std_error(&jerr);
    jpeg_create_decompress(&cinfo);
  }

  ~ImageProcessing()
  {
    jpeg_destroy_decompress(&cinfo);
  }

  void handleImage(uint8_t *buffer, size_t size)
  {
    jpeg_mem_src(&cinfo, buffer, size);

    switch (jpeg_read_header(&cinfo, TRUE)) {
    case JPEG_SUSPENDED:
    case JPEG_HEADER_TABLES_ONLY:
      return;
    case JPEG_HEADER_OK:
      break;
    }

    cinfo.out_color_space = JCS_EXT_BGR;

    jpeg_start_decompress(&cinfo);

    cv::Mat src = cv::Mat(
                      cv::Size(cinfo.output_width, cinfo.output_height),
                      CV_8UC3);

    while (cinfo.output_scanline < cinfo.output_height) {
      JSAMPLE *row = src.ptr(cinfo.output_scanline);
      jpeg_read_scanlines(&cinfo, &row, 1);
    }

    jpeg_finish_decompress(&cinfo);

    cv::imshow("test", src);
    cv::waitKey(0);
  }
};

Summary: I decode the JPEG-buffer using libpjeg-turbo (pre-installed) directly into the buffer allocated by cv::Mat, using the BGR-colorspace as expected by OpenCV.

Line 30: is where I’m telling libjpeg to decode directly to BGR-colorspace
Line 39-40: does the decoding line-by-line directly into the cv::Mat-buffer correspond to the line which can be easily retrieved by the cv::Mat::ptr()-method.

Careful, this code is just a snippet showing how I did. It is neither complete nor self-standing.