如何从 Raspberry Pi Camera v2 中读取交错的二进制数据

Question

我正在尝试为植被测绘的遥感项目拍摄未编码的图像。我的设置是 Raspberry Pi 3 上的 PiCamera v2 NoIR，其中包含贴在传感器上的 Roscolux #2007 StoraroBlue 凝胶过滤器和用于环境记录的 SenseHat。

我可以很好地拍摄普通的 PNG 照片，但我不知道如何正确读取二进制数据，因为我所有的尝试都会导致颜色出现条纹。

以下是 Python 脚本捕获图像的部分：

def picamera_logging_thread():
    logger.info('Started camera logging thread')
    while time() < start_time + timeout:
        with PiCamera() as camera:
            # set values
            camera.resolution = (3280, 2464)

            # let automatic exposure settle
            sleep(2)
            image_name = 'image_' + str(int(time()))

            # capture in PNG format at native resolution
            camera.capture(os.path.join(image_dir, image_name + '.png'))
            logger.info('Saved image ' + image_name + '.png')

            # let automatic exposure settle
            sleep(2)
            image_name = 'image_' + str(int(time()))

            # capture in unencoded RGB format
            camera.capture(os.path.join(image_dir, image_name + '.bip'), 'rgb')
            logger.info('Saved image ' + image_name + '.bip')

        # delay the specified interval
        sleep(picamera_logging_interval - 4)
    logger.info('Stopped camera logging thread')

start_time = time()

# start logging threads
Thread(target = sensehat_logging_thread).start()
Thread(target = picamera_logging_thread).start()

使用 picamera Python 库捕获未编码 RGB 图像的在线文档是这样说的：

The resulting RGB data is interleaved. That is to say that the red, green and blue values for a given pixel are grouped together, in that order. The first byte of the data is the red value for the pixel at (0, 0), the second byte is the green value for the same pixel, and the third byte is the blue value for that pixel. The fourth byte is the red value for the pixel at (1, 0), and so on.

现在，我收集到这意味着生成的二进制数据将采用标准 BIP（按像素交错的波段）格式，但我不确定他们的措辞是否推断像素被分组为行或列，因为他们说第二个像素位于 (1, 0)，这让我觉得数据中的第二个像素位于第 2 行第 1 列，而不是预期的第 2 列第 1 行。

我在 R 中编写了一个脚本，该脚本将数据读取为 1 字节整数 (0 - 255) 的矩阵，并将它们添加到三个 RasterLayer 对象的 RasterStack 中，每个波段 (RGB) 一个。我什至尝试添加一个 .hdr 文件并在 QGIS 中打开它，但它总是条纹噪声。

这是 R 脚本：

# This script parses BIP pixel data from a binary file, assuming order band first then rowwise

# imports
#install.packages("raster")
library(raster)

# define filename of binary data
binary_filename <- "image_1490191326.bip"

# define pixel dimensions of the data, width then height
pixel_dimensions <- c(width = 3280, height = 2464)

# open file connection in Binary Read mode
data_file <- file(binary_filename, "rb")

# get data (and upshift)
binary_data <-
  readBin(
    data_file,
    what = "int",
    size = 1,
    n = pixel_dimensions["height"] * pixel_dimensions["width"] * 3 # hopefully the number of bytes in the file
  ) + 128 # upshift data from (-128, 127) to (0, 255) by adding 128

# close connection to file
close(data_file)

# create red raster layer of every 3rd byte, starting at the first one
red <-
  raster(
    matrix(binary_data[seq(from = 1,
                           to = length(binary_data),
                           by = 3)], nrow = pixel_dimensions["width"], ncol = pixel_dimensions["height"]),
    ymn = pixel_dimensions["height"] / 2 * -1,
    ymx = pixel_dimensions["height"] / 2,
    xmn = pixel_dimensions["width"] / 2 * -1,
    xmx = pixel_dimensions["width"] / 2
  )
names(red) <- "red"

# create green raster layer of every 3rd byte, starting at the second one
green <-
  raster(
    matrix(binary_data[seq(from = 2,
                           to = length(binary_data),
                           by = 3)], nrow = pixel_dimensions["width"], ncol = pixel_dimensions["height"]),
    ymn = pixel_dimensions["height"] / 2 * -1,
    ymx = pixel_dimensions["height"] / 2,
    xmn = pixel_dimensions["width"] / 2 * -1,
    xmx = pixel_dimensions["width"] / 2
  )
names(green) <- "green"

# create blue raster layer of every 3rd byte, starting at the third one
blue <-
  raster(
    matrix(binary_data[seq(from = 3,
                           to = length(binary_data),
                           by = 3)], nrow = pixel_dimensions["width"], ncol = pixel_dimensions["height"]),
    ymn = pixel_dimensions["height"] / 2 * -1,
    ymx = pixel_dimensions["height"] / 2,
    xmn = pixel_dimensions["width"] / 2 * -1,
    xmx = pixel_dimensions["width"] / 2
  )
names(blue) <- "blue"

# create raster stack of the three layers
rgb_stack <- stack(red, green, blue)

# plot hopefully correct raster
plot(rgb_stack)
plotRGB(rgb_stack)

# remove unused large data variables from memory
rm(red, green, blue, binary_data)

Here is a link to a Google Drive folder with the binary data, R output, and a JPG version of the PNG image.

我在这里做错了什么？我该如何阅读并提取可用的像素数据？

Answer 1

啊哈！我想我已经解决了。该图像实际上在右侧和底部边缘填充了一些未使用的像素（您可以在下图中看到黑色），因此在原始图像的情况下，公开的 3280x2464 尺寸不正确。正确的是 3296x2464，如果您将这些数字相乘，然后乘以 3（对于 R、G 和 B 像素），您将得到与您的匹配的文件大小。

您可以使用 ImageMagick 将 RGB 文件转换为 JPEG，大多数 Linux 发行版都安装了该文件，并且可用于 macOS/OSX 和 Windows（和 Raspberry Pi）：

magick -depth 8 -size 3296x2464 image.rgb result.jpg

注意 1：如果您的文件名实际上没有以 .rgb 结尾，您可以在它前面加上 RGB: 前缀，以给 ImageMagick 一个提示其内容，例如

magick -depth ... -size ... RGB:image.xyz ...

注2：如果您使用旧版本的ImageMagick，请将magick替换为convert。

注3：如果你想用ImageMagick裁掉黑色填充像素，使用：

convert -depth 8 -size 3296x2464 image.rgb -trim result.jpg

如何从 Raspberry Pi Camera v2 中读取交错的二进制数据

How to read interleaved binary data from a Raspberry Pi Camera v2

binary

r

raster

matrix

raspberry-pi