Uncategorized

Taking Screenshots from Dice Device

Small warning – this article is going to be a little bit technical. So if you are not into programming, you might just scroll down to the screenshots at the bottom :-). In short, it is possible to capture screenshots from the Dice Device.

Surprisingly, capturing screenshots from a microcontroller device is not entirely simple. After conducting some research, it appears that there are two main approaches:

  • One option is to transfer the screen data over the serial port back to a PC and then reconstruct it into an image.
  • The second option is to utilize Wi-Fi on the device, initiate a web server, and transfer the image using the HTTP protocol.

The second method, utilizing a web server, seems more practical for capturing screenshots. This is mainly because the action can be triggered from a PC at any time, regardless of what is currently running on the device. All you need is a web browser to connect to the device.

In this article, let me provide a brief explanation of the mechanism itself. All the classes and functions used are part of the standard Esp32 framework that comes with the device.

First we have to connect to Wi-Fi:

WiFi.begin(WIFI_SSID, WIFI_PASSWORD);

Then we can start the web server and register a get handler – a function that would be invoked whenever somebody in web browers sends request to the device.

if (httpd_start(&server, &config) == ESP_OK)
{
  httpd_register_uri_handler(server, &uri_get);
}

The main wisdom is in the function get_handler.

httpd_uri_t uri_get =
{
  .uri      = "/screenshot",
  .method   = HTTP_GET,
  .handler  = get_handler,
  .user_ctx = NULL
};

esp_err_t get_handler(httpd_req_t *req)
{
  // Get background image
  lv_color_t* backgroundBuffer;
  ...
  lv_snapshot_take_to_buf(..., &backgroundDsc, ...)

  // Encode screenshot into bmp stream
  std::vector<uint8_t> bmpStream = EncodeBMP(backgroundBuffer, 480, 480);

  // Send image over http
  httpd_resp_set_type(req, "image/bmp");
  httpd_resp_send(req, (const char *)bmpStream.data(), bmpStream.size());

  // Free the memory used by the background
  heap_caps_free(backgroundBuffer);

  return ESP_OK;
}

And that’s basically it. The function is invoked whenever somebody from browser connects to the device like this:

http://192.168.175.30/screenshot

The function get_handler will grab the image with lv_snapshot_take_to_buf call, adds to the raw data header in function EncodeBMP (we chosen BMP for its simplicity), then set type of the data using function httpd_resp_set_type to notify the client it should treat it as an image, and then sends the data with httpd_resp_send.

On the client side in your favourite browser it will look like this:

The entire mechanism described above has now been incorporated into the Dice Device code under the macro DD_ENABLE_WIFI. The implementation is a bit more rich than explained above – for instance the black border is added to the image.

Here are a few screenshots captured using the aforementioned mechanism:

Leave a Reply

Your email address will not be published.