ImageRecorder

A ImageRecorder allows you to record a screenshot from any stream object, e.g. from:

• VideoStream

• WidgetStream

• CameraStream

import ipywidgets as widgets
from ipywebrtc import ImageRecorder, VideoStream

video = VideoStream.from_url('Big.Buck.Bunny.mp4')
video

Using the image recorder, you can take screenshot of the stream clicking the camera button

image_recorder = ImageRecorder(stream=video)
image_recorder

image_recorder.image

Or do it, programatically:

image_recorder.recording = True

image_recorder.autosave = False

image_recorder.download()

image_recorder.image.height

''

The data is PNG encoded (by default), so we show how to use PIL to read in the data

import PIL.Image
import PIL.ImageFilter
import io
im = PIL.Image.open(io.BytesIO(image_recorder.image.value))

---------------------------------------------------------------------------
UnidentifiedImageError                    Traceback (most recent call last)
<ipython-input-1-88478ab353f7> in <module>
      2 import PIL.ImageFilter
      3 import io
----> 4 im = PIL.Image.open(io.BytesIO(image_recorder.image.value))

~/checkouts/readthedocs.org/user_builds/ipywebrtc/conda/latest/lib/python3.6/site-packages/PIL/Image.py in open(fp, mode, formats)
   3022         warnings.warn(message)
   3023     raise UnidentifiedImageError(
-> 3024         "cannot identify image file %r" % (filename if filename else fp)
   3025     )
   3026

UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f35d8467518>

PIL Images display by default as image in the notebook. Calling the filter methods returns a new image which gets displayed directly.

im.filter(PIL.ImageFilter.BLUR)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-886fedf7a729> in <module>
----> 1 im.filter(PIL.ImageFilter.BLUR)

NameError: name 'im' is not defined

Example with scikit image

We first convert the png encoded data to raw pixel values (as a numpy array).

import numpy as np
im_array = np.array(im)
im_array

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-11fff624d0a1> in <module>
----> 1 import numpy as np
      2 im_array = np.array(im)
      3 im_array

ModuleNotFoundError: No module named 'numpy'

Now we can do easy manipulatios, such as reordering the channels (red, green, blue, alpha)

PIL.Image.fromarray(im_array[...,::-1])

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-b8ec24da4056> in <module>
----> 1 PIL.Image.fromarray(im_array[...,::-1])

NameError: name 'im_array' is not defined

Or build a slightly more sophisticated example using scikit-image (run this notebook with a live kernel, such as mybinder for this to work)

from skimage.filters import roberts, sobel, scharr, prewitt
from skimage.color import rgb2gray
from skimage.color.adapt_rgb import adapt_rgb, each_channel, hsv_value
from skimage import filters


image = widgets.Image()
output = widgets.Output()
filter_options = [('roberts', roberts), ('sobel', sobel), ('scharr', scharr), ('prewitt', prewitt)]
filter_widget = widgets.ToggleButtons(options=filter_options)

@output.capture()
def update_image(change):
    # turn into nparray
    im_in = PIL.Image.open(io.BytesIO(image_recorder.image.value))
    im_array = np.array(im_in)[...,:3] # no alpha

    # filter
    filter_function = filter_widget.value
    im_array_edges = adapt_rgb(each_channel)(filter_function)(im_array)
    im_array_edges = ((1-im_array_edges) * 255).astype(np.uint8)
    im_out = PIL.Image.fromarray(im_array_edges)

    # store in image widget
    f = io.BytesIO()
    im_out.save(f, format='png')
    image.value = f.getvalue()

image_recorder.image.observe(update_image, 'value')
filter_widget.observe(update_image, 'value')
widgets.VBox([filter_widget, video, widgets.HBox([image_recorder, image]), output])

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-7d48ae9c1545> in <module>
----> 1 from skimage.filters import roberts, sobel, scharr, prewitt
      2 from skimage.color import rgb2gray
      3 from skimage.color.adapt_rgb import adapt_rgb, each_channel, hsv_value
      4 from skimage import filters
      5

ModuleNotFoundError: No module named 'skimage'