Guides & Tutorials

Guide to Recording Augmented Reality (AR) Videos

Background

Flow AR Videos provide engaging and enjoyable ways to share a data story with your audience. This makes consuming a Flow as simple as watching a video. Flow AR Videos are recorded by a user on an iPhone or Android and can be played back on any device or shared online. This guide provides the process for generating Flow AR Videos in which the data story presenter is viewed in an AR scene along with a Flow containing the visualizations for the presentation. Please note that currently the Flow objects should always appear in-front of the presenter or objects in the scene. Image obfuscation will be a future feature.

Flow_AR_Video_Quick_Guide_2022-03-23a.jpg

Figure 1: Example scene in a Flow AR Video

 

Generating a Flow AR Video

Equipment required

You will need two smart phones to make a Flow AR Video in which both the data story presenter and the Flow are viewed within the same video scene. iPhone 7 or higher is recommended, or any Android released 2016 or later. The WebXR app needs to be installed on the iPhone. For Android, no special app needs to be installed, just the Chrome browser. Newer phones utilize AR Kit and AR Core on the hardware, and in general will be more stable than older phones. You need 2 persons, the data story presenter and the person who films the presentation. The two phones do NOT need to be the same type of phone. In this document, these persons are called User A (who uses Phone A) and User B (who uses Phone B).

Generation of the Flow Data Story

We assume that the Flow data story has already been constructed using the Flow Editor.  Usage of the Flow Editor is described in various documents and training videos, and such usage is not covered in this document.

Important Note: Because the data objects of the Flow need to be seen in consistent locations within the physical 3D space, it is important that the following conventions be followed for the camera settings when generating the Flow that will be used when recording the AR video.

  1. For each Step of the Flow, make sure that the camera position is reset (using the Camera Reset button), and then position all data objects on each of the steps so that they are in the positions that you desire for the AR video.
  2. It is usually desirable to make the swarms a consistent size (i.e. Use the same width, height, depth dimensions (or perhaps just two dimensions)) for each swarm within the Flow.
  3. Place the swarms at consistent locations along the depth axis.
  4. Make location (0,0,0) the center position for each swarm.
    (Items 2 through 4 above will eliminate any distracting jumping of the swarms when advancing to new steps within the Flow.)
  5. The default setting for AR center-point offset above the floor and for AR center-point height offset above a table are shown in the following screenshot.  These settings can be found near the bottom of the list of settings, and for most AR videos, these settings should not be changed.

    Flow_AR_Video_Quick_Guide_2022-03-23a.jpg
    Figure 2: Default settings for AR center-point offsets from table and floor

  6. A helpful technique is to insert a text object containing the text "0,0,0" at the location of (0,0,0) within the scene. This text object should be visible on all steps, and the text object should be locked in place across all steps. This marker can be used to provide visual feedback regarding the locations of all data objects within the scene. A similar text marker containing the text "0,-1.25,0" can be placed at location (0,-1.25,0) within the scene to provide guidance if the user wishes for swarms to be a certain distance off the floor. This text marker should be visible on all steps and should be locked in place across all steps. The marker position (0,-1.25,0) will typically place the marker on the floor. If you want the marker to indicate 0.35 meters above the floor, you could set this marker to occur at position (0,-0.90,0).
  7. After the positioning of the swarms within your Flow have been confirmed to be working well, you can make the text markers invisible on each of the steps. Doing this will leave the markers in place in case you ever need to use them again.

Making the Flow Video

  1. User A is the Flow user who launches a Flow multiuser meeting and delivers the Flow AR data story presentation while using Phone A for guidance in positioning within the physical space and for advancing to the next steps.   User B is the meeting attendee who films the Flow Video using Phone B.
  2. User A and User B agree on a specific location within their physical space at which the center point of the Flow will be located and oriented.  This specific location is called the “Flow Position”.
    They also agree on the position within the physical space from which they will place the white circle indicator onto the "Flow Position".  This specific location is called the "Initial Viewing Position". The "Initial Viewing Position" should be the position from which User B will begin recording the AR Video.  Setting the Initial Viewing Position in this manner ensures the cameras on both phones will see the correct and consistent view of the Flow from whatever positions are taken by User A and User B during the filming of the Flow AR Video.
  3. At this point, using Flow's meeting controls, User A displays the QR code for the meeting and User B accesses the meeting by scanning the QR code with "Phone B" (use the camera application of your phone and point it at the QR code - or within the WebXR app, there is a camera to scan QR codes).
  4. Using the Augmented Reality (AR) functionality on Phone A, User A walks to the "Initial Viewing Position" and then uses the “Recenter” button to place the white circle indicator at the "Flow Position” for the AR Video.  After doing this, User A will be able to navigate around the data in the AR Space while delivering the Flow presentation.   At this time, User A typically walks to a position either in back of or parallel to the Flow Position.
  5. User B also walks to the "Initial Viewing Position" and then places the white circle indicator at the "Flow Position" for the AR Video.   From this point onward, the positions of the Flow in Phone A and Phone B are synchronized when viewing the Flow from various angles within the AR space.  User B will typically be positioned at the Initial Viewing Position when beginning the filming.
  6. User B starts screen recording (see Screen Recording section below) and navigates back to the Flow AR experience on Phone B.
  7. While filming the Flow data story, User A and User B should be positioned so that User A is either behind the Flow or parallel to the Flow when viewed by User B.
  8. At this point User A can navigate around the Flow and deliver the data story presentation while being guided through the AR space by viewing the Flow displayed on Phone A, and User B can film the Flow and the presentation of User A while moving to positions within the AR space that are optimal for filming.  Navigating around the AR space while filming will require some practice, so please budget some practice time when making your first Flow AR video. It is useful for User B move around a bit, rather than standing still, because this helps to show depth over the 2D/flat screen video recording.
  9. Optionally, to have better quality audio, Phone A can record the audio by doing the screen recording with a microphone. This requires some work using video editing software after recording, in order to sync the audio from one phone with the video from the other. Alternatively, a microphone that is located close to User A can be plugged into (or connected with Bluetooth) to Phone B.

How to Create the Screen Recording on Mobile

  • With iOS 14 or later:
    • Go to Settings > Control Center and tap More Controls (or tap Customize Controls if you have iOS 13 or earlier), then tap the Add button next to Screen Recording.
    • Open Control Center on your iPhone or iPod touch, or on your iPad.
    • Touch and hold the gray Record button, then tap Microphone to turn your audio on, so your narration will be recorded with the video.
    • Tap Start Recording, then wait for the three-second countdown. 
    • Record your Flow data story.
    • To end your recording, open Control Center on your iPhone or iPod touch, or on your iPad, then tap the red Record button. Or tap the red status bar at the top of your screen and then tap Stop.
    • Go to the Photos app to find your screen recording.
  • With Android:
    • Download and install the app AZ Screen Recording
    • Launch the app, which puts a tiny red circle onto the screen that you can drag to the bottom left corner of the screen
    • Navigate to your Flow in the browser (use Chrome for AR).
    • Record your Flow data story.
    • To end your recording, tap on the tiny red circle and choose stop recording.
    • Go to your camera roll where videos are saved to find your screen recording.
  • Make any trims or edits as necessary
  • Use any social media platform (TikTok, Instagram, Twitter, Facebook, etc.) to upload your video!
  • Post and tag us @flowimmersive or use the hashtag #flowimmersive

Additional Equipment Recommendations

To provide enhanced stability for AR scenes, we recommend using an inexpensive  phone gimbal, such as the DJI Osomo or OM, https://store.dji.com/product/dji-om-5?vid=106401. A gimbal is a handheld mechanical stabilizer that can help you shoot smooth, cinematic footage without the need for a larger tripod or other stabilization device.

If you want to achieve high quality audio, you can use a variety of microphones.  A good wireless microphone that has recently been used by a broadcasting company in Europe is the Rode Wireless Go, https://www.rode.com/wirelessgo

Additional Tips

Hiding smart phone controls in the AR scene - The smart phone controls cannot be removed from the scene, but the visibility of the controls can be minimized by changing the control color from white to black.  The black color will cause the controls to be much less visible for many backgrounds.  You can change the color from white to black by clicking the "full screen" button as shown in the image below.

Picture1.jpg

Figure 3: Method for changing smart phone controls from white to black

Updated