RotoView Technology White Paper

Background and Problem Definition

Customer demand for information and entertainment on the go, combined with the rapid miniaturization of complex electronic circuits and the emergence of high-resolution active TFT Liquid Crystal and OLED displays have vastly increased the number and variety of smart hand-held devices with information displays. Such devices now include smartphones, tablets, gaming devices, wearable virtual reality (VR) devices, and a variety of hand-held computers, GPS-based maps and others. In addition to the vivid displays, these devices also utilize advanced processors, Internet connectivity and a large amount of internal memory.

The main limitation with hand held devices is the display's small size. It will always remain small due to the hand held device's small form factor and the increasing amount of contents to display. This fact leads to the challenge of displaying large amounts of complex information on a small screen. An additional challenge is to enable single-hand operation of the device.

Most of the modern mobile devices employ a touch screen design that utilizes the Flick and the Drag (panning) gestures. These gestures normally require cumbersome, two-hand operation, and may cause unhealthy ergonomic strains on users that attempt to perform a single hand Flick gesture. Touch screen gestures often cause unintended activation of links that may be present on the screen during scrolling, and often result in fingerprints and dirt being left on the screen.

Another problem arises in touch screen navigated smartphones when the user wishes to mark a "cut-and-paste" section on the display. Such marking can easily be confused with unintentional commands to scroll the display.

INNOVENTIONS' patented RotoView technology was developed to address these problems and to provide intuitive, single-hand view navigation for hand-held devices.

The RotoView® Solution

RotoView Smart Scroll technology pioneered the "tilt and scroll" approach for the view navigation of small hand held devices. Using the rotation of the device to perform scrolling, it immediately achieved the goal of a single hand view navigation (and many years before the emergence of the modern smartphone), as shown in Fig. 2.


RotoView IP portfolio

Fig. 2 RotoView-enabled devices allow the user to scroll any virtual display just by tilting the device. This reduces the need for pressing switches or using your fingers to navigate the display.

The Oblique Angle Viewing Problem

However, there are more objectives that need to be met. One challenging problem occurs when a user rotates the device at wide angles for scrolling, what we refer to as the Oblique Angle Viewing Problem of tilt-based view navigation. It is well known that even with the best retina displays that are capable for wide angle viewing, the user is still much more comfortable watching the screen when it is perpendicular to their line of vision, as shown in Fig. 3 (left). The user is not comfortable when tilt-based scrolling uses wide oblique angles, as depicted in the right side of Fig. 3. As can seen below, the Dynamic Scroll of the RotoView protocol provides a good solution for the Oblique Angle Viewing Problem.




Fig. 3. The Oblique Angle Viewing Problem. The user is much more comfortable viewing the screen at a right angle (top) compared to viewing the screen at an oblique angle (bottom).


The RotoView Protocol

Modern smartphones and other hand held devices already include the rotation and movement sensors necessary for the RotoView solution. The RotoView solution is therefore incorporated as a software protocol that can be easily adapted to any existing smartphone operating system. As we continue to develop the RotoView technology as a ubiquitous solution applicable to diverse types of screen views (e.g. images, documents, maps, web pages), the RotoView Protocol includes numerous alternative commands, so that developers can choose and modify the gestures and commands that they prefer.

The RotoView Protocol defines two main operation modes: the RotoView Navigation mode, where the screen scrolls in response to rotations and movements; and the Fixed mode (or normal mode) where the screen view position on the contents view remains fixed. The RotoView Protocol provides the commands to start and stop the Navigation mode and controls the scrolling based on the device's rotation and movement. Clearly, the transition between Navigation mode and Fixed mode must be efficient and intuitive.

Fig. 4 illustrates the overall view of the RotoView Protocol. During Navigation mode, scrolling in response to device rotation occurs within the Dynamic Scroll mode (that includes the THROW and GLIDE gestures) and the Proportional Scroll mode. The Suspend mode can be optionally activated when the user starts to pan the screen view using a touchscreen drag while RotoView is active. The protocol always returns to the Fixed mode, where the user uses the device without scrolling. The RotoView protocol provides several options to start and stop the Navigation mode, during which the screen view scrolls in response to device rotations. In addition to the use of a switch or touch screen command to activate Navigation mode, RotoView may activate Navigation mode by tapping on the enclosure of the hand held device. Another embodiment activates the Navigation mode by a specific hand gesture like shaking up and down. All of these activation options are well suited for single hand operation.

RotoView
          Protocol Diagram

Fig. 4. The RotoView Protocol includes the Dynamic and Proportional Scroll Modes as well as various Start and Stop commands.

In the following sections we discuss the various components of the RotoView Protocol in detail. We recommend that you review the following video tutorial for the RotoView Protocol as it is currently implemented in our RotoView Photo Magnifier and RotoView PDF Reader apps for Android and the RotoView development app for the iPhone. Readers are also encouraged to experiment with these apps.

RotoView Protocol Tutorial Video



RotoView Dynamic Scroll Mode

In order to resolve the Oblique Angle Viewing Problem, it is important to perform tilt-based scrolling with relatively small rotations of the device, and preferably with frequent returns to the perpendicular viewing position shown in Fig. 3. RotoView Dynamic Scroll mode achieves this goal by converting rotation changes to scrolling speed changes. This guarantees frequent returns to the preferred perpendicular viewing position whenever the scrolling changes direction or stops. As a result, oblique angle viewing is very limited and only performed for short durations.

To allow both wide distance scrolling and short distance scrolling in response to small tilt changes, RotoView Dynamic Scroll mode dynamically changes the scrolling intensity parameter. This is done using our proprietary RotoView Dynamic Response algorithms. The Dynamic Response algorithms determine the amount of view navigation in response to the tilt and movement of the hand held device, exhibiting the following main features:

   1. Use of time response curves providing the instantenous relation between the current relative tilt (i.e. current tilt compared to the inital baseline tilt) and the resulting rate of scrolling.
   2. If the relation between the relative tilt to the scrolling rate is linear, the time response curve is a graph R(t) of the linear coeficients over time, providing the scrolling_rate directly by
      scrolling_rate(t) = R(t)* relative_tilt(t).
   3. If the relation between the relative tilt and the scrolling rate follows a fixed non-linear graph G, namely
       scrolling_rate(t)= G * relative_tilt(t),
then the relative time response curve Rel_R(t) is a relative scaling factor that is used to multiply the resultant non-linear conversion, getting the actual scrolling rate by
      scrolling_rate(t) = Rel_R(t) *G * Relative_tilt(t).
   4. During Dynamic Scroll, the screen view continues to scroll at the current rate when the relative rotation remains the same.
   5. Selection of different response curves for use by different applications, or selection of different response curves depending on the dimensions of each instant of the media.
   6. The response curves may further change dynamically during the navigation process.

RotoView Dynamic Scroll mode provides two general gestures controlled by the Dynamic Response algorithms: the THROW and the GLIDE gestures (or modes). The THROW gesture performs extremely fast scrolling, allowing relatively small rotations to quickly navigate all the areas of the contents view. The GLIDE gesture performs more refined and slower scrolling speed, providing fine navigation. With Dynamic Scroll mode, the Navigation mode typically starts with the THROW and followed by a GLIDE. The initial THROW gesture provides coarse navigation, and the GLIDE gesture allows exact placement of the screen view. User can reactivate the THROW gesture if needed, for example, by touching the Scroll Controller described below.

An important feature of the RotoView protocol is that it allows the user to pan the screen view while RotoView is on. The panning touch gestures brings the screen view to its final location and ends the RotoView scrolling operation once the finger is lifted from the screen.

The Dynamic Scroll mode does not require an exact correlation between orientation changes and actual screen view scrolling, allowing the use of relatively low cost coarse sensors to determine the orientation changes.


Fig. 5. Typical response start with the THROW gesture (coarse scrolling) followed by a GLIDE gesture (fine scrolling) from which the navigation mode is exited.


Fig. 5 illustrates a time response graph for the RotoView Dynamic Scroll taken from our iPhone app. Additional information regarding the Dynamic Response algorithms is available here.

RotoView Proportional Scroll Mode

Unlike the Dynamic Scroll mode, the RotoView Proportional Scroll mode translates the changes in orientations directly to a scrolling distance that is proportional to the amount of the rotation from the baseline. The roll and pitch baselines are obtained by locking the initial attitude of the device when the scrolling process starts. The Proportional Scroll must use a preset rotation range that allows full scrolling of the contents view from edge to edge.

Fig. 6 shows a bottom side view of a device performing horizontal Proportional Scroll along the roll axis. The rotation range angle α is defined as the rotation that is required to scroll the screen view from one edge to the other edge of the contents view. Fig. 7 illustrates the response graph that relates the relative roll angle to the horizontal screen view position.


Fig. 6. Side view of a device performing Proportional Scroll horizontally around the roll axis.




Fig. 7. Proportional Scroll graph relating relative roll to screen view position.


To minimize the Oblique Viewing Angle Problem of Fig. 3, the Proportional Scroll mode should be used with relatively small magnification. The Dynamic Scroll is better for larger magnification as it avoids the Oblique Viewing Angle Problem altogether.

The Proportional Scroll mode can be used for viewing 360 panoramic photos, where it is desirable to correlate the actual device rotaion with the scrolling. For example, a 360 degree device rotation should cause a 360 rotation of the panoramic photo.

Automatic Tilt Baseline Update

When the user tilts the device in a fixed direction, the screen view scrolls the contents view and eventually reaches one of the edges. In a Proportional Scroll mode, the device will be tilted at a particular "edge angle" (relative to a tilt baseline acquired at the start of the scrolling) when the screen view reaches this edge. If the user continues to rotate the device beyond the edge angle in the same fixed direction, the screen view must remain at the left edge of the Contents View. However, when the user rotates back in an opposite direction there will be a rotation range with no response until the tilt angle crosses back the edge angle. RotoView proprietary technology eliminates this problem with an automatic baseline update as outlined in Fig. 8.


Fig. 8. Proportional Scroll baseline update to avoid an unresponsive rotation range.


The example shown in Fig. 8 relates to the proportional scroll graph discussed in Fig. 7, and will be discussed in the context of horizontal scrolling. Clearly, vertical scrolling has the same baseline update mechanism. When the device is tilted counter-clockwise, the screen view is scrolling to the left until it reaches the left edge of the contents view. At this point, the left edge angle is equal to β-α and the screen view must remain set at the left edge. When the user continues to rotate the device in the counter-clockwise direction (see arrow A) to a relative angle equal to β-α-γ, RotoView shifts the baseline (see arrow B) to perform the automatic base line update.

When the user now rotates the device back in the clockwise direction, the automatic shifting of the baseline allows an immediate response (see arrow C) to the user's rotation. This eliminates the problem of unresponsive range that would be exhibited without this baseline update. A small padding angle ε may be added as a small "padding angle" (see arrow D). RotoView technology also uses another method to perform automatic baseline update for the Dynamic Scroll mode when the screen view comes near to any edge.

RotoView Scroll Controller

RotoView assigns a small rectangular area of your phone’s screen view as the Scroll Controller (Fig. 9a and 9b). The Scroll Controller appears on the screen only during RotoView scrolling mode to help guide you while scrolling, and to accept various touch gestures. The Scroll Controller includes a map that represents the page boundary (outside rectangle) and the screen view (inside filled rectangle). The position of the screen view shows which portion of the page you currently see.

Unlike traditional scroll bars where you need to view both horizontal and vertical edges of the screen view in order to know where you are within a magnified contents view, the RotoView Scroll Controller's map provides an immediate bearing on the screen view position.


Fig. 9a. An overview of the RotoView Scroll Controller as used in the RotoView PDF Reader app (Android).


Fig. 9b. A closeup of the RotoView Scroll Controller area (red rectangle boundary) as implemented in the RotoView Photo Magnifier app (iOS).


The RotoView Scroll Controller page details the various colors the Protocol uses to identify the RotoView scroll modes. During the GLIDE gesture, you can reactivate the THROW gesture by tapping the map inside the Scroll Controller (Re-Throw command). You can also reset the baseline from which the tilt is computed by tapping the bottom part of the Scroll Controller (Baseline Reset command).

You can drag the Scroll Controller to change its location on the screen. The Controller's last location is preserved. We recommend placing the controller around the top right area section of the smartphone's screen, within an easy reach of your thumb. If you do not remember where you have last placed the Scroll Controller, long press anywhere on the screen to activate it.

For multiple documents implementation (API 2.3.1 and higher) the extended lines on the sides of the page boundary indicate horizontal page scrolling.

Fig. 10a. Vertical page scrolling.

Vertical page
	            scrolling

Fig. 10b. Horizontal scrolling.

Horizontal
		            page scrolling

Fig. 10c. Horizontal last page.

Horizontal last
	            page

Fig. 10a shows the Scroll Controller Map for multiple documents with vertical page scrolling, while Fig. 10b illustrates horizontal page scrolling. Fig. 10c indicates the last page (in a left to right language like English) of a multi-page document using horizontal page scrolling.

RotoView Start and Stop Commands

The RotoView Protocol provides several Start and Stop commands to switch between RotoView Navigation (scrolling) and Fixed (normal) modes. These commands were designed to facilitate convenient single hand operation, as well as achieving good integration between rotation/movement and touchscreen gestures. Although we divide the commands into distinguished groups in the following paragraphs, it should be noted that the user can intermix implemented commands from various groups, e.g. to start with a shake command and stop with a touch screen command.

Continuous Touch Operation

You can start with a long press anywhere on the screen. If you keep your finger pressed more than a second, RotoView activates the Continuous Touch operation where scrolling ends once you lift your finger.

The RotoView Scroll Controller indicates the Continuous Touch operation mode by a narrow orange marker.

Single Hand Operation (Non-Continuous Touch operation)

Shake Commands

Shaking the device up and down, as implemented in the RotoView Android and iPhone apps, is very convenient for a single hand operation. It can be used for a start and stop command. An important feature of the RotoView Protocol prohibits the end shake command from disturbing the final location of the screen view. RotoView maintains a stored trail of screen movements and when an end shake is detected, the screen is restored to the last position prior to the shake.

Touchscreen Commands

RotoView is well designed to integrate with the standard touch screen user interface. Accordingly, RotoView activation commands are implemented with the unique RotoView Scroll Controller as shown in Fig. 11. Start RotoView by tapping the screen area assigned for the Scroll Controller (Fig. 11 left). Remember that you can place the Scroll Controller anywhere on the screen. Stop RotoView by tapping anywhere outside the screen area assigned for the Scroll Controller (Fig. 11 right).


Fig. 11. RotoView tap on (left) and tap off (right) commands.


Single hand operation also starts with a long press anywhere on the screen, which lasts less than a second. If you continue to hold beyond one second, the Continuous Touch mode will be triggered.

Suspend Mode

Suspend ModeAn important feature of the RotoView protocol is that it allows the user to pan the image while RotoView is on without the need to explicitly turn RotoView scrolling off. RotoView transits into the Suspend Mode, as evidenced by a change of color. The panning touch gestures brings the screen view to its final location and ends the RotoView scrolling operation once the finger is lifted from the screen.

This feature is another example of RotoView's smooth integration with the dominant touch screen technology of mobile devices.

Inactivity RotoView Mode Exit

The RotoView Protocol can be set to exit the RotoView Navigation mode if the device rotations stop for a preset time. This feature eliminates the need for an explicit stop command, significantly improving the user experience. You can set the inactivity time and define the threshold rotation that is regarded as idle.

Sound and Voice Commands

RotoView navigation can be optionally activated by voice commands or by sound commands. Sound commands are created when your finger taps the back or the side of the device, creating a sound that activates or deactivate the scrolling. In the case of RotoView Smart Watch tilt control implementation, a sound sensor within the watch can detect user's finger taps on the device or even when the user tap two fingers together on the same hand that holds the watch (true single hand operation). Single or double finger taps can be used to initiate different commands.

Tapping the back of the device to stop the scrolling may inadvertently rotate the device and cause the screen view position to change. Again, the RotoView Protocol corrects for this inadvertent screen view movement with the stored trail, as explained for the shake and movement commands.

Gimbal Lock Exit

When using gyroscopes for tilt control, rotating the device pitch over the vertical to ground may result in a gimbal lock condition, causing the reading from the gyroscope to flip over. This condition is very rare, and may occur if you start RotoView standing up and then lying down while RotoView Navigation is still active. RotoView simply turns off the RotoView navigation mode with a unique indicator that appears briefly in the Scroll Controller. The gimbal lock condition is automatically removed when you reactivate RotoView Scroll.

Moving from Page to Page

For multiple documents implementation (API 2.3.1 and higher) you can switch between pages with a double-tilt hand gesture as described in Fig. 12. Please note that the double-tilt hand gesture must be activated in the app Settings.

Page Switching Hand Gesture

Fig. 12. Double-tilt hand gesture to switch from one page to the next.


To cross to the next page where the red arrow is pointing, first rotate backwards when the red arrow appears. Once the green arrow appears, rotate the device following the green arrow to complete the page crossing.

RotoView Implementation

It is important that the entire operation of RotoView will be intuitive and therefore easy to use, and yet be implemented at a low cost. There is no need to implement the entire Protocol listed above - a developer can pick and choose which commands and modes to implement based on the application at hand.

RotoView can use any tilt sensors that may already be built into the hand held device. INNOVENTIONS has been promoting the RotoView technology for many years with the hope that the smartphone manufacturers will license and add it to their OS.

Choice of Orientation Sensors

RotoView technology can work with any orientation sensors that are available in today's hand held devices. These include the following sensors:
1. Gyroscopes, which are typically based on MEMS technology.
2. Accelerometers, which are also silicon-based MEMS sensors.
3. Visible or IR Camera-based tilt sensors which determine orientation based on changes between captured images.
4. Magnetic sensors that detect orientation changes relative to Earth's magnetic field.

Additional issues relating to the user interface experience with RotoView are detailed here.

RotoView Strategic Patents

The RotoView technology and strategic patents gained major publicity starting in the early 2000s. In addition to numerous editorial mentions over the past 10+ years, INNOVENTIONS spent significant marketing efforts to bring the RotoView idea to the attention of the major manufacturers of hand-held devices. Therefore, the reader of this document should not be confused as to the originality and novelty of the RotoView patents when noticing that most of today's hand held devices already include the accelerometer, gyroscope or other form of sensor to sense the orientation.

The RotoView patents strategic prominence is corroborated by the fact that they are highly cited, as described in this patent citation analysis.

Summary of RotoView Features and Benefits

Document Major Revisions
Last update: May 6, 2016.
1. July 2003: This document was first published with the introduction of the RotoView PC-Based development system.
2. Dec. 2009 revision: supporting the first version of the RotoView iPhone app.
3. Aug. 2013 revision: supporting the new features employed in the RotoView iPhone app version 2.0.
4. Feb. 2014 revision: supporting the new features of the RotoView Protocol API 2.1.1.
5. June 2014 revision: supporting the implementation of RotoView Protocol API 2.2.1 in Android.
6. April 2015 revision: supporting the new page switching double-tilt gesture of RotoView Protocol API 2.3.1.
7. May 2016 revision: expanding the explanation of new features covered by the recently issued RotoView patents.

RotoView is covered by US Patents 6,466,198, 6,933,923, 8,675,019, 9,181,760, 9,348,435, 9,424,810 and European patent EP1290672 in the UK, France, Germany, and Italy. Other patents pending. RotoView Protocol © 2000-2016 by INNOVENTIONS, Inc. All rights reserved. For more information, please contact Scott LaRoche, 1-281-879-6226, scott@innoventions.com.

INNOVENTIONS® Inc. "INNOVATIVE PRODUCTS FROM INVENTIVE MINDS" Copyright © 2016 INNOVENTIONS, INC. All rights reserved. Android is a trademark of Google Inc. Apple, the Apple logo, iPhone, iPod, iPod touch, and iTunes are trademarks of Apple Inc., registered in the U.S. and other countries. App Store is a service mark of Apple Inc. RotoView and INNOVENTIONS are registered trademarks of INNOVENTIONS, Inc.