
% Andrew G. West - prop_spec.tex
% Main LaTeX file for CIS400/401 Project Proposal Specification
%
% Once built and in PDF form this document outlines the format of a project proposal. However, in raw (.tex) form, we also try to comment on some basic LaTeX technique. This is not intended to be a LaTeX tutorial, instead just (1) a use-case thereof, and (2) a template for your own writing.

	% AW - Ordinarily we'd begin by specifying some broad document properties like font-size, page-size, margins, etc. -- We have done this (and much more) for you by creating a 'style file', which the 'documentclass' command references.
\documentclass{sig-alternate}
 
	% AW - These 'usepackage' commands are a way of importing additional LaTeX styles and formattings that aren't part of the 'standard library'
\usepackage{mdwlist}
\usepackage{url}
\usepackage{hyperref}

\begin{document} 

\title{CIS400/401 Project Proposal - The Effectiveness of Using Touch-Input Gestures in Rhythm Games}
\subtitle{Dept. of CIS - Senior Design 2011-2012}
\numberofauthors{3}
\author{
\alignauthor Philip H. Peng \\ \email{pengp@stwing.upenn.edu} \\ Univ. of Pennsylvania \\ Philadelphia, PA
\alignauthor Stephen H. Lane \\ \email{shlane@cis.upenn.edu} \\ Univ. of Pennsylvania \\ Philadelphia, PA}
\date{}
\maketitle

\begin{abstract}
\textit{As touchscreen devices become increasingly popular, new software applications are expected to support touch-focused interfaces for user interaction. This project focuses on the evaluation of the effectiveness of various touch-input gestures in designing rhythm games. This will be accomplished through the development of a touch-input gesture recognition toolkit, the development of rhythm game prototypes using that toolkit, and finally a quantative evaluation of the effectiveness of the touch-input gestures in each prototype through an Android demo app.}
\end{abstract}

\section{Introduction}
\label{sec:intro}
Over the past few years, touchscreen devices have become increasingly common in the consumer market. According to a report in the March 2010 publication of \textit{Information Display}, consumer-device manufacturers are rapidly adopting touch, with revenues increasing \textit{10x} and unit production \textit{3x} faster than the display industry~\cite{information_display}. With the adoption of this new input technology comes the natural expectation of increased software support for new touch-focused interfaces, allowing for more natural, intuitive, and powerful human-device interactions~\cite{hayes_thesis}. \\

One area that touchscreen support can be leveraged is in the design of rhythm games. A \textit{touchscreen} is a specialized display that receives user input through physical contact from a finger or stylus. \textit{Rhythm games} are a genre of music-based games in which the player performs specific actions in response to audio and visual cues. Rhythm games often focus the the player's beat recognition abilities and consequently their timing accuracy, aided through visual patterns that match the rhythm of the song. These visual patterns consist of a series of \textit{note} objects that appear or move across the screen in a manner referred to in this project as the \textit{gameplay mode}. Interaction with these notes would typically involve hand actions occuring in the \textit{hitbox} area, a pre-defined area for interaction. In non-touchscreen rhythm games, such actions may be the pressing of a button; in a touchscreen scenario, such actions would be either \textit{soft button} touches, defined as virtual buttons interacted by through tap, or \textit{touch-input gestures}, defined as touch events with predefined timing and path properties. \\

Experimental analysis of touchscreen gesture designs show that gestures can offer significantly increased reaction times and reduced attention load over soft buttons, especially under circumstances where the physical interaction is not the centre of the user's attention~\cite{bragdon_paper}. In rhythm games, lowering the attention load required for executing the associated note action can result in increased focus on the song's rhythm. Hence, it can be assumed that designing rhythm games to use touch gestures that users find most effective can lead to improved timing accuracy. This project address the question of: when designing rhythm games, which touch-input gestures are the most effective for which gameplay mode.

\section{Project Proposal}
\label{sec:project_proposal}

This project will compare touch-input gestures for rhythm game design through the following three stages:
\begin{enumerate}
	\item \textbf{Development} of a touch-input gesture recognition toolkit for rhythm games\vspace{-3pt}
	\item \textbf{Prototyping} of various rhythm games created using the toolkit\vspace{-3pt}
	\item \textbf{Evaluation} of the prototypes through collecting user feedback\vspace{-3pt}
\end{enumerate}
These three stages will answer the question of which touch-input gestures are most effective in rhythm game design, allowing for future development of new rhythm games. All code and documentation for this project will be hosted at \url{http://code.google.com/p/beats2}.

\subsection{Development}
\label{subsec:project_proposal}
In the \textbf{Development} stage of the project, a touch-input gesture recognition toolkit will be developed for extending the \textit{Unity 3} game development tools. As of the moment, the \textit{Unity 3} scripts handles only simple raw touch events~\cite{unity_input_doc}. The proposed toolkit will extend the \textit{Unity 3} scripts with the following goals:
\begin{itemize*}
	\item Track multiple touch events as either independent events or gestures\vspace{3pt}
	\item Support for pre-defined single-touch gestures including taps, holds, slides, and swipes\vspace{3pt}
	\item Support for pre-defined multi-touch gestures including press-and-tap, pinch, spread, rotate, and spin\vspace{3pt}
	\item Provide an interface for associating gestures with \textit{Unity 3} game objects\vspace{3pt}
\end{itemize*}

\subsection{Prototyping}
\label{subsec:project_proposal}
In the \textbf{Prototyping} stage of the project, the touch-input gesture recognition toolkit will be used to develop rhythm game prototypes. There will be prototypes featuring different gestures for each of the following generalized gameplay modes:
\begin{itemize*}
	\item Scrolling sheet of notes with a set hitbox region, \textit{e.g. Dance Dance Revolution, Beatmania, Guitar Hero}\vspace{3pt}
	\item Scrolling hitbox region with fixed note regions, \textit{e.g. DJMAX Technika, Lumines}\vspace{3pt}
	\item Fullscreen note region with fixed hitbox regions, \textit{e.g. Guitaroo Man, jubeat, Hatsune Miku: Project DIVA}\vspace{3pt}
	\item Fullscreen note region with fullscreen hitbox region, \textit{e.g. Osu! Tatakae! Ouendan!}\vspace{3pt}
\end{itemize*}

\subsection{Evaluation}
\label{subsec:project_proposal}
In the \textbf{Evaluation} stage of the project, the rhythm game prototypes will be packaged together as a playable demo with a built-in user feedback system. This playable demo will be built for the Android 3.0 OS due to the facts that 1) Android 3.0 targets tablets, which usually feature large multi-touch displays~\cite{honeycomb_tablet}, 2) Android tablet use is on the rise~\cite{tablet_use}, and 3) \textit{Unity 3} supports Android 2.1 and higher~\cite{unity_hardware}. This playable demo will be tested by random samples of UPenn students as well as published online through the Android Market. The prototypes will be evaluated on the efficiency of the touch-input gestures with relation to the gameplay modes. The metrics used for measuring efficiency will be a 5-star rating scale on the following properties:
\begin{itemize*}
	\item Natural: Do the touch-input gestures feel intuitive for the gameplay mode?\vspace{3pt}
	\item Reliable: Are the touch-input gestures easy to replicate and repeat?\vspace{3pt}
	\item Reactive: Do the touch-input gestures allow for fast reaction times?\vspace{3pt}
	\item Non-Distracting: Are the touch-input gestures non-distracting to gameplay?\vspace{3pt}
\end{itemize*}
The results of these comparisons may potentially lead to the future development of the more popular prototypes as full rhythm games.

\section{Technical Resources}
\label{sec:resources}

\vspace{4pt}

\noindent \textbf{Test Device} \\
The target touchscreen device for testing in this project will be the Samsung Galaxy Tab 10.1. The Galaxy Tab is an Android 3.0 tablet running on a 1GHz dual-core processor and has a 10.1-inch capacitive touchscreen supporting up to 10 multi-touch points. It also has a built-in vibrating motor, allowing for additional haptic-feedback support by the toolkit. \\

\noindent \textbf{Unity 3} \\
\url{http://unity3d.com/unity/publishing/android.html} \\
The \textit{Unity 3} development tools consists of the \textit{editor}, the series of tools for developing games, and the \textit{game engine}, the software backend that allows the developed games to run on target platforms. \textit{Unity 3} was chosen due to its cross-platform support of other touchscreen-supporting platforms and large community and professional support base. In this project, the toolkit will be developed as script extentions for the editor, while the prototypes will be run using the game engine. The \textit{Unity 3} license required for this project will be the regular \textit{Unity 3} package with the additional Android add-on to allow for development of Android apps. \\

\noindent \textbf{uniTUIO} \\
\url{http://xtuio.com/index.php/projectsmain/utuiomain} \\
\textit{uniTUIO} is a set of \textit{Unity 3} scripts that add support for TUIO based multi-touch input. TUIO is an open framework that defines the common protocol and API for interacting with tangible multi-touch surfaces~\cite{tuio}. Although \textit{Unity 3} uses a different protocol and has its own set of APIs for interacting with the Android touchscreen, the \textit{uniTUIO} scripts may be a good starting point or reference material for developing the toolkit in this project. \\

\noindent \textbf{Android SDK} \\
\url{http://developer.android.com/sdk/} \\
The \textit{Android SDK} is the set of development tools and core libraries required for developing Android applications. The \textit{Unity 3} engine builds on top of the \textit{Android SDK}. \\

\noindent \textbf{Eclipse} \\
\url{http://www.eclipse.org/} \\
\textit{Eclipse} is the standard IDE (Integrated Development Environment) for developing Android apps. Both the \textit{Android SDK} and the \textit{Unity 3} Android add-on are designed to integrate with \textit{Eclipse}. \\

\noindent \textbf{Google Code} \\
\url{http://code.google.com/p/beats2/} \\
\textit{Google Code} is a group of online resources, tools, and hosting for project development. In this project, \textit{Google Code} will be used for hosting the SVN source code, as well as bug tracking and wiki hosting. \\

\noindent \textbf{Google Analytics} \\
\url{http://www.google.com/analytics/} \\
\textit{Google Analytics} is a free online service for tracking usage of features in applications. It will be used in this project to track the user feedback during the \textbf{Evaluation} stage.

\section{Work Plan}
\label{sec:work_plan}

The following is a rough timeline of the various milestones that need to be achieved for this project, along with their estimated completion dates. See the Appendix for Figure~\ref{fig:timeline_fall} for the Gantt chart for the fall semester timeline, Figure~\ref{fig:timeline_spring} for the Gantt chart for the spring semester timeline, and Figure~\ref{fig:timeline_all} for the Gantt chart for the complete project timeline. \\

\vspace{-3pt}\noindent \textbf{1) Setup} - \textit{Nov. 11th}
\vspace{-6pt}\begin{itemize*}
	\item Obtain Unity 3 license with Android add-on
	\item Set up Google Code project
	\item Set up Google Analytics
	\item Set up Eclipse with Android SDK and Unity 3
	\item Create a ''Hello World'' Android app through Unity 3
\end{itemize*}

\vspace{-3pt}\noindent \textbf{2) Investigation} - \textit{Nov. 25th}
\vspace{-6pt}\begin{itemize*}
	\item Investigate the uniTUIO framework
	\item Create a touch-input tracking Android app through Unity 3
	\item Experiment with tracking touch-input features and characteristics
\end{itemize*}

\vspace{-3pt}\noindent \textbf{3) Drafting} - \textit{Dec 9th}
\vspace{-6pt}\begin{itemize*}
	\item Draft specifications for touch gestures
	\item Draft specifications for toolkit components
	\item Create a simple prototype demoing one or two gesture recognitions
\end{itemize*}

\vspace{-3pt}\noindent \textbf{4) Development} - \textit{Feb 17th}
\vspace{-6pt}\begin{itemize*}
	\item Develop touch-input gesture recognition toolkit
	\item Create prototypes demoing each gesture recognitions
\end{itemize*}

\vspace{-3pt}\noindent \textbf{5) Prototyping} - \textit{Mar 16th}
\vspace{-6pt}\begin{itemize*}
	\item Develop simple rhythm game engine based on \textit{Beats}~\cite{beats}
	\item Create rhythm game prototypes for each gameplay mode using different touch-input gestures
\end{itemize*}

\vspace{-3pt}\noindent \textbf{6) Evaluation} - \textit{Apr 13th}
\vspace{-6pt}\begin{itemize*}
	\item Create an app showcasing the rhythm game prototype
	\item Collect user feedback on each prototype through random surveys and market publishing
	\item Evaluate the effectiveness of the different touch-input gestures through quantitive analysis of feedback
\end{itemize*}

\vspace{-3pt}\noindent \textbf{7) Report} - \textit{Apr 20th}
\vspace{-6pt}\begin{itemize*}
	\item Draw conclusions based on results of feedback analysis
	\item Write report summarizing findings
\end{itemize*}

\section{Related Work}
\label{sec:related_work}

\subsection{Touch-Input Rhythm Games}
\label{subsec:related_work}

\noindent \underline{\textbf{Beats, Advanced Rhythm Game}}\vspace{3pt} \\
\textbf{From:} Philip Peng, 2010~\cite{beats}\vspace{3pt} \\
\textbf{Platform:} Android \\
\textbf{Description:} \\
\textit{Beats, Advanced Rhythm game} (\textit{Beats}) is a \textit{Dance Dance Revolution} (\textit{DDR}) simulator for Android devices. \textit{Beats} uses fixed hitbox regions that simulate the \textit{DDR} dance mat, and a scrolling sheet of notes representing \textit{DDR} arrows. \textit{Beats} uses the tap and hold gestures. \\

\noindent \underline{\textbf{DJMAX Technika}}\vspace{3pt} \\
\textbf{From:} Pentavision, 2009~\cite{djmax_technika} \\
\textbf{Platform:} Arcade \\
\textbf{Description:} \\
\textit{DJMAX Technika} is an arcade rhythm game on a machine with a large touchscreen display. Touch notes would appear on screen in fixed regions and a moving hitbar would scroll across the screen. The game uses taps, holds and slides gestures. \\

\noindent \underline{\textbf{jubeat}}\vspace{3pt} \\
\textbf{From:} Konami, 2008~\cite{jubeat} \\
\textbf{Platform:} Arcade \\
\textbf{Description:} \\
\textit{jubeat} is an arcade rhythm game on a machine consisted of a grid of 16 touchscreen boxes that had to be tapped to the rhythm of the music, similar to ''whack-a-mole''. All 16 touchscreen boxes were used, each with individual hitboxes that recognized the tap gesture. \\

\noindent \underline{\textbf{Osu! Tatakae! Ouendan!}}\vspace{3pt} \\
\textbf{From:} iNiS, 2005~\cite{music_games_rock} \\
\textbf{Platform:} Nintendo DS \\
\textbf{Description:} \\
\textit{Osu! Tatakae! Ouendan!} (\textit{Ouendan}), known as \textit{Elite Beat Agents} in North America, is a rhythm game where the player must use the Nintendo DS stylus to interact with note objects that appear on the screen. \textit{Ouendan} uses the full screen for note appearances and uses the tap, slide, and spin gestures. \\

\noindent \underline{\textbf{Taiko No Tatsujin}}\vspace{3pt} \\
\textbf{From:} Namco, 2001~\cite{music_games_rock}\vspace{3pt} \\
\textbf{Platform:} Nintendo DS, iOS \\
\textbf{Description:} \\
The \textit{Taiku No Tatsujin} series focuses on simulating the Japanes Taiko drumming experience. The game has fixed hitbox regions representing two large Taiko drums and uses a scrolling sheet of notes that move toward the hitbox region. It only uses the tap gesture.

\subsection{Touch-Input Toolkits}
\label{subsec:related_work}

\noindent \underline{\textbf{MT4j}}\vspace{3pt} \\
\url{http://www.mt4j.org} \\
\textit{MT4j} (\textit{Multitouch for Java}) is an open source Java framework for designing multi-touch applications. It is a complete framework rather than a toolkit or library, but it can be used as reference. It includes a flexible, customizable multi-touch gesture system that may be adapted in this project. In addition, it includes support for multi-touch in Windows 7, which \textit{Unity 3} currently does not natively support and may be something this project's toolkit may add. \\

\noindent \underline{\textbf{GestureToolkit}}\vspace{3pt} \\
\url{http://gesturetoolkit.codeplex.com} \\
\textit{GestureToolkit} is a toolkit for developing and testing multi-touch applications running on Windows Presentation Foundation or Silverlight. It includes predefined gestures as well as a language for defining new gestures in multi-step, multi-user, or multi-touch scenarios. The toolkit also includes a test framework for validating touch interactions and gesture definitions. The gestures expected to be used with this toolkit are far more complex than those expected to be used in rhythm games, but a test framework may be something that is worth developing alongside this project. \\

\noindent \underline{\textbf{PyMT}}\vspace{3pt} \\
\url{http://pymt.eu/} \\
\textit{PyMT} is an open source pure Python library for developing cross-platform multi-touch applications. It does not, however, support Android or iOS as neither natively support Python. \textit{Unity 3} does, however, support Boo~\cite{unity_scripting}, a dialect of Python, so some code snippets may modified for use in this project.


\subsection{Touch-Input Research}
\label{subsec:related_work}

\noindent \underline{\textbf{NUI Group Community Book}}\vspace{3pt} \\
\url{http://nuicode.com/projects/wiki-book/} \\
The \textit{NUI Group} is a global research group focusing on the \textit{natural user interface} concept, under which touch-input interfaces is included. The \textit{Community Book}, titled \textit{Multitouch Technologies}, is an online e-book summarizing the cummulative research of the \textit{NUI Group} community. While it focuses mainly on vision recognition of objects and interpretting hand gestures, Chapters 2.2 and 2.3 focus on multi-touch tracking and gesture recognition. The book describes the use of the Kalman filter and a k-Nearest Neighbour (k-NN) approach for tracking movements and connecting previous states for gestures, as well as provides guidelines for the gesture recognition techniques.~\cite{nui_book}. This book will be used as a reputable design reference guide throughout this project. \\

\noindent \underline{\textbf{Experimental Analysis of Touch-Screen Gesture}} \\
\noindent \underline{\textbf{Designs in Mobile Environments}}\vspace{3pt} \\
\url{http://yangl.org/pdf/gesturestudy-chi2011.pdf} \\
In their research paper "Experimental Analysis of Touch-Screen Gesture Designs in Mobile Environments", Andrew Bragdon et al. study the impact of environmental distractions on touchscreen interaction. In particular, the paper focuses on different design factors for touch-input gestures compared to soft buttons. The results of their analysis was that gestures can offer significant performance gains and reduced attention load compared to soft buttons, with bezel-initiated gestures offering faster performance and mark-based gestures offering better accuracy~\cite{bragdon_paper}. The results of Andrew Bragdon et al.'s paper can be used for selecting and defining the gestures being tested, while some of the evaluation techniques used in the paper can be used in the Evaluation stage of this project.

\section{Evaluation Criteria}
\label{sec:eval_criteria}
The project will be evaluated based on 1) the usability of the toolkit and its ease of integration with \textit{Unity 3}, 2) the interest level of users trying out the rhythm game prototypes, and 3) the conclusiveness of the gesture comparison results. For 1), the final touch-input gesture recognition toolkit should allow for easy addition of gestures, easy integration with rhythm game engines, and provide an interface similar to those of \textit{Unity 3}. For 2), users should find the rhythm game prototypes distinct, clearly demonstrative of the touch-input gestures that they compare, and fun to try out. For 3), the results should ideally conclude with certain touch-input gestures being predominantly more effective with particular gameplay modes (as opposed to the tap gesture being the most effective for all gameplay modes).

\section{Appendix}
\label{sec:appendix}
\textit{See figures on following pages.}

\bibliographystyle{plain}
\bibliography{prop_spec}

\begin{figure*}[htb!]
	\begin{center}
		\includegraphics[width=1\linewidth]{timeline_fall}
	\end{center}
	\vspace{-12pt}
	\caption{Fall semester timeline}
	\label{fig:timeline_fall}
\end{figure*}

\begin{figure*}[htb!]
	\begin{center}
		\includegraphics[width=1\linewidth]{timeline_spring}
	\end{center}
	\vspace{-12pt}
	\caption{Spring semester timeline}
	\label{fig:timeline_spring}
\end{figure*}

\begin{figure*}[htb!]
	\begin{center}
		\includegraphics[width=1\linewidth]{timeline_all}
	\end{center}
	\vspace{-12pt}
	\caption{Complete project timeline}
	\label{fig:timeline_all}
\end{figure*}

\end{document} 

