Stream Query Repository: Immersive Environments

Schema

Introduction

This page provides schema and query specifications from an Immersive Environment. These environments allow a user to become immersed within an augmented or virtual reality environment in order to interact with people, objects, places, and databases. The schema and queries were provided by members of the USC-AIMS project.

Immersive Classroom Application

Within Immersive Classroom, a 3-D immersive environment, a group of normal and ADHD-diagnosed (Attention Deficit Hyperactivity Disorder) children are subjects to do particular attention tasks. The objective of the application is to differentiate between these two groups of users by analyzing their interactions with the environment. The virtual environment consists of a typical classroom containing student desks, a teacher's desk, a virtual teacher, a blackboard, a large window looking out onto a playground with buildings, vehicles, and people, and a pair of doorways on each end of the wall opposite the window through which activity occurs.

A typical task consists of alphabetical characters being displayed on the blackboard and having the child press a button when a particular pattern is seen. In one of the designed tasks, AX task, users are instructed to press mouse button as quickly as possible upon detecting an X after an A (a hit) and withhold their responses to any other pattern. At the same time, a series of typical classroom distractions are systematically manipulated within the environment (i.e., ambient classroom noise, paper airplane flying around the room, students walking into the room, activity occurring outside the window). During the test, the trackers placed on the head, hands and legs monitor body movements of the child and stream the data continuously.

Schema

The schema consists of three relations - User, StaticObject, and MovingObject, and four streams - Characters, UserResponse, UserBodyPositions, and MovingObjectPositions shown below.

User: Relation containing information about users who are experiencing the environment.

User(sid          /* unique identifier */,
     name         /* name of the user */,
     age          /* age of the user */,
     gender       /* gender of the user */,
     description  /* description of the user,
		     e.g., her performance on previous experiments */);

StaticObject: Relation containing geometry of scene objects that constitute the virtual space.

StaticObject(soid	/* unique identifier */,
	     name	/* name of the object */,
	     type	/* keeps object's presence type such as
			   visual, audio or audio-visual */
    	     position   /* 3-d placement of the object in the reference 3-d coordinate
			   space, can be represented as a 3-d point (x, y, z) or
			/* a minimum bounding box (MBR) */);

MovingObject: Relation containing information about dynamic moving objects in the immersive classroom.

MovingObject(aoid    /* unique identifier */,
	     name    /* name of the object */,
	     type    /* maintains the object's presence type such as
		       visual, audio or audio-visual */);

Characters: Stream of characters which are being displayed on the blackboard.

Characters(cid		/* unique identifier for the character, can be a sequence number */,
	   value   	/* ASCII value of the character */,
	   timestamp	/* time of displaying the character */);

UserResponse: Stream of user mouse button clicks.

UserResponse(sid	   /* id of the user. References User.sid */,
             timestamp     /* time of click */);

UserBodyPositions: Stream of positions for each part of user body; they are generated by trackers attached to her body in a sensing system.

UserBodyPositions(sid     	/* id of the user . References User.sid */,
		  partID    	/* id of the tracker attached to one part of her body */,
	          position	/* position of the part in the reference space */,
	          rotation  	/* rotation of the part towards each axis in the coordinate space */, 
	          timestamp  	/* time when the position data aqquired */);

MovingObjectPositions: Stream of position data for each moving object. A special value for position can be specified to represent object disappearance. Non-visual objects are specified by position of their source (e.g., speaker position for audio objects).
```
MovingObjectPositions(aoid     	 /* the object identifier. References MovingObject.aoid */
		      position	 /* position of the object in the reference space */,    
		      timestamp  /* time when the position has been monitored */); 
```

Queries

Queries in English

Queries in English

Hits Query: For each user, display the number of mouse clicks following each AX pattern in characters. Click should be performed prior to displaying the character that follows X on the blackboard.
Average Response Time Query: For each user, report the average response time of all hits. Response time for each hit is the time between displaying character X and click time.
Body Movement Query: For each male user, report average speed of hand movements during all hits.
User Status Query: Which user is shaking during the test. We can define a common frequent pattern of movement among all body parts as shaking.
Distraction Query: For each pattern that a user missed, show which distractions were around.
Attention Query 1: For each pattern that a user missed, show if she was looking at a distractor instead of the blackboard .
Attention Query 2: Monitor the user attention to each static object in the past 2 minutes. Update the results every minute. Attention can be reported as the times she was looking at the object vs. the times she was looking at the blackboard.
Gesture Recognition Query: When the user raises her hand, pause the task.
Nearest Neighbor Query: Every 2 minutes return the 3 closest static/moving object to the subject.
Touch Query: Every 1 minute return the static object which the user has touched more in the past 3 minutes.
Monitor New Moving Objects Query: Find moving objects that have been around within 2 minutes period prior to the first miss of each user.

Last modified: Feb 6 2003. Please send comments and questions to shivnath@stanford.edu