Introduction

On Monday, we gave a quick tour of the Javascript Language.
We described the primitive types, arrays, and objects.
We went over Javscript assignment statements, and operators.
We said the control structures of Javascript: if/else, switch/case, for, while, do while were pretty much the same as Java.
We then talked about how to declare and use functions in Javascript.
We said how to use Javascript to manipulate HTML via the DOM.
We then began talking about A-Frame, a framework of custom tags added to HTML via Javascript to make it easy to describe VR scenes.
A-Frame handles the lower level WebGL, WebVR, gamepad api browser calls.
We went over a basic A-Frame example that used a-assets for textures and made use of box, cone, plane and sphere primitives.
We said objects in A-Frame are a-entity's to which various components are added to give new functionality, and that even the primitive types just mentioned are defined this way.
Besides entities and components, A-Frame also includes system objects useful for defining things like camera and controller info.
Today, we begin by looking at some more A-Frame primitives, components, and systems, then briefly talk about how a-entities can be defined.

An Second A-Frame Example

<!DOCTYPE html>
<html>
<head>
<title>Sky Camera Controls</title>
<script src="https://aframe.io/releases/0.8.2/aframe.min.js"></script>
</head>
<body>
<a-scene>
    <a-assets>
        <img id="city" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/city.jpg">
        <img id="city-thumb" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/thumb-city.jpg">
    </a-assets>;
<!-- 360-degree image. -->
<a-sky id="image-360" radius="10" src="#city"></a-sky>
<!-- Image links. -->
<a-link class="link" href="hello.html" title="Hello Page"  on="click"
    position="0 -1 -4"></a-link>
<!-- Camera + cursor. -->
<a-entity camera look-controls>
<a-cursor id="cursor"
    animation__click="property: scale; startEvents: click; from: 0.1
    0.1 0.1; to: 1 1 1; dur: 150"
    animation__fusing="property: fusing; startEvents: fusing; from:
    1 1 1; to: 0.1 0.1 0.1; dur: 1500"
    fuse="true"
    raycaster="objects: .link">
</a-cursor>
</a-entity>
</a-scene>
</body>
</html>

Remarks on Sky and Links Primitives

This example uses an a-sky tag. This is an image on the inside of a sphere of, in this case, radius 10 that makes it look like you are in a city.
We also make use of an a-link tag. This provides a link between one A-Frame scene and another.
In this case, if the a-link is clicked we go to the scene we had from last day.
The title attribute of the a-link is the text that is displayed above it. You can also provide an image attribute to make the disk look different.
To make the link clickable, we first give an entity with a camera component and with a look-controls component.

Remarks on Camera and Controls

The camera component says from what perspective the viewer sees the scene.

If a camera is not specified for a scene a default camera is injected to the scene as:

<a-entity camera="active: true" look-controls wasd-controls 
    position="0 0 0" data-aframe-default-camera></a-entity>

One can have more than one camera in a scene, but only one can be active at a time.

To switch camera, we can use Javascript in response to an event:

var secondCameraEl = document.querySelector('#second-camera');
secondCameraEl.setAttribute('camera', 'active', true);

The position of the entity a camera is on says where in the scene the camera is.
The wasd-controls component says we can control an entity using arrow keyboard keys or using the keys w,a,s,d.
Other control-types include: daydream-controls, gearvr-controls, laser-controls, oculus-touch-controls, windows-motion-controls, etc.
The look-controls component:
- Rotates the entity when we rotate a VR head-mounted display (HMD).
- Rotates the entity when we move the mouse.
- Rotates the entity when we touch-drag the touchscreen.

Remarks on Cursor

The entity on which a camera lives is allowed to have sub-entities.
These will stay in view regardless of where the user looks.
In our case, we are attaching a a-cursor.
By default a cursor looks like a ring. If it is on the entity of a camera, it will appear directly in front of you and can be used to select objects.
The fuse attribute says, if we look at things for more than a certain amount of time, then they are "clicked".
The animation__ attributes can be used to control how appearances changes when an event occurs. To specify it, one gives the property that is being animates, the from: value at the start of the animation and the to: value at the end, followed by its duraction.
The raycaster attribute lets us control those objects which can be clicked. In this case, any object of class "links".

In-Class Exercise

Modify the example given to have two links on our example page.
Also, change the position of the camera to make it higher, and change the fuse animation.
Post your solutions to the Feb 6 In-Class Exercise Thread.

Clickable Objects

Below is an A-frame example in which we use three clickable planes with images on them to control what the sky looks like.

It pretty much comes from the A-Frame site, but where I modified the included libraries so that it works with the Oculus Go.

View Third Example

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>360° Image Gallery</title>
<meta name="description" content="360° Image Gallery - A-Frame">
<script src="https://aframe.io/releases/0.8.2/aframe.min.js"></script>
<script onkeyup="" src="https://unpkg.com/aframe-environment-component/dist/aframe-environment-component.min.js"
    ></script>
<script src="https://unpkg.com/aframe-event-set-component/dist/aframe-event-set-component.min.js"></script>
<script src="https://unpkg.com/aframe-layout-component/dist/aframe-layout-component.min.js"></script>
<script src="https://unpkg.com/aframe-template-component/dist/aframe-template-component.min.js"></script>
</head>
<body>
<a-scene>
    <a-assets>
        <img id="city" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/city.jpg">
        <img id="city-thumb" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/thumb-city.jpg">
        <img id="cubes-thumb" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/thumb-cubes.jpg">
        <img id="sechelt-thumb" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/thumb-sechelt.jpg">
        <audio id="click-sound" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/audio/click.ogg"></audio>
        <img id="cubes" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/cubes.jpg">
        <img id="sechelt" crossorigin="anonymous"
            src="https://cdn.aframe.io/360-image-gallery-boilerplate/img/sechelt.jpg">
        <!-- Image link template to be reused. -->
        <script id="link" type="text/html">
            <a-entity class="link"
                geometry="primitive: plane; height: 1; width: 1"
                material="shader: flat; src: ${thumb}"
                event-set__1="_event: mousedown; scale: 1 1 1"
                event-set__2="_event: mouseup; scale: 1.2 1.2 1"
                event-set__3="_event: mouseenter; scale: 1.2 1.2 1"
                event-set__4="_event: mouseleave; scale: 1 1 1"
                event-set__5="_event: click; _target: #image-360; src: ${src}"
                >
            </a-entity>
        </script>
    </a-assets>
<!-- 360-degree image. -->
<a-sky id="image-360" radius="10" src="#city"></a-sky>
<!-- Image links. -->
<a-entity id="links" layout="type: line; margin: 1.5" position="0 1 -4">
    <a-entity position="-2 0 0" template="src: #link" data-src="#cubes"
        data-thumb="#cubes-thumb" sound="on: click; src: #click-sound"></a-entity>
    <a-entity position="-2 0 0" template="src: #link" data-src="#city"
        data-thumb="#city-thumb" sound="on: click; src: #click-sound"></a-entity>
    <a-entity position="2 0 0" template="src: #link" data-src="#sechelt"
        data-thumb="#sechelt-thumb" sound="on: click; src: #click-sound"></a-entity>
</a-entity>
<!-- Camera + cursor. -->
<a-entity camera look-controls>
<a-cursor id="cursor"
    animation__click="property: scale; startEvents: click; from: 0.1
    0.1 0.1; to: 1 1 1; dur: 150"
    animation__fusing="property: fusing; startEvents: fusing; from:
    1 1 1; to: 0.1 0.1 0.1; dur: 1500"
    event-set__1="_event: mouseenter; color: springgreen"
    event-set__2="_event: mouseleave; color: black "
    fuse="true"
    raycaster="objects: .link">
</a-cursor>
</a-entity>
</a-scene>
</body>
</html>

Remarks on Example

Notice this example imports more scripts than our previous examples.
These let us define template entities and let us use layout components.
A template is a little like a C++ template that can be used to build new entities by filling in missing parameters of information.
Notice we use the template attribute of an a-entity to say which template we want the a-entity to use.
The data- attributes are then used via string interpolation to fill in the parameters needed by the template.
A layout component lays out sub-entities of a entity in a scene in a particular way.
For example, we use a layout of type line to make entities appear in a line.
Other possibilities are: box, circle, cube, dodecahedron, line, pyramid.
Notice in this example we also have a sound resource.
Notice also we are responding to different kinds of events using attributes of the form event-set__some-name.

The generic form of an event set is:

event-set__${id}="_event: ${eventName}; ${someProperty}: ${toValue}"

In particular, it is event-set__5 of the link template that is causing the sky to change when instance of the template is clicked.

Defining New Entities and Components

That completes our first brief introduction to A-Frame.
Before we conclude, we mention how we can make new primitives.
To specify an entity, we need to give it a name and then specify its default components and mappings from these components to attributes.

For example, for the a-box tag this looks like:

var extendDeep = AFRAME.utils.extendDeep;

// The mesh mixin provides common material properties
// for creating mesh-based primitives.
// This makes the material component a default component
// and maps all the base material properties.
var meshMixin = AFRAME.primitives.getMeshMixin();

AFRAME.registerPrimitive('a-box', extendDeep({}, meshMixin, {
  // Preset default components. These components and component 
  //properties will be attached to the entity out-of-the-box.
  defaultComponents: {
    geometry: {primitive: 'box'}
  },

  // Defined mappings from HTML attributes to component properties (using dots as delimiters).
  // If we set `depth="5"` in HTML, then the primitive will automatically set `geometry="depth: 5"`.
  mappings: {
    depth: 'geometry.depth',
    height: 'geometry.height',
    width: 'geometry.width'
  }
}));

Similarly, one can create new components using registerComponent providee one overrides:

AFRAME.registerComponent('foo', {
  schema: {/* properties on component*/},
  init: function () {/* when instatiated*/},
  update: function () {/* when schema changed*/},
  tick: function () {/* called during render loop */},
  remove: function () {/* if delete */},
  pause: function () {/* if no longer rendering*/},
  play: function () {/*if resume */}
});

VR Hardware

We now return to the theory of VR.
To understand how VR works, one needs to consider the VR System
This consists of the computers, controllers, headset, etc., as well as the user.
As we can see above, the VR hardware might not only need to track the user's head position, controller positions, etc., but might also need to track the environment in which the user is.
Using thse, it then needs to provide outputs to the user.
Observe that the user is receiving inputs not only from the VR Hardware, but also, the user's real world environment.

Sensors and sense organs

How does the VR system extract information from the physical world?
In engineering, a transducer refers to a device that converts energy from one form to another.
A sensor is a special transducer that converts energy it receives into a signal for an electrical circuit.
A sensor typically has a receptor that collects energy for conversion.
Organisms work similarly. For organism, a sensor is called a sense organ. For example, eyes and ears.
For organism, the sense organs convert energy ( for the ear, vibrations of the ear drum) into neural impulses.
Engineering sensors for sound operate similarly.
Are example is for the ear and hearing, but this will often be true of other sensors/sense organ pairs as well.

More A-Frame, VR from Hardware to Software

Outline