Outline

• iOS Game Projects
• Quiz
• Touch on the iPhone
• More touch stuff

Introduction

• Last week, we were covering the basics of how Core Graphics works on the iPhone.
• We also looked at how sensors, location, and heading work for both platforms.
• Today, we briefly look at the auto-generated code you get when you try to create an OpenGL ES game in iOS.
• This serves as a starting point to OpenGL on iOS.

Creating an OpenGL ES Application

• One easy place to get started is to create a new iOS project and select game.
• This will you choose from among several methods of creating an iOS game: SceneKit, SpriteKit, OpenGL ES, and Metal.
• You can think of SceneKit as a higher level 3D front-end for OpenGL which comes with its own physics and particle apis, and think Metal as a lower level 3D library. SpriteKit is an API to help you with 2D games.
• Currently, the default iOS SceneKit example consists of a rotating fighter jet.
• The fighter is specified using a dae file. This is in the Collada 3D XML format specified by the same Khronos group responsible for OpenGL ES.
• The only code is in the controller and it basically loads this model and specifies that there should be a runAction rotating the model:

  [ship runAction:[SCNAction repeatActionForever:[SCNAction rotateByX:0 y:2 z:0 duration:1]]];
  

• Kinda cool so I thought I'd mention it -- Now on to the built-in OpenGL example, that I'll talk about in a little more detail.
• This creates an app with two rotating codes in it.
• The storyboard for this app should have as its controller a GameViewController and as its view GLKView object and otherwise all the non-boilerplate code is the GameViewController.m. We write our GameViewController so it is a subclass of GLKViewController.

Understanding the OpenGL ES Example

• The main entry point into the example code is in viewDidLoad:

  - (void)viewDidLoad
  {
      [super viewDidLoad];
      
      self.context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
  
      if (!self.context) {
          NSLog(@"Failed to create ES context");
      }
      
      GLKView *view = (GLKView *)self.view;
      view.context = self.context;
      view.drawableDepthFormat = GLKViewDrawableDepthFormat24;
      
      [self setupGL];
  }
  

• The initWithAPI method above is used to select which version of OpenGL ES we want. As of iOS7 OpenGL ES 3.x is supported but we aren't using anything beyond ES2 (just shaders), hence the above.
• We then set up the color depth of our GLKView and call setupGL to continue with the initialization.

setupGL and tearDownGL

• The example code then has a method setupGL for setting up the OpenGL context.
• Here is the setupGL code (the cube data is defined earlier) in the file:

      [EAGLContext setCurrentContext:self.context];
      
      [self loadShaders];
      
      self.effect = [[GLKBaseEffect alloc] init];
      self.effect.light0.enabled = GL_TRUE;
      self.effect.light0.diffuseColor = GLKVector4Make(1.0f, 0.4f, 0.4f, 1.0f);
      
      glEnable(GL_DEPTH_TEST);
      
      glGenVertexArraysOES(1, &_vertexArray);
      glBindVertexArrayOES(_vertexArray);
      
      glGenBuffers(1, &_vertexBuffer);
      glBindBuffer(GL_ARRAY_BUFFER, _vertexBuffer);
      glBufferData(GL_ARRAY_BUFFER, sizeof(gCubeVertexData), gCubeVertexData, GL_STATIC_DRAW);
      
      glEnableVertexAttribArray(GLKVertexAttribPosition);
      glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, 24, BUFFER_OFFSET(0));
      glEnableVertexAttribArray(GLKVertexAttribNormal);
      glVertexAttribPointer(GLKVertexAttribNormal, 3, GL_FLOAT, GL_FALSE, 24, BUFFER_OFFSET(12));
      
      glBindVertexArrayOES(0);
  

• This loads the shaders, specifies one light location, says _vertexArray will be an array of vertices, its data comes from gCubeVertexData, and then where in that array vertex point info is and where normal to point info is.
• When our app is done we also need to tear down the OpenGL stuff we have set-up above. This is done in tearDownGL which is called from two of our controller's parent methods:

  - (void)dealloc
  {    
      [self tearDownGL];
      
      if ([EAGLContext currentContext] == self.context) {
          [EAGLContext setCurrentContext:nil];
      }
  }
  
  - (void)didReceiveMemoryWarning
  {
      [super didReceiveMemoryWarning];
  
      if ([self isViewLoaded] && ([[self view] window] == nil)) {
          self.view = nil;
          
          [self tearDownGL];
          
          if ([EAGLContext currentContext] == self.context) {
              [EAGLContext setCurrentContext:nil];
          }
          self.context = nil;
      }
  
      // Dispose of any resources that can be recreated.
  }
  

• The tear down code gets the EAGLContext and then deletes our vertex buffers, arrays, and shader program. Here is the tear down code:

      [EAGLContext setCurrentContext:self.context];
      
      glDeleteBuffers(1, &_vertexBuffer);
      glDeleteVertexArraysOES(1, &_vertexArray);
      
      self.effect = nil;
      
      if (_program) {
          glDeleteProgram(_program);
          _program = 0;
      }
  

Shader Handling

• The demo app comes with a vertex shader and a fragment shader in the files: Shader.vsh and Shader.fsh respectively. They look like:

  //
  //  Shader.vsh
  //  MyGame2
  //
  //  Created by Chris Pollett on 11/17/14.
  //  Copyright (c) 2014 Chris Pollett. All rights reserved.
  //
  
  attribute vec4 position;
  attribute vec3 normal;
  
  varying lowp vec4 colorVarying;
  
  uniform mat4 modelViewProjectionMatrix;
  uniform mat3 normalMatrix;
  
  void main()
  {
      vec3 eyeNormal = normalize(normalMatrix * normal);
      vec3 lightPosition = vec3(0.0, 0.0, 1.0);
      vec4 diffuseColor = vec4(0.4, 0.4, 1.0, 1.0);
      
      float nDotVP = max(0.0, dot(eyeNormal, normalize(lightPosition)));
                   
      colorVarying = diffuseColor * nDotVP; // how much color we should see from this vertex
          // this is passed on to fragment shader
      
      gl_Position = modelViewProjectionMatrix * position; //final output position of vertex
  }
  

  //
  //  Shader.fsh
  //  MyGame2
  //
  //  Created by Chris Pollett on 11/17/14.
  //  Copyright (c) 2014 Chris Pollett. All rights reserved.
  //
  
  varying lowp vec4 colorVarying;
  
  void main()
  {
      /* fragment shader just directly uses interpolated color
         from vertex without any further processing
         gl_FragColor is the final output color
       */
      gl_FragColor = colorVarying; 
  }
  

• The demo code in GameViewController.m then comes with code to read in these files, compile, and link them:

  #pragma mark -  OpenGL ES 2 shader compilation
  
  - (BOOL)loadShaders
  {
      GLuint vertShader, fragShader;
      NSString *vertShaderPathname, *fragShaderPathname;
      
      // Create shader program.
      _program = glCreateProgram();
      
      // Create and compile vertex shader.
      vertShaderPathname = [[NSBundle mainBundle] pathForResource:@"Shader" ofType:@"vsh"];
      if (![self compileShader:&vertShader type:GL_VERTEX_SHADER file:vertShaderPathname]) {
          NSLog(@"Failed to compile vertex shader");
          return NO;
      }
      
      // Create and compile fragment shader.
      fragShaderPathname = [[NSBundle mainBundle] pathForResource:@"Shader" ofType:@"fsh"];
      if (![self compileShader:&fragShader type:GL_FRAGMENT_SHADER file:fragShaderPathname]) {
          NSLog(@"Failed to compile fragment shader");
          return NO;
      }
      
      // Attach vertex shader to program.
      glAttachShader(_program, vertShader);
      
      // Attach fragment shader to program.
      glAttachShader(_program, fragShader);
      
      // Bind attribute locations.
      // This needs to be done prior to linking.
      glBindAttribLocation(_program, GLKVertexAttribPosition, "position");
      glBindAttribLocation(_program, GLKVertexAttribNormal, "normal");
      
      // Link program.
      if (![self linkProgram:_program]) {
          NSLog(@"Failed to link program: %d", _program);
          
          if (vertShader) {
              glDeleteShader(vertShader);
              vertShader = 0;
          }
          if (fragShader) {
              glDeleteShader(fragShader);
              fragShader = 0;
          }
          if (_program) {
              glDeleteProgram(_program);
              _program = 0;
          }
          
          return NO;
      }
      
      // Get uniform locations.
      uniforms[UNIFORM_MODELVIEWPROJECTION_MATRIX] = glGetUniformLocation(_program, "modelViewProjectionMatrix");
      uniforms[UNIFORM_NORMAL_MATRIX] = glGetUniformLocation(_program, "normalMatrix");
      
      // Release vertex and fragment shaders.
      if (vertShader) {
          glDetachShader(_program, vertShader);
          glDeleteShader(vertShader);
      }
      if (fragShader) {
          glDetachShader(_program, fragShader);
          glDeleteShader(fragShader);
      }
      
      return YES;
  }
  
  - (BOOL)compileShader:(GLuint *)shader type:(GLenum)type file:(NSString *)file
  {
      GLint status;
      const GLchar *source;
      
      source = (GLchar *)[[NSString stringWithContentsOfFile:file encoding:NSUTF8StringEncoding error:nil] UTF8String];
      if (!source) {
          NSLog(@"Failed to load vertex shader");
          return NO;
      }
      
      *shader = glCreateShader(type);
      glShaderSource(*shader, 1, &source, NULL);
      glCompileShader(*shader);
      
  #if defined(DEBUG)
      GLint logLength;
      glGetShaderiv(*shader, GL_INFO_LOG_LENGTH, &logLength);
      if (logLength > 0) {
          GLchar *log = (GLchar *)malloc(logLength);
          glGetShaderInfoLog(*shader, logLength, &logLength, log);
          NSLog(@"Shader compile log:\n%s", log);
          free(log);
      }
  #endif
      
      glGetShaderiv(*shader, GL_COMPILE_STATUS, &status);
      if (status == 0) {
          glDeleteShader(*shader);
          return NO;
      }
      
      return YES;
  }
  
  - (BOOL)linkProgram:(GLuint)prog
  {
      GLint status;
      glLinkProgram(prog);
      
  #if defined(DEBUG)
      GLint logLength;
      glGetProgramiv(prog, GL_INFO_LOG_LENGTH, &logLength);
      if (logLength > 0) {
          GLchar *log = (GLchar *)malloc(logLength);
          glGetProgramInfoLog(prog, logLength, &logLength, log);
          NSLog(@"Program link log:\n%s", log);
          free(log);
      }
  #endif
      
      glGetProgramiv(prog, GL_LINK_STATUS, &status);
      if (status == 0) {
          return NO;
      }
      
      return YES;
  }
  
  - (BOOL)validateProgram:(GLuint)prog
  {
      GLint logLength, status;
      
      glValidateProgram(prog);
      glGetProgramiv(prog, GL_INFO_LOG_LENGTH, &logLength);
      if (logLength > 0) {
          GLchar *log = (GLchar *)malloc(logLength);
          glGetProgramInfoLog(prog, logLength, &logLength, log);
          NSLog(@"Program validate log:\n%s", log);
          free(log);
      }
      
      glGetProgramiv(prog, GL_VALIDATE_STATUS, &status);
      if (status == 0) {
          return NO;
      }
      
    
  

GLKView and GLKViewController delegate methods

• Our GameViewController class is a subclass GLKViewController and so needs an update method to say how things should update in a scene.
• The rate at which update is called is determined by the preferredFramesPerSecond property of this class. It defaults to 30.
• The actual drawing of frames is done by the GLKView's (void)glkView:(GLKView *)view drawInRect:(CGRect)rect method.
• Here is the code for each of these methods:

  - (void)update
  {
      float aspect = fabsf(self.view.bounds.size.width / self.view.bounds.size.height);
      GLKMatrix4 projectionMatrix = GLKMatrix4MakePerspective(GLKMathDegreesToRadians(65.0f), aspect, 0.1f, 100.0f);
      
      self.effect.transform.projectionMatrix = projectionMatrix;
      
      GLKMatrix4 baseModelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, -4.0f);
      baseModelViewMatrix = GLKMatrix4Rotate(baseModelViewMatrix, _rotation, 0.0f, 1.0f, 0.0f);
      
      // Compute the model view matrix for the object rendered with GLKit
      GLKMatrix4 modelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, -1.5f);
      modelViewMatrix = GLKMatrix4Rotate(modelViewMatrix, _rotation, 1.0f, 1.0f, 1.0f);
      modelViewMatrix = GLKMatrix4Multiply(baseModelViewMatrix, modelViewMatrix);
      
      self.effect.transform.modelviewMatrix = modelViewMatrix;
      
      // Compute the model view matrix for the object rendered with ES2
      modelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, 1.5f);
      modelViewMatrix = GLKMatrix4Rotate(modelViewMatrix, _rotation, 1.0f, 1.0f, 1.0f);
      modelViewMatrix = GLKMatrix4Multiply(baseModelViewMatrix, modelViewMatrix);
      
      _normalMatrix = GLKMatrix3InvertAndTranspose(GLKMatrix4GetMatrix3(modelViewMatrix), NULL);
      
      _modelViewProjectionMatrix = GLKMatrix4Multiply(projectionMatrix, modelViewMatrix);
      
      _rotation += self.timeSinceLastUpdate * 0.5f;
  }
  
  - (void)glkView:(GLKView *)view drawInRect:(CGRect)rect
  {
      glClearColor(0.65f, 0.65f, 0.65f, 1.0f);
      glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
      
      glBindVertexArrayOES(_vertexArray);
      
      // Render the object with GLKit
      [self.effect prepareToDraw];
      
      glDrawArrays(GL_TRIANGLES, 0, 36);
      
      // Render the object again with ES2
      glUseProgram(_program);
      
      glUniformMatrix4fv(uniforms[UNIFORM_MODELVIEWPROJECTION_MATRIX], 1, 0, _modelViewProjectionMatrix.m);
      glUniformMatrix3fv(uniforms[UNIFORM_NORMAL_MATRIX], 1, 0, _normalMatrix.m);
      
      glDrawArrays(GL_TRIANGLES, 0, 36);
  }
  

Quiz

Touch Event Terms

Responder Chain

• Gesture get passed through the iPhone OS as events and get passed through the responder chain.
• Any class that has UIResponder as a super class is a responder. (For example, all UIView, UIController, UIViewController).
• If the first responder for an event doesn't handle a particular event, it passes that event up the responder chain.
• A responder can also both partially handle an event and still pass it along the chain, or it can choose to handle it and stop the percolation.
• The event chain proceeds up the view hierarchy for view that contained the events coordinates, until finally it reaches UIApplication.
• If this does not handle the event, then the event is discarded.

Forwarding Events

• Events aren't forwarded by default.
• Suppose your view responds to a swipe but not a tap.
• However, some enclosing view might respond to a tap.
• In order to forward, in your event you might have code like:

  -(void)respondToMyEvent: (UIEvent *) event {
     if(condition) {
        [self handleEvent: event];
     } else {
        [self.nextResponder respondToMyEvent: event];
     }
  }
  

Multitouch Architecture

• You can put your code to handle gestures in either a UIView object or in your UIViewController
• Examples of the former are things like UISwitch and UISlider.
• A switch might want to turn itself on or off based on a touch.
• However, if the gesture being processed affects more than the object being touched one typically puts the code in the view's controller class.

Gesture Notification

• There are four methods used to notify a responder about touches and gestures.
• When the user first touches the screen, the iPhone looks for a responder that has a method of signature touchesBegan:withEvent.
• Code for such a method might look like:

  - (void)touchesBegan: (NSSet *)touches withEvent:(UIEvent *)event {
    NSUInteger numTaps = [[touches anyObject] tapCount];
    NSUInteger numTouches = [touches count]; 
  }
  

• touches is a set of UITouch's, each represents one finger touching the screen.
• If there are more than one numTouches is more than 1 than numtaps == 1 as taps are defined as only involving one finger.
• If taps were 2 than it would correspond to a double tap.

More on Gesture Notification

• It is often useful to know about those touches which occurred only within the current view.
• You could find this out with code like:

  NSSet *myTouches = [event touchesForView:self.view];
  

• To find out where a touch occurred, given a UITouch object we can do:

  CGPoint point = [touch locationInView:self];
  

• In addition to static touches one can get notified while a user is moving fingers across the screen by implementing, touchesMoved:withEvent:
• This method will actually get called multiple times during a long drag event, and each time it is called you get another events which you can handle.
• UITouch objects also hold the previous location of a touch.

Still More on Gesture Notification

• You can find out when a finger is removed from the screen by having the method touchesEnded:withEvent:
• Finally, there is also a touchesCancelled:withEvent:
• This might be called if a user is touching the screen and it gets interrupted by something like an incoming phone call.

Touch Application

• We will now create a simple app to experiment with touch events on the iPhone.
• It will display a message saying what kind of touch event has occurred, and it will also say how many touches or taps there were.
• The touch events are better demo'd on an actual device than in the simulator.
• For this new project we use the Single-View App template.

View Controller Header

• We then modify the view controller header as follows:

  #import <UIKit/UIKit.h>
  
  @interface ViewController : UIViewController
  @property (weak, nonatomic) IBOutlet UILabel *messageLabel;
  @property (weak, nonatomic) IBOutlet UILabel *tapsLabel;
  @property (weak, nonatomic) IBOutlet UILabel *touchesLabel;
  
  -(void)updateLabelsFromTouches: (NSSet *)touches;
  @end
  

Changes to TouchApp Storyboard file

• We next modify the view controller's storyboard file. We turn off Use Autolayout for the controller and set it be just for iPhone. The we add three labels.
• We center the labels and change the text to Messages, Taps, Touches.
• We control drag from the File's Owner to each of the labels and either select, the messagesLabel outlet, tapsLabel outlet, or touchesLabel outlet.
• Finally, we click on the view in interface builder and click on the View Attributes tab.
• We make sure that under the Interaction settings both User Interaction Enabled and Multiple Touch are checked.

View Controller Code

• The view controller's code is then set up as follows:

  #import "TouchAppViewController.h"
  
  @implementation TouchAppViewController
  
  -(void) updateLabelsFromTouches:(NSSet *)touches {
      NSUInteger numTaps = [[touches anyObject] tapCount];
      NSString *tapsMessage = [[NSString alloc]
                               initWithFormat:@"%lu taps detected", (unsigned long)numTaps];
      _tapsLabel.text = tapsMessage;
  
      NSUInteger numTouches = [touches count];
      NSString *touchMsg = [[NSString alloc] initWithFormat:
                            @"%lu touches detected", (unsigned long)numTouches];
      _touchesLabel.text = touchMsg;
  }
  - (void)viewDidLoad {
      [super viewDidLoad];
      // Do any additional setup after loading the view, typically from a nib.
  }
  
  - (void)didReceiveMemoryWarning {
      [super didReceiveMemoryWarning];
      // Dispose of any resources that can be recreated.
  }
  
  
  #pragma mark -
  - (void) touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
      _messageLabel.text = @"Touches Began";
      [self updateLabelsFromTouches:touches];
  }
  
  - (void) touchesCancelled:(NSSet *)touches withEvent:(UIEvent *)event {
      _messageLabel.text = @"Touches Cancelled";
      [self updateLabelsFromTouches:touches];
  }
  
  - (void) touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
      _messageLabel.text = @"Touches Ended";
      [self updateLabelsFromTouches:touches];
  }
  
  - (void) touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
      _messageLabel.text = @"Drag detected";
      [self updateLabelsFromTouches:touches];
  }
  
  @end
  

Touch App Screenshot

Handling More Complicated Touch Events

• To handle more complicated events on the iPhone such as swipes, pinches, and proper handling of multiple taps we use the basic structure we've already created and make a note of where things happen on the screen.
• We say something is a swipe in some direction based on the coordinate where the finger first touched the screen and based on the coordinate on which it was released from the screen.
• Similarly, for a pinch we check that two touches were registered on the screen and their starting coordinates are farther apart then their ending coordinates.
• To check if something is a double tap, but not a triple tap, you can use methods like:

  
  case 2: //double tap
  [NSObject cancelPreviousPerformRequestWithTarget:self
     selector:@selector(singleTap) object:nil];
  
  [self performSelector:@selector(doubleTap) withObject:nil afterDelay: .4]
  break;
  
  case 3: //triple tap
  [NSObject cancelPreviousPerformRequestWithTarget:self
     selector:@selector(doubleTap) object:nil];
  
  [self performSelector:@selector(tripleTap) withObject:nil afterDelay: .4]
  break;
  
  

  in touchesBegan.
• More complicated gestures like making a check-mark on the screen are similarly constructed out of the basic touch events
• We conclude our discussion of iPhone Touch Events by giving an example of how to do swipes.

Swipe Example

• To begin we create a new view-based application.
• This application will display in a label whether a vertical or horizontal swipe was detected.
• We modify the boiler-plate view controller header to look like:

  #import <UIKit/UIKit.h>
  
  #define kMinimumGestureLength  25
  #define kMaximumVariance 5
  
  @interface SwipeAppViewController : UIViewController 
  @property (weak, nonatomic) IBOutlet UILabel *label;
  @property CGPoint gestureStartPoint;
  -(void)eraseText;
  @end
  

• Next we edit the view controller's stroyboard file and add a label and ctrl-click from the File's Owner to the label and connect it to our outlet.

Swipe Controller Code

#import "SwipeAppViewController.h"

@implementation SwipeAppViewController

-(void)eraseText {
   label.text = @"";
}


- (void)didReceiveMemoryWarning {
	// Releases the view if it doesn't have a superview.
    [super didReceiveMemoryWarning];
	
	// Release any cached data, images, etc that aren't in use.
}

- (void)viewDidUnload {
	// Release any retained subviews of the main view.
	// e.g. self.myOutlet = nil;
}


- (void)dealloc {
	[label release];
    [super dealloc];
}

#pragma mark -
-(void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
	UITouch *touch = [touches anyObject];
	gestureStartPoint = [touch locationInView:self.view];
}

-(void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
	UITouch *touch = [touches anyObject];
	CGPoint currentPosition = [touch locationInView:self.view];
	
	CGFloat deltaX = fabsf(gestureStartPoint.x - currentPosition.x);
	CGFloat deltaY = fabsf(gestureStartPoint.y - currentPosition.y);
	
	if(deltaX >= kMinimumGestureLength && deltaY <= kMaximumVariance) {
		label.text = @"Horizontal swipe detected";
		[self performSelector:@selector(eraseText) withObject:nil afterDelay: 2];
		
	} else 	if(deltaY >= kMinimumGestureLength && deltaX <= kMaximumVariance) {
		label.text = @"Vertical swipe detected";
		[self performSelector:@selector(eraseText) withObject:nil afterDelay: 2];
	}
}

@end

Android Key and Touch Events

• We have already seen that in Android we can create a class which implements View.OnClickListener and then set use a View's setOnClickListener method to get this class to handle click events for that view.
• Similarly, you can set a View's OnKeyListener with an object of a class implementing View.OnKeyListener to handle KeyEvents.
• To implement this interface you have to define a method:

  boolean onKey(View v, int keyCode, KeyEvent event)
  

  the key code tells you what key was pressed.
• KeyEvent's have predefined constants for the different keyCode's as well as methods to figure out keyboard state (was shift pressed? etc).
• A View also can have a OnTouchListener which is an Object of a class implementing View.OnTouchListener.
• To implement this class, you need to implement the method:

  boolean onTouch(View v, MotionEvent event)
  

• MotionEvent has a method getAction() which returns an int from a list of constants which says what kind of touch event it was.
• The event actually holds an array of x, y coordinates associated with the event and has member functions getHistoricalX(int i), getHistoricalY(int i), getHistorySize() for accessing these.
• The most recent points can be obtained with getX() and getY().