Menu
Amazon AppStream
Developer Guide

This documentation is for an older version of Amazon AppStream. For information about the latest version, see the Amazon AppStream 2.0 Developer Guide.

Sending Your Client Inputs to the Application

The client can send keyboard, mouse, touch, or raw inputs to the application. The client sends an input by filling out the XStxInputEvent structure that describes the input and then calling a function that passes the structure to the application.

Keyboard

Before the client can fill out the XStxInputEvent structure, the keyboard input must first be converted to a Windows keyboard input from an OS X keyboard input. In the processKeyEvent function is the call to getConvertedKeyUsingCharacter which converts the keyboard input to a Windows keyboard input.

The following excerpt from the example client in the Amazon AppStream SDK illustrates the function that converts the OS X keyboard input to a Windows keyboard input. The excerpt is from the getConvertedKeyUsingCharacter function in AppStreamSampleClientWindowController.m.

Copy
(int) getConvertedKeyUsingCharacter: (NSEvent *) theEvent withModifierMask: (uint16_t &) modifierKeyMask withModifierKeyIgnoreMask: (uint16_t &) modifierKeysIgnoreMask { //Get the characters from the key being pressed NSString *theKeyString = [theEvent charactersIgnoringModifiers]; if ([theKeyString length] <= 0) { //Reject dead keys return -1; } //Get the characters as a char uint16_t theChar = [theKeyString characterAtIndex:0]; //First check the generic key mapping, this should convert any non-system // specific keys bool didMapKey = getVirtualKeyUsingChar(theChar, modifierKeyMask, modifierKeysIgnoreMask); if (!didMapKey) { //The generic key mapping didn't map the key so check the platform specific ones didMapKey = appleGetVirtualKeyUsingChar(theChar, modifierKeyMask, modifierKeysIgnoreMask); } if (!didMapKey) { //Neither the generic nor the system specific mapping could remap the key NSLog(@"\n======== Unmapped KEY! ========="); unichar upperKeyChar = [[theKeyString uppercaseString] characterAtIndex:0]; NSString *modKeyString = [theEvent characters]; NSLog(@"Key: %hu String: %@ Mod: %@", upperKeyChar, theKeyString, modKeyString); return -1; } return theChar; }

The following excerpt from the example client in the Amazon AppStream SDK illustrates the function that processes the keyboard action from the client. The excerpt is from the processKeyEvent function in AppStreamSampleClientWindowController.m.

Copy
(void) processKeyEvent: (NSEvent *) theEvent { //Depending on your application you might want to block repeat keys // if ([theEvent isARepeat]) { // //Ignore repeats // return; // } //Setup a couple bitmasks for the command keys uint16_t modifierKeyMask = 0; uint16_t modifierKeysIgnoreMask = 0; int newKey = [self getConvertedKeyUsingCharacter:theEvent withModifierMask:modifierKeyMask withModifierKeyIgnoreMask:modifierKeysIgnoreMask]; if (newKey < 0) { //Not a valid key mapping just return return; } BOOL needAlt = (modifierKeyMask & STX_ALT_KEY_MASK) == STX_ALT_KEY_MASK; BOOL needControl = (modifierKeyMask & STX_CONTROL_KEY_MASK) == STX_CONTROL_KEY_MASK; BOOL needShift = (modifierKeyMask & STX_SHIFT_KEY_MASK) == STX_SHIFT_KEY_MASK; BOOL needWin = (modifierKeyMask & STX_WINDOWS_KEY_MASK) == STX_WINDOWS_KEY_MASK; //Check the status of the command keys if ((modifierKeysIgnoreMask & STX_SHIFT_KEY_MASK) != STX_SHIFT_KEY_MASK) { if (needShift && !shiftIsDown) { //Need shift pressed but it isn't [[AppStreamClient sharedClient] sendKeyDown:VK_SHIFT]; } else if (!needShift && shiftIsDown) { //Need shift released but it is pressed [[AppStreamClient sharedClient] sendKeyUp:VK_SHIFT]; } } if ((modifierKeysIgnoreMask & STX_ALT_KEY_MASK) != STX_ALT_KEY_MASK) { if (needAlt && !altIsDown) { [[AppStreamClient sharedClient] sendKeyDown:VK_MENU]; } else if (!needAlt && altIsDown) { [[AppStreamClient sharedClient] sendKeyUp:VK_MENU]; } } if ((modifierKeysIgnoreMask & STX_CONTROL_KEY_MASK) != STX_CONTROL_KEY_MASK) { if (needControl && !controlIsDown) { [[AppStreamClient sharedClient] sendKeyDown:VK_CONTROL]; } else if (!needControl && controlIsDown) { [[AppStreamClient sharedClient] sendKeyUp:VK_CONTROL]; } } if ((modifierKeysIgnoreMask & STX_WINDOWS_KEY_MASK) != STX_WINDOWS_KEY_MASK) { if (needWin && !windowsIsDown) { [[AppStreamClient sharedClient] sendKeyDown:VK_LWIN]; } else if (!needWin && windowsIsDown) { [[AppStreamClient sharedClient] sendKeyUp:VK_LWIN]; } } //Now send the actual keydown event if (theEvent.type == NSKeyDown) { [[AppStreamClient sharedClient] sendKeyDown:newKey]; } else { //Not keyDown so must be keyUp [[AppStreamClient sharedClient] sendKeyUp:newKey]; } //Reset the command keys state if ((modifierKeysIgnoreMask & STX_SHIFT_KEY_MASK) != STX_SHIFT_KEY_MASK) { if (needShift && !shiftIsDown) { //Need shift pressed but it isn't [[AppStreamClient sharedClient] sendKeyUp:VK_SHIFT]; } else if (!needShift && shiftIsDown) { //Need shift released but it is pressed [[AppStreamClient sharedClient] sendKeyDown:VK_SHIFT]; } } if ((modifierKeysIgnoreMask & STX_ALT_KEY_MASK) != STX_ALT_KEY_MASK) { if (needAlt && !altIsDown) { [[AppStreamClient sharedClient] sendKeyUp:VK_MENU]; } else if (!needAlt && altIsDown) { [[AppStreamClient sharedClient] sendKeyDown:VK_MENU]; } } if ((modifierKeysIgnoreMask & STX_CONTROL_KEY_MASK) != STX_CONTROL_KEY_MASK) { if (needControl && !controlIsDown) { [[AppStreamClient sharedClient] sendKeyUp:VK_CONTROL]; } else if (!needControl && controlIsDown) { [[AppStreamClient sharedClient] sendKeyDown:VK_CONTROL]; } } if ((modifierKeysIgnoreMask & STX_WINDOWS_KEY_MASK) != STX_WINDOWS_KEY_MASK) { if (needWin && !windowsIsDown) { [[AppStreamClient sharedClient] sendKeyUp:VK_LWIN]; } else if (!needWin && windowsIsDown) { [[AppStreamClient sharedClient] sendKeyDown:VK_LWIN]; } } }

After converting the keyboard input, the client populates the XStxInputEvent structure. The structure is then sent to AppStream.

The following excerpt from the example client in the Amazon AppStream SDK illustrates a keyboard action that populates the structures and then sending that structure to the application. The excerpt is from the XStxModule::keyPress function in XStxModule.cpp.

Copy
void XStxModule::keyPress(int key, bool down) { XStxInputEvent xstxevent = { 0 }; xstxevent.mTimestampUs = mud::TimeVal::mono().toMilliSeconds(); xstxevent.mType = XSTX_INPUT_EVENT_TYPE_KEYBOARD; xstxevent.mInfo.mKeyboard.mVirtualKey = key; xstxevent.mInfo.mKeyboard.mIsKeyDown = down; xstxevent.mSize = sizeof(XStsxInputEvent); sendInput(xstxevent); }

Mouse

The following excerpt from the example client in the Amazon AppStream SDK illustrates how the mouse actions are sent to the structure. The excerpt is from AppStreamSampleClientWindowControll.m.

Copy
(void) mouseDown:(NSEvent *)theEvent { NSPoint theLoc = [theEvent locationInWindow]; theLoc.y = _glView.bounds.size.height - theLoc.y; [[AppStreamClient sharedClient] sendMouseEvent:theLoc flags:RI_MOUSE_BUTTON_1_DOWN]; } (void) mouseUp:(NSEvent *)theEvent { NSPoint theLoc = [theEvent locationInWindow]; theLoc.y = _glView.bounds.size.height - theLoc.y; [[AppStreamClient sharedClient] sendMouseEvent:theLoc flags:RI_MOUSE_BUTTON_1_UP]; } (void) mouseDragged:(NSEvent *)theEvent { NSPoint theLoc = [theEvent locationInWindow]; theLoc.y = _glView.bounds.size.height - theLoc.y; [[AppStreamClient sharedClient] sendMouseEvent:theLoc flags:0]; } (void) rightMouseDown:(NSEvent *)theEvent { NSPoint theLoc = [theEvent locationInWindow]; theLoc.y = _glView.bounds.size.height - theLoc.y; [[AppStreamClient sharedClient] sendMouseEvent:theLoc flags:RI_MOUSE_BUTTON_2_DOWN]; } (void) rightMouseUp:(NSEvent *)theEvent { NSPoint theLoc = [theEvent locationInWindow]; theLoc.y = _glView.bounds.size.height - theLoc.y; [[AppStreamClient sharedClient] sendMouseEvent:theLoc flags:RI_MOUSE_BUTTON_2_UP]; } (void) rightMouseDragged:(NSEvent *)theEvent { NSPoint theLoc = [theEvent locationInWindow]; theLoc.y = _glView.bounds.size.height - theLoc.y; [[AppStreamClient sharedClient] sendMouseEvent:theLoc flags:0]; } (void) mouseMoved:(NSEvent *)theEvent { NSPoint theLoc = [theEvent locationInWindow]; theLoc.y = _glView.bounds.size.height - theLoc.y; [[AppStreamClient sharedClient] sendMouseEvent:theLoc flags:0]; }

The client then populates the XStxInputEvent structure. The structure is then sent to AppStream.

The following excerpt from the example client in the Amazon AppStream SDK illustrates a mouse action that populates the structures and then sends that structure to the application. The excerpt is from the XStxModule::mouseEvent function in XStxModule.cpp.

Copy
void XStxModule::mouseEvent(int x, int y, uint32_t flags) { // no video renderer? Return! if (!mVideoRenderer) { return; } int xOff, yOff; float scale; mVideoRenderer->getScaleAndOffset(scale, xOff, yOff); // no scale yet; ignore! if (scale == 0) return; XStxInputEvent xstxevent = { 0 }; xstxevent.mTimestampUs = mud::TimeVal::mono().toMilliSeconds(); xstxevent.mType = XSTX_INPUT_EVENT_TYPE_MOUSE; xstxevent.mInfo.mMouse.mButtonFlags = flags; xstxevent.mInfo.mMouse.mFlags = 1; //absolute static int lastX = 0, lastY = 0; if (flags & CET_MOUSE_WHEEL) { LOGV("Mouse wheel data: %d", x); // mouse wheel data goes in mButtonData. xstxevent.mInfo.mMouse.mButtonData = x; xstxevent.mInfo.mMouse.mLastX = lastX; xstxevent.mInfo.mMouse.mLastY = lastY; } else { lastX = xstxevent.mInfo.mMouse.mLastX = (int)((x + xOff) * scale); lastY = xstxevent.mInfo.mMouse.mLastY = (int)((y + yOff) * scale); // LOGV("Mouse data: x:%d,y:%d", lastX, lastY); } sendInput(xstxevent); }

On this page: