Using ESP-8266/32 as a wireless HID device. part_1

HID or Human Interface Devices are the peripherals used to get human input. Examples include your mouse, keyboards , game controllers etc.

Now I wanted to control my systems over the wifi. SSH or telnet would't suffice as I NEEDED mouse input. (Basically wanted to make a wifi enabled scroll wheel). This problem gave birth to this project.

---

Basic idea -

• Set up a local server to get requests from the local network.

• Use esp to send get request to the server. Request may include specific data (like mouse coordinates or keystrokes) or just Boolean data to tell system to perform a specific function.

• Every time a server gets a request, perform the specific function by using appropriate system level commands to send hid events to system.

• Reduce the input lag as much as possible by maintaining a constant connection between server and the esp.

1. Setting UP the server :

To server the requests I decided to use FLASK - A lightweight-micro web framework written in python. I selected it for two reasons - 1) I have used it before(duh!) , 2) Its laughably simple to get up and running , 3) Its based on python so I can use plethora of libraries to interface with the I/O.

Here's the basic setup ->

import flask

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/', methods=['GET'])
def home():
    return "<h1>SUP!<h1>"

app.run(host = '0.0.0.0')

Only important thing to note here is that we are running it at 0.0.0.0 instead of localhost to make it visible to the whole network network.

2. Generating hard-coded HID events :

To send keystrokes or mouse input we are going to use pyautogui . It an amazing cross platform library which hides all the complexity in interacting with different de(s) for different os(es). To quote from its official github page :

On Windows, PyAutoGUI accesses the Windows API (also called the WinAPI or win32 API) through the built-in ctypes module. 
On macOS, PyAutoGUI uses the rubicon-objc module to access the Cocoa API.
On Linux, PyAutoGUI uses the Xlib module to access the X11 or X Window System.

Install it on your system using :

pip install pyautogui
#for linux only
sudo apt-get install python3-xlib

Try it by running the following on your system :

import flask
import pyautogui 

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/up', methods=['GET'])
def home():
    pyautogui.scroll(-20) 
    return "<h1>Scrolling Up<h1>"

app.run(host = '0.0.0.0')

After running this script(in su mode!) go to your pc's local ip address on your default port (in my case it 192.168.1.78:8888) and see if reloading the page sends scroll down command to the system.

If it works good !. If it doesn't go here to troubleshoot.

3. Sending get requests using esp :

To do this first you have to connect your esp module to your local wireless network, detect hardware input and send send the get request to the server's address.

Here's the code :

#include <ESP8266WiFi.h>

#include <ESP8266HTTPClient.h>

const char * ssid = "ssid";
const char * password = "pass";

void setup() {

  Serial.begin(115200);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {

    delay(1000);
    Serial.print("Connecting..");
  }

}

void loop() {

  if (WiFi.status() == WL_CONNECTED) { //check connection

    HTTPClient http;
    if (digitalRead(BUTTON) == LOW) {
      http.begin("DESTINATION"); //request destination
      int httpCode = http.GET();
      if (httpCode > 0) {
        String payload = http.getString();
        Serial.println(payload); //response
      }
      http.end();
    }
  }
}

Replace "ssid" , "password" with your wifi details. Replace "DESTINATION" with the address you entered in your browser in the last step. Replace (BUTTON) with the pin number your connected your trigger to.

4. Sending KeyStrokes/Dynamic data :

To receive dynamic in flask change -

@app.route('/', methods=['GET'])

to -

@app.route('/<stroke>', methods=['GET'])

To send keystrokes use :

pyautogui.typewrite(stroke, interval=0)

Refer to this document to get into details.

Here's the final code :

import flask
import pyautogui 

app = flask.Flask(__name__)
app.config["DEBUG"] = True

@app.route('/<stroke>', methods=['GET'])
def home():
    pyautogui.typewrite(stroke, interval=0)
    return "<h1>done<h1>"

app.run(host = '0.0.0.0')

Now your can send keystokes to your system by editing the url after you ip (eg: http://192.168.1.78:8888/hello)

---

This is it for part one. In part two i will be assembling this shit into a neatly structured library with all the hid options from joysticks to scroll wheels, and game controllers. I will also try to reduce the lag/latency between hardware button presses and actual input by using something better to send requests .