Drones

Autonomous drones represent a new breed of mobile computing system. Compared to smartphones and connected cars that only opportunistically sense or communicate, drones allow motion control to become part of the application logic. The efficiency of their movements is largely dictated by the low-level control enabling their autonomous operation based on high-level inputs.

The largest benefit of human-drone interaction is that drones can precisely transport user interfaces to any point and orientation in 3D space, making drones a new medium for both input and output. Not only can hardware for sensing user input be positioned at any convenient position in 3D space, but actuators, displays, and other output devices can also be placed in any position around the user.

The most basic feature of human-drone interaction is a user or system telling a drone to change its position. With more intelligent systems, how one controls the drone changes. There are four different ways of controlling a drone's position: direct position control, absolute position control, relative position control, task-oriented control.

Direct position control: In the traditional form, the user directly controls the flight path using an interface (e.g., a joystick). The drone does not need to carry additional location sensors, as the user is flying it within their line of sight (LoS) or in first-person view (FPV).

Absolute position control: With more advanced systems, the user can tell the drone where to go just by defining location targets that the drone flies to autonomously using an on-board location processor. This can be, for example, GPS or a more complex camera-based optical location tracking system (e.g., SLAM).

Relative position control: With even more tracking capabilities, drones can be positioned relatively to a tracked object. A user can therefore issue commands to a drone that are relative to a person, position, or object. A user could tell the drone to "follow me at a distance of 10 meters" or "stay next to this tree."

Task-oriented control:. In the most advanced setting, drones can be controlled by giving them a task that the drone needs to achieve. Such a task could be "build a wall out of these plastic cubes." The drone would then continuously pick up cubes and place them at the specified position in order to build a wall. Another scenario of task-oriented control is when multiple drones work together to form a display in the sky, where every drone represents a flying pixel (e.g., the SPAXELS project)