Augmented Hacking

Catalog description

With recent advances in hardware and software, millions of us now carry unprecedented spatial computing and world-sensing technologies in our pockets. Notably, the latest technologies extend beyond simple recognition into areas of comprehension and semantic understanding. And yet despite these advancements, most augmented reality experiences still closely resemble those from previous decades – 3D content overlaying the camera feed. Few experiences explore the full possibility space enabled by the latest technologies. In this hands-on workshop, we’ll explore a wide range of technologies and techniques for augmenting reality on iOS. Emphasis will be placed on experimentation – exploring the boundaries of what’s possible, embracing unpredictability, and hacking things together in unexpected ways. Topics will include spatial audio, computer vision, scene understanding and semantics, speech recognition and synthesis, music recognition (ShazamKit), haptics, body tracking, and more. Should Apple release its AR headset prior to the course or during the semester, we’ll find ways to incorporate the device into the class. For design and development, we'll use Apple technologies – ARKit, RealityKit and RealityComposer. 3D modeling skills are helpful but NOT required. While we'll cover the basics, students should expect to spend additional time outside of class learning Swift and other related programming concepts. Full-time access to an iOS device (LiDAR-enabled is ideal but not required) and a Mac laptop running the latest operating systems are required.