Our world is messy, unpredictable, and packed with rich sensory stimuli. While precise modeling and careful engineering have powered robots to revolutionize the factory floor, getting such robots to robustly manipulate everyday objects in everyday environments is a largely unsolved problem. Consider the seemingly easy task of opening an unlocked door – a feat effortlessly performed by most people, who can adapt to a wide variety of door designs, handle mechanisms, and opening mechanics. Developing a robotic algorithm capable of autonomously solving this task on even a subset of real-world variation presents a formidable challenge. My research aims to bridge this gap and create versatile robotic systems that can interact with our world. Towards this vision, my research program will build new learning-based frameworks that will enable robots to self-collect data, continuously learn from their mistakes, and seamlessly integrate human preferences.