Testing with AI Meetup (In-Person)
Details
Hey SF JUG folks, we are quite excited about our next in-person meetup in San Francisco with 3 great talks on AI and Testing, two hot topics in the industry today. Check out the agenda and the more info on the talks below.
AGENDA
5-5:30pm Networking
5:30 - 6 Talk #1: DragonCrawl: Shifting mobile testing to the left with AI By Juan Marcano (Staff ML Engineer at Uber) and Sowjanya Puligadda (Eng Manager at Uber)
6:15-6:45 Talk #2: Codeless Unit Testing with Sapient AI By Rishi Singh (Founder and CEO of Sapient.ai, former CTO & Co-founder at Harness)
7-7:30 Talk #3: Using your AI dev system to further automate test creation and maintenance by Ty Dunn, Co-founder & CEO of Continue
7:30-8pm Networking
DragonCrawl: Shifting mobile testing to the left with AI
By Juan Marcano (Staff ML Engineer at Uber) and Sowjanya Puligadda (Eng Manager at Uber)
Mobile testing remains an unresolved challenge especially at our scale, encompassing thousands of developers and over 3,000 simultaneous experiments. Manual testing is usually carried out, but with high overhead, and cannot be done extensively for every minor code alteration. Furthermore, the substantial maintenance costs of these tests significantly hinder their adaptability and reusability across diverse cities and languages (imagine having to hire manual testers or mobile engineers for the 50+ languages that we operate in!), which makes it really difficult for us to efficiently scale testing and ensure Uber operates with high quality globally.
To solve these problems, we created DragonCrawl, a system that uses large language models (LLMs) to execute mobile tests with the intuition of a human. It decides what actions to take based on the screen it sees and its goals, and independently adapts to UI changes, just like a real human would. In this talk, we will deep dive into our architecture, challenges, and results. We will close by touching a little on what is in store for DragonCrawl.
Codeless Unit Testing with Sapient AI
By Rishi Singh (Founder and CEO of Sapient.ai, former CTO & Co-founder at Harness)
Unit testing, as commonly practiced, is fraught with inefficiencies, often requiring developers to write significantly more test code than functional code. This results in increased code sprawl, burdensome maintenance, and reduced development velocity, leading many software teams to neglect unit testing or fail to achieve the industry-standard 70% coverage. Recent AI assistants have exacerbated this issue by generating more code at a faster pace, further contributing to code sprawl. Rishi shares how they developed Sapient Codeless, an approach that eliminates the need to maintain unit tests as code. It focuses on the core principles of testing—inputs and outputs—abstracting the complexity of test maintenance entirely from the developer's hands. This no-code approach allows developers to automatically generate unit tests by simply selecting the methods to test, with AI handling the underlying logic.
Using your AI dev system to further automate test creation and maintenance
Ty Dunn, Co-founder & CEO of Continue
Until now, the primary use cases for most AI software development systems have been to autocomplete lines of code and answer questions that used to be directed to Stack Overflow, but organizations like Meta and Uber have been exploring using LLMs in other parts of the software development lifecycle too. From fine-tuning an LLM to create tests in your style to setting up an async workflow to improve test suites en-masse, this talk will explore the different ways we might use and reuse AI dev system components for further automating test creation and maintenance.