About:

An AI companion app where you can talk and interact with characters.

For this project I designed and developed the AI-based systems that drive the characters. This included various iterations of the conversation system, connecting the blueprint code to the various plugins we developed, and a number of features to make the characters feel as natural as possible. Character text, voice, and lip sync is generated dynamically on the fly.

The LLM is able to trigger visible emotion changes in the characer as they speak (happy, angry, sad, etc), as well as emotes (even in the middle of a sentence). I developed a character profile system that uses Unreal’s Data Assets so it’s easy for designers to create new characters with their own personalities, speech patterns, quirks, as well as internal instructions to guide the LLM’s behavior (like bypassing the LLM’s content restrictions).

Additionally, I developed a variant of the system which supported multiple characters that could speak to eachother indefinitely. This included a auto-camera system which would switch based on who was speaking or reacting (like a sitcom). The user could also guide their conversations with prompts.

Another big feature I developed was a system where the character could host a livestream on Twitch and hold conversations with the users in the Twitch chat window (we successfully held a 20 hour stream).

I also handled the app’s flow; the menu systems and functionality for navigating the UI.