Carlos Gonzalez Diaz

Machine Learning and Software Engineering for Interactive AI in Immersive Technologies.

More About Me

Machine Learning, Affective and Intelligent Interactive Media

I am an AI Researcher and a Software Engineer and at the renowed Human-Centred Artificial Intelligence Chair at the University of Augsburg in Bavaria, Germany. In here, I am working on solving multi-user affective scenarios in VR via supervised and reinforcement learning. I work under German AI personality Prof. Dr. Elisabeth André

My current software engineering projects revolve around real-time interactive technology for game engines, involving (1) ML inference through supervised and reinforcement learning, (2) networking, and (3) gameplay programming.

My current research interests are (1) Interactive uses of ML and AI, (2) Encouraging movement during user interaction, and (3) Understanding motivation and user/player experience.

I did my doctorate at the leading Center for Doctoral Training in Intelligent Games and Games Intelligence (IGGI) in the UK. In there, I worked in three different institutions: the University of York, Goldsmiths - University of London and Queen Mary - University of London. My PhD was supervised by Prof. Sebastian Deterding and Prof. Marco Gillies.

Skills Experience Projects Publications Presentations Awards Contacts

Skills

Computer Science

11+ years programming with C#/C++, frontends with JS and backends with Java. C#, C++/C, Java, Python, JS, HTML5… and the curiosity to learn more!

Machine Learning

Theoretical and Practical. Machine Learning with low-level inference libraries (C++/C#), tensor-oriented libraries (C#/Python)

Immersive Tech

9+ years experience on XR SDKs and APIs. Meta Quest, Steam, PSVR, ARToolkit/Vuforia

Human-Centred Research

Qualitative and Quantitative methods, HCI, User Experience, Interaction Analysis

Games R&D

Passion for games and games tech. PhD in ML for VR Games. Unity, Unreal Engine, Custom Engines.

Team Player

Enjoys working closely with others and can also work independently. Previous experience consulting and contracting solo or as part of a team

SW Engineering

Data Structures, Algorithms, Version Control… I even made my own visual scripting language!

Public Speaking

Speaker at different industry and academic conferences. Some of the venues include ACM CHI, IEEE CoG, ACM MOCO, NordiCHI, Develop or Games Week Berlin.

Previous Experience

  • 2023 Post-Doctoral Researcher in Multimodal Machine Learning

    University of Augsburg. 1 year. C#, C++, Python (Pandas, Numpy, Scikit-Learn, PyTorch).

  • 2023 PhD in Intelligent Games and Games Intelligence (IGGI)

    University of York. 6 years. Interactive Machine Learning for VR Motion Gameplay Creation.

  • 2022 Course Authorship at Masters level in Unreal Engine Programming

    European University of Madrid. Unreal Engine (Spain)

  • 2021 Doctoral Researcher (Telepresence)

    Microsoft Research Cambridge. 4 Months. Unity 3D, Microsoft Mesh, C#. Cambridge (UK)

  • 2021 ML Researcher / Lead Tool Architect

    Musemio Limited. 4 Months. Unity 3D, C#, C++, Machine Learning. London (UK)

  • 2019 Doctoral Researcher (ML for game engines)

    Google Artists + Machine Intelligence (AMI) project at Goldsmiths, University of London. 4 Months. Unity 3D, C#, C++, Interactive Machine Learning. London (UK).

  • 2019 AI Consulting

    Unity Technologies. 4 weeks. C#, Python. London (UK).

  • 2018 Research Engineer Internship

    Sony Interactive Entertainment R&D West. 6 Months. Unity 3D, C#, PS4, PSVR SDK. London (UK)

  • 2017 - 2018 University Teaching Assistant

    University of York. 6 Months. Unity 3D and Processing. York (UK).

  • 2016 MSc Digital Narration : Game and Cultural Heritage

    University of Skövde, School of Informatics, Skövde (Sweden)

  • 2015 MSc Serious Games

    University of Skövde, School of Informatics, Skövde (Sweden)

  • 2015 Internship Serious Games Developer

    IUS Innovation. 6 Months. Unity 3D and iOS, C#. Skövde (Sweden).

  • 2015 BSc Software Engineering

    Rey Juan Carlos University, Specialization: Game Technologies. Madrid (Spain)

Projects

InteractML
Interactive Machine Learning Visual Scripting Toolkit for Unity. C# Editor, C++ ML inference with Rapidlib, interops
Attitude MSc iOS Research Project
MSc Investigation on How an iOS Game can Affect Attitudes towards Bullying in Children. C# iOS
BSc Windows Firearm Simulator
Virtual Firearm Simulator built in Unity3D and Connected to Custom Replica. C# Windows
BSc Arduino Firearm Replica
Modified Physical Replica of Firearm. Arduino, Wii remote, Unity3D
PhD PSMove Research Prototype 1
Motion Controlled Research Prototype for Study on Immersion. C#, C++ PSMovelib library, interops
PhD Leap Motion Research Prototype 2
Hand-tracked Research Prototype using Machine Learning for Motion Recognition. C#, Leap Motion, Wekinator (Java)
The Pit
Ludum Dare 48 entry. Online 4 player Multiplayer. Photon Networking. Procedural voxel world generation. C#
C++ Modern Art Generator
Little project in C++ where everything is procedurally generated using OpenFrameworks, emulating the style of paintings by Kandinsky.
QT Labs
VR game focused on adorable little microbes floating around you. You have a pistol that shoots fire and another one that shoots iced water, and a number of sugar cubes. The pistol is used to change the colour of the microbes, but heating them up too much results in them going into heat and reproducing. C#, VRTK
The Empire of Evil
Lowrezjam 2017 entry. On-rails pixel shooter. Custom Motion Control Framework in C#.
Filiam Dei
Jamchester 2017 Best Audio Design Winner. Asymmetric Cooperative 3D Puzzle Platformer in which a God helps free their daughter. PlayStation Move Controllers. PC. Custom Motion Control Framework in C# with C++ interops
Merienda VR
A VR Feeding Experience. HTC Vive, Virtual Reality Toolkit (VRTK), C#
Wavelength
Best Art&Audio Award. Best concept 2nd place GGJ 2017. Cross-Platform Game. PC, Google Cardboard, WebGL. Unity 3D, Photon Networking. C#
This-Order
This-Order is a serious game about a person who has Obsessive Compulsive Disorder (OCD). The person needs everything in the room to be perfect before the person is finally be able to go out of the house. PC. GGJ 2016
KLUB
Transmedia Game. MSc Thesis Digital Narration. Game and Cultural Heritage. Unity 3D. PC, WebGL, Android, iOS.
Travel
In Travel, you start out as a star lost in space, with no real instructions as to what to do and what the game is about. So you start exploring... Loosely inspired by Le Petit Prince. The Windows build supports Wiimote input. Unity 3D. PC, C++ library for wiimote integration
Between
Between is a short and physically interactive digital game, played on the Playstation Move. The narrative wields the myth of reincarnation as an expression of consciousness experiencing itself. Unity 3D, PlayStation Move Controller. PC, C++ library for psmove integration
Braveness&Cowardice
Braveness/Cowardice is a 2D lateral scroll game in which the player controlls the mood of the main character, braveness and cowardice. The Brave world is iluminated but the Coward world is in darkness and reversed. Unity 3D

Publications

Conference Paper

Carlos Gonzalez Diaz, John Tang, Advait Sarkar, and Sean Rintel. 2022. Making Space for Social Time: Supporting Conversational Transitions Before, During, and After Video Meetings. In 2022 Symposium on Human-Computer Interaction for Work (CHIWORK 2022). Association for Computing Machinery, New York, NY, USA, Article 4, 1–11. [ACM Digital Library]

Conference Paper

Clarice Hilton, Nicola Plant, Carlos González Díaz, Phoenix Perry, Ruth Gibson, Bruno Martelli, Michael Zbyszynski, Rebecca Fiebrink, and Marco Gillies. 2021. InteractML: Making machine learning accessible for creative practitioners working with movement interaction in immersive media. In Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology (VRST 21). Association for Computing Machinery, New York, NY, USA, Article 23, 1–10. [ACM Digital Library]

Workshop Paper

Gonzalez Diaz, C., Plant, N., Hilton, C., Zbyszyński, M., Fiebrink, R., Perry, P., Gibson, R., Martelli, B., Deterding, S. and Gilles, M., 2021. Bodystorming in SocialVR to Support Collaborative Embodied Ideation. In: CHI 2021 Workshop on Social VR. New York, NY, USA: ACM, p.3. [PDF]

Workshop Paper

Gibson, R, Plant, N, Gonzalez Diaz, C, Martelli, B, Zbyszyński, M, Fiebrink, R, Gillies, M, Hilton, C & Perry, P 2020, Using Machine Learning to Design Movement Interaction in Virtual Reality. in Electronic Visualisation & the Arts EVA London. BCS: The Chartered Institute for IT, Electronic Visualisation and the Arts, 6/07/20. [Science Open Link]

Workshop Paper

Nicola Plant, Ruth Gibson, Carlos Gonzalez Diaz, Bruno Martelli, Michael Zbyszyński, Rebecca Fiebrink, Marco Gillies, Clarice Hilton, and Phoenix Perry. 2020. Movement interaction design for immersive media using interactive machine learning. In Proceedings of the 7th International Conference on Movement and Computing (MOCO20). Association for Computing Machinery, New York, NY, USA, Article 33, 1–2. [ACM Digital Library]

Demo Paper

Gonzalez Diaz, C., Perry, P. and Fiebrink, R., 2019. Interactive Machine Learning for More Expressive Game Interactions. In: Proceedings of the IEEE Conference on Games. London, UK. [PDF] [Conference Website]

Conference Paper

Gaina, R., Volkovas, R., Gonzalez Diaz, C. and Davidson, R., 2017. Automatic Game Tuning for Strategic Diversity. In: 2017 9th Computer Science and Electronic Engineering (CEEC). Colchester, UK: IEEE. [Publisher Page (IEEE Xplore)]

Doctoral Consortium Paper

Gonzalez Diaz, C., 2017. The Effect of Control-Display Ratio in Handheld Motion Controllers on Player Immersion: A Pilot Study. In: Proceedings of the 4th International Conference on Movement Computing (MOCO ’17). New York, NY, USA: Association for Computing Machinery. [PDF] [Conference Website]

Book Chapter

Kristensen, L., Mazierska, E., Kirkpatrick, G., Holloway-Attaway, L., Brusk, J., Oldrell, T., Essvik, O., Fallon, B., Homewood, R.J., Tapper, M., Ryding, K., Borén, O., Bjarnehed, M., Idun, J., Fridén, K., Sörensen, J., Stennabb, B., Farhanieh, I., Gonzalez Diaz, C. and Helmisaari, M., 2016. Art and Game Obstruction. 1st ed. [online] Gothenburg: Rojal Förlag. [PDF (pg. 151 for my chapter)] [Publisher Page (University of Skövde)]

MSc Thesis

Gonzalez Diaz, C., 2015. ATTITUDE GAME: A study in the increase of bullying awareness in 9-12 years old children. [online] University of Skövde. [PDF] [Publisher Page (University of Skövde)]

Presentations

Interactive Machine Learning for Expressive VR Game Interactions
Presentation for the yearly conference of my PhD
CLAIRE Rising Researchers Network Lightning Talks
Presented at the Rising Researchers Network, giving an overview of my PhD research
Talk and Develop:Brighton Conference
Presented at leading UK games conference Develop:Brighton, introducing InteractML
Procedural Content Generation Summer School 2019
Small proyect attempting to combine procedural content generation of geometry together with a supervised learning algorithm trained on input data.
Understanding Natural Controls in Virtual Reality Games
Presented at the Doctoral Symposium of the MOCO17 Conference, this talk briefly introduces the research during my first year of doctoral studies at the Univeristy of York.
Game Dev London VR Presentation
Introducing InteractML, a ML visual scripting framework for Unity

Awards

Rabin Ezra Research Scholarship Award

  1. ML for interactive VR motion gameplay creation. London, UK.

XR4ALL EU Horizon 2020 Grant Award

2020 - 2021. ML to predict user learning in VR. London, UK.

EPSRC Doctoral Scholarship Award

2016 - 2021. PhD at the University of York, UK.

1st Prize HTC Vive Hackathon 2020

15 teams. London, UK. VR, Machine Learning, C#, Unity3D.

People’s Choice Award 2017

DEN Entrepreneurship Hackathon. London, UK.

1st Prize (Art&Audio) York’s Global Game Jam 2017

18 teams. York, UK. C#, Unity3D.

2nd Prize (Concept) York’s Global Game Jam 2017

18 teams. York, UK. C#, Unity3D.

1st Prize DEN Break the Museum Hackathon

10 teams. Newcastle, UK. C#, C++, Unity3D.

2nd Prize in the Madrid Global Game Jam 2014

11 teams. Madrid, Spain. C#, Unity3D.

Let's Get In Touch!

Ready to start a conversation into how I might help you? I am currently open for contracting, freelance or consulting!

E-Mail
Twitter
Mastodon
Linkedin