This project was a partnership between Tencent and WebMD, Where we created an interactive medical viewer which is designed to be used for medical consultations for the Tencent Doctor company in Unreal. The application provides anatomical reference for the patient, As well as interactive animations which display different treatments and procedures.
As a R&D Project in the Tencent Medical department, We worked with Epic China to utilize the latest features in Unreal Engine such as pixel streaming and procedural mesh generation.
Working as a Technical Artist/Designer, I was involved in most aspects of the development of the project, Such as breaking down project requirements from management and delegating tasks to the respective departments and also implementing many of the systems and creating assets myself. Work included:
- Building C++ classes which load compressed data from a remote server and present it in Unreal, Including constructing Widget Interfaces, Generating Meshes/Materials and loading 'MoA Timeline' from parametric information.
- Creating assets inside UE such as Master Materials, Widget Blueprints, Structs, Particle Systems and Blueprints which draw mesh data.
- Loading molecular/protein plaintext data and converting them to 3D models with dynamic materials and actor metadata.
- Loading MRI DICOM data and converting it to volume textures to be segmented and rendered in engine.
- Creating tools which assisted with scene construction and sequences/timelines to demonstrate mechanism of action of different procedures/therapies.
Below is a video of the demo version of the Medical Dictionary application. Some of the dynamic functionality was implemented, However this is a sample scene which had a lot of pre-baked assets.
Stills:
