Deep learning for 3D model synthesis

  • Application end date: 11th February 2018
  • Funding Availability: Funded PhD project (International students only)
  • Department: School of Creative Technologies
  • PhD Supervisor: Dr Dongjoe Shin, Dr Hui Yu

Project code: CCTS4360218 - This project is only open to International (non-EU) students.

Project description

Realistic 3D contents are vital to construct an immersive virtual environment and demand for such 3D models is expected to increase rapidly as AR and VR technologies evolve. However, obtaining a quality 3D model from manual scanning is still a challenging and labour-intensive task. For example, conventional vision-based reconstruction pipeline used in the photogrammetry workflow in Unity generally requires multiple images from different viewing angles to triangulate an initial 3D point cloud. In addition, an albedo texture map should be prepared by fusing colour values, which will be mapped on the approximated surfaces using different image processing and 3D modelling tools.

There are better solutions with active sensors. For example, Microsoft KinectFusion can perform real-time reconstruction. However, this approach needs to track the sensor position to integrate an intermediate depth map into a single reference volume. Thus, the quality of a final shape is highly dependent on the result of vision-based tracking, and the volumetric shape representation internally used in merging multiple depth maps makes us difficult to retrieve a photorealistic texture map.

Another practical issue of the conventional scanning process is that it is often physically not feasible to scan every part of a target object when it is too tall or having complex concave geometry. As a result of this, many practical 3D scans are prone to have undesirable artefacts, such as holes on the final result (i.e. missing photometric and geometric information). In order to address this, many ideas have been suggested but most conventional approaches are focussed on solving one isolated problem (i.e. either shape recovery or texture map correction), instead of solving them as an interconnected problem.

This 3-year studentship will investigate a novel solution to tackle this problem. Particularly, we are interested in obtaining a complete 3D model of a complex non-rigid object (such as garment) using recent deep learning approach. A successful candidate will develop a range of unsupervised learning algorithms particularly developed based on the generative adversarial network. Therefore, a student will learn about how to use standard deep learning frameworks, such as Theano, TesnorFlow, or Caffe to configure new architectures and conduct experiments. Also, this research will require to populate, process, and manage 2D and 3D training data.

As an application of the proposed research, a simple interactive virtual reality application will be developed. For this application, a student can have an opportunity to experiment with garment data obtained from commercial retailers from our collaborators, and to use state-of- the art 3D scanning facility at our collaborator to populate test data if necessary.

Supervisor profiles

Dr Dongjoe Shin

Dr Hui Yu

Admissions criteria

You’ll need a good first degree from an internationally recognised university (depending upon chosen course, minimum second class or equivalent) or a Master’s degree in an appropriate subject. Exceptionally, equivalent professional experience and/or qualifications will be considered. English language proficiency at a minimum of IELTS band 6.5 with no component score below 6.0.  

Enquiries

Informal enquiries are encouraged and can be made to Dr Dongjoe Shin at dongjoe.shin@port.ac.uk (02392 848598) or Dr Hui Yu at Hui.Yu@port.ac.uk (02392 845470).

For administrative and admissions enquiries please contact cci-enquiries@port.ac.uk.

How to Apply

We welcome applications from highly motivated prospective students who are committed to develop outstanding research outcomes. You can apply online at www.port.ac.uk/applyonline. You are required to create an account which gives you the flexibility to save the form, log out and return to it at any time convenient to you.

A link to the online application form and comprehensive guidance notes can be found at www.port.ac.uk/pgapply.

Applications should include:

- Full CV including personal details, qualifications, educational history and, where applicable, any employment or other experience relevant to the application

- Contact details for two referees able to comment on your academic performance

- Research proposal of 1,000 words outlining the main features of a research design you would propose to meet the stated objectives, identifying the challenges this project might present and discussing how the work will build on or challenge existing research in the above field.

- Proof of English language proficiency (for EU/ international students)

When applying, please quote project code: CCTS4360218

Interview date: TBC

Start date: October 2018.

Funding notes

This project is only open to International (non-EU) students.

Eligible applicants will be considered for the Portsmouth Global PhD scholarship scheme.

Research at The School of Creative Technologies

Discover more about our research areas on our webpages.

Visit us

Visit us at a Postgraduate Information Day to discover more about the research programmes we offer. Book your place at www.port.ac.uk/pginfoday