Neural Scene Representation and Neural Rendering

Seminar – Fall Semester 2024

Instructor: Lingjie Liu

TAs: Chuhao Chen, Xuyi Meng

Organization | Content | Format | Resources


NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis ECCV, 2020	NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction NeurIPS, 2021

Format: Summary

Each week:

Before the seminar:
- Read the papers of the week and think about them.
- Submit two questions (one question per paper/bundle) for discussion before the seminar to this Google Form. This is important – your contribution will be marked. The deadline for submitting questions is one hour before each class session (so Monday 2:30 PM and Wednesday 2:30 PM).
- Homework drop policy: you can submit paper questions late up to two occurences without any penalty.
During the seminar:
- Overview (10 minutes)
  - The instructor or TAs give a brief introduction on the topic.
- 2x Presentations (each 25 minutes, 25% of grade):
  - Two pre-assigned participants present the papers of their choice.
  - ~5 minutes on motivation, background and related work.
  - ~20 minutes of presentation of one paper.
- Discussion and Feedback (30 minutes, 25% of grade across weeks):
  - One participant is assigned at random at the beginning of the seminar to lead the discussion. Everyone leads the discussion at least once in the seminar series.
  - The discussion leader receives a digest of the submitted questions just before the seminar.
  - The discussion leader raises questions appropriately throughout the discussion, covers future work aspects, and finally provides a summary of the strengths and weaknesses of the techniques and of the discipline.
  - The students provide feedback to the presenting student on their presentation with respect to what has worked well, and what could be improved and how

Format: Details

Part 1: Presentation (50% of grade)

Every participant will make two presentations in the seminar series. In each presentation, the participant will perform a detailed study of one paper or a bundle of several papers. If a bundle is chosen, the participant only needs to present one of the papers in the bundle according to their preference, but they are encouraged to discuss the connections between the papers in the bundle. The participant will present the main ideas of the papers in a presentation of approximately 25 minutes, and it will often not be possible to discuss all results presented therein. Work on the topic usually requires reading and understanding the papers, as well as acquiring background information necessary to understand the topic. For extra background knowledge, some of the papers and books referenced in the papers may have to be consulted.

If you use formulas, make sure that all symbols are introduced properly. Similarly, make sure that figures are labelled correctly and that new terminology is introduced appropriately. You may assume that your audience is familiar with topics that have been presented in class earlier, and have read the papers for your week. It is very important for a good presentation to find a balance between overview and detail. Most importantly, the general theme of your topic should become clear from your presentation. Your goal is that the other participants will have understood your critical analysis of the two papers after your presentation. Finally, prepare your presentation on time and plan to spend some time on practicing your talk. Our feedback afterwards will help you improve future presentations.

Please select and register for the two (bundles of) papers you would like to present in this Google Docs. Ensure you are fully prepared one class before your scheduled class for presentation. Upload your slides to this Google Folder at least one hour before the class prior to your assigned class for presentation. This is important in case of an emergency requiring us to reschedule your talk. For example, if you’re presenting on Monday, upload your slides by the previous Wednesday at 2:30 PM. If presenting on Wednesday, upload by the previous Monday at 2:30 PM.

The grading criteria for the presentation are as follows:

Form (30%): Time / Verbal speed & clarity / Body posture / Engagement with audience
Content (50%): Structure or storyline / Main points / Paper connections / Valid conclusions
Answers (20%): Good answers to questions / Knowledgeable

Part 2: Discussion (25% of grade across weeks)

The presentation will be followed by a discussion among the seminar participants. This discussion will be chaired by the discussion leader, who will be one of the participants. Before the presentation, one participant will be chosen at random to be the discussion leader. Their job is to direct discussion such that the strengths and weaknesses of the techniques and of the field are discussed, that open questions and future directions of the work are discussed, and that questions of other participants are fielded.

Before each class, every participant will submit two questions (one question per paper/bundle) that they would like answered on the upcoming class's papers. These questions will be submitted to this Google Form before the seminar. The questions will be given to the discussion leader, and it is their job to integrate them into the discussion. For participants, these questions act as evidence of participation, and your overall participation score will be based on the quality of your questions and your involvement in the discussion.

Participants are expected to read every paper in preparation for the upcoming presentations. Initially, we will teach you how to read an academic paper, and each week at least one expert member of staff will be available to answer questions. We expect students to actively engage in discussions for further understanding of the presented material. We aim for a creative atmosphere – ideas developed during the seminar work might lead to Masters thesis projects.

The grading criteria for the discussion are as follows:

Moderation (30%): Integrates questions well / Pushes forward discussion / Good summary / Strengths and weaknesses of paper
Questions (70%): The questions submmited before the class (two per class). Graded by completion

Part 3: Other Activity Participation (25% of grade)

In addition to the in-class presentation and discussion, we will have three special lectures at the end of the semester.

In the practice lecture, we will introduce and demonstrate wonderful tools (e.g. Nerfstudio) for you, and you are welcome to try them to create your own neural scene representation and neural rendering results at or after the class.
In the followed lecture, we will have a discussion on favorite papers. Each participant has 5 minutes to present their favorite paper.
In the last lecture, we will have a summary of the seminar series and a brainstorming session for future research ideas.

The grading criteria for the special lectures are as follows:

Practice Lecture (30%): Attentively listening / Engagement of coding
Discussion on your favourite papers (35%): Clear presentation and explanation
Brainstorming (35%): Contribution of ideas / Active participation