Maschinelles Lernen für Geometrisches Design

Syllabus

Digital models are represented by polygonal meshes that can be edited by moving, removing, or adding vertices into the mesh.  Many models require considerable time and artistic effort to build.  With the help of modern algorithms, it is possible to simplify and in some cases even improve the geometry design process and automate certain tasks.  These algorithms include the generation of new models based on a given set of examples and the identification of related models by learning their visual similarities.  Others provide more intuitive design handles; whereas, traditionally, the vertices of a mesh were the basic design handle for geometric modeling, innovations in the last few years have introduced novel ways of modifying geometric models that respect the integrity of the model.

Deep learning is the new buzzword trending in various research venues today, in particular among researchers studying these issues related to geometry design.  Deep learning encompasses the various methods for generating meaningful representations of data and automatically extracting features from that data with as little human supervision as possible.  In this seminar, deep learning and other related techniques with their applications specifically in geometry design will be introduced.  Below is a list of the specific topics with the corresponding papers that will be studied.

Each student will choose a paper on a first-come-first-serve basis and prepare and hold an hour-long lecture about it. The organizers will also hold lectures on shape space, convolutional neural networks, and probability programming to introduce the audience to the basic concepts underlying this seminar. We will arrange the dates of the lectures at our first meeting.

Geometry Design with Constrained Handles

Instead of manipulating individual control points, an artist or engineer can change a model by using algorithmically generated and intuitively constrained design handles.  For example, by moving the wall of a bookcase, the shelves expand or contract with the movement of the wall.

Yumer, M. E. and Kara, L. B. Co-Constrained Handles for Deformation in Shape Collections. ACM, 2014.
Schulz, A., Shamir, A., Levin, D. I. W., Sitthi-amorn, P., and Matusik, W. Design and Fabrication by Example. ACM, 2014.

Deep Learning of Visual Similarities

Visual similarities among different images or between a shape and an image are attributes particularly useful in geometry design and computer vision, e.g. when visually searching through an image collection, identifying similar objects in scenes, or identifying a 3D object with a 2D image of it. These similarities can be learned using, for example, a convolutional neural network, which is one of the many tools available in deep learning technology.

Bell, S. and Bala, K. Learning visual similarity for product design with convolutional neural networks. ACM, 2015.
Li, Y., Su, H., Qi, C. R., Fish, N., Cohen-Or, D., and Guibas, L. J. Joint Embeddings of Shapes and Images via CNN Image Purification. ACM, 2015.

Probabilistic Programming for Generating Geometries

Having artists design hundreds of different kinds of individual models for use in games or movies would obviously require considerable human resources. Instead, an artist can define a procedural model using a high-level stochastic programming language that describes how to construct e.g. a generic tree. Then, the computer automatically generates different exemplars by interpreting the model and inferring plausible results. A probabilistic programming language (PPL) is a programming language designed to describe probabilistic models and then perform inference in those models. As such, they provide a more expressive and flexible representation compared to graphical models such as Bayesian or Markov networks. Probabilistic Programming provides a generic and powerful methodology in different areas such as machine learning, statistics, robotics, vision, biology, neuroscience, artificial intelligence (AI), and cognitive science. Very recently, they were successfully applied to the domain of content creation and design.

Ritchie, D., Mildenhall, B., Goodman, N. D., and Hanrahan, P. Controlling Procedural Modeling Programs with Stochastically-ordered Sequential Monte Carlo. ACM, 2015.
Ritchie, D., Lin S., Goodman, N. D., and Hanrahan, P. Generating Design Suggestions under Tight Constraints with Gradient-based Probabilistic Programming. Eurographics, 2015.

Course Requirements

  • 35-50 minute presentation (English/Deutsch)
    • If you want feedback, send your slides to your mentor at least one week before your presentation.
    • Using the KIT templates is not mandatory.
  • 2 questions about paper from one of the other groups
  • Written report summarizing presentation including graphics from slides (max. 8 pages)
    • Must didactically expand the content of the slides.
    • Think “slides + transcript.”
    • Due one week after presentation at the latest.
    • Include references and sources of all borrowed graphics.
    • Include declaration of authorship.
    • May be handwritten.
  • Attendance

Grading Criteria

  • Self-sufficiency
    • Was the presentation a translation of the paper?
    • Did the lecturer read off the slides?
    • How many references were used?
    • Did the lecturer supplement his or her own ideas/explanations of the concepts?
  • Comprehension
    • How easy was it to understand the presentation?
    • Were there didactic gaps or false information in the presentation?
  • Structure of the slides
    • Were there spelling/grammar mistakes?
    • How well did the slides convey the core ideas?
  • Presentation 70%; Report 30%

Guidelines

  • DO NOT put too much text on a slide.
  • DO NOT include an overview if it contains unfamiliar concepts/vocabulary.
  • DO NOT create too many slides.
  • DO NOT include complex formulas without deriving and/or explaining them.
  • DO NOT simply read off the slides.
  • DO put more pictures/graphics on a slide.
  • DO keep slides simplistic (details on board).
  • DO start with a bang.
  • DO grab the audience's attention at regular intervals (15 minutes).
  • DO focus on teaching the audience the core ideas of the paper.

Presentation Skills