The Alien Style of Deep Learning Generative Design

Source: http://www.arup.com/news/2015_05_may/11_may_3d_makeover_for_hyper-efficient_metalwork

What happens when you have Deep Learning begin to generate your designs? The commons misconception would be that a machine’s design would look ‘mechanical’ or ‘logical’. However, what we seem to be finding is that they look very organic, in fact they look organic or like an alien biology. Take a look at some of these fascinating designs.
The photo above design is described as follows:

“This is not only an exciting development for the construction sector, but many other industries as well. In the case of this particular piece, the height is approximately half that of one designed for traditional production methods, while the direct weight reduction per node is 75%. On a construction project that means we could be looking at an overall weight reduction of the total structure of more than 40%. But the really exciting part is that this technique can potentially be applied to any industry that uses complex, high quality, metal products.”
— Salomé Galjaard, Team Leader at Arup

This is a car frame that is designed by a generative algorithm:

The design on the right of an antenna is twice more effective that that on the left:

Source: http://hub-apac.insight.com/h/i/189108757-cad-is-a-lie-generative-design-to-the-rescue/166669

A Lightweight bike stem generated by an algorithm:

Source: http://www.instructables.com/id/How-to-Design-a-Bike-Stem-in-Dreamcatcher/

A lightweight engine block:

Source: http://inthefold.autodesk.com/in_the_fold/2015/07/autodesk-within-generative-design-optimized-for-3d-printing.html

Many of these designs come from Autodesk’s DreamCatcher research.

The Dreamcatcher system allows designers to input specific design objectives, including functional requirements, material type, manufacturing method, performance criteria, and cost restrictions. Loaded with design requirements, the system then searches a procedurally synthesized design space to evaluate a vast number of generated designs for satisfying the design requirements.

Generative designs also exist not only in the physical world but also in the design of neural networks themselves:

These Long Short Term Memory (LSTM) are designed by an algorithm and shown to be more effective than the conventional LSTM (Note: These are Neural Networks designed with memory elements). These are generative neural architectures, machines that learn to learn, more like meta meta-models. Learning apparently is not uniform and I highly suspect that meta-level reasoning is a primary mechanism in learning and that is reflected by its biological manifestation. After all, isn’t learning enhanced by diversity as well as adaptability? The same ingredients to biological survival?

What is surprising is that these designs do not exist for the sake of style. Rather, these designs are actually the optimal solutions to multiple competing design requirements. Why do they look organic or biological? Is there some underlying fundamental principle that exists in biological systems that leads to this? Why aren’t the solutions sparse, but rather complex?
Even a more deeper question is, if these were the optimal designs, then why don’t inanimate objects look like this? Inanimate objects that are complex tend to have a fractal style:

Credit: http://www.fhshh.com/the-discovery-of-fractals-extraordinary-examples-of-geometric-complexity-of-mother-nature.html

The self-similar repeating patterns that we see in crystals and coastlines ,despite looking complex, certainly have a style that is different from organic or biological styles. Deep Learning clearly has similar capabilities as biological systems. I suspect that this difference originates from the difference in computational machinery that generates these. It indeed is fascinating that the style of these generated objects are a reflection of the process of its originator.

Self-similarity are created through uniform processes that occur in multiple scales. In contrast, biological processes also do occur in multiple scales, diversity rather uniformity is encouraged. Humans for example, exhibit a five fold symmetry. We see similarity occurring at multiple scales, meaning we see interesting features different coarse levels of granularity, however we also see diversity. This is why designs by Deep Learning systems look biological, the construction process use similar mechanisms.
A recent study “Inside an AI brain — What does machine look like?” Reveals similar biological like complexity in Deep Learning systems. Here is a visualization of a ResNet-50 Deep Learning network:

Credit: https://www.graphcore.ai/blog/what-does-machine-learning-look-like

It indeed is fascinating that the complexity if very different from the complexity we find in chaos theory.

One misconception is that Artificial Neural Networks or Deep Learning are biologically inspired. This is NOT true. It may have thought as being true back in the 1950’s when the Perceptron was first introduced. However, at best, Deep Learning employs a cartoonish version of a neuron and the newer architectures consist of computational elements that are very different from how a biological neuron would function. Nevertheless, it is indeed surprising though, that despite being very different from biology, that the resulting observable behavior appear to be very similar between both. Clearly, there is a set of fundamental processes underlying all of this that is both shared by biology and Deep Learning. That fundamental processes is what we are all seeking to discover.

One final thing though, just because DL exhibits some behavior that appears to be biological, it still is a far cry away from something that is intelligent. However, we at least know of one instance of an intelligence, and that happens to be biological.

Originally published on Medium: https://medium.com/intuitionmachine/the-alien-look-of-deep-learning-generative-design-5c5f871f7d10